Jul 22 02:19:38.363050 ip-10-0-138-202 systemd[1]: Starting Kubernetes Kubelet...
Jul 22 02:19:38.689616 ip-10-0-138-202 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 02:19:38.689616 ip-10-0-138-202 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 22 02:19:38.689616 ip-10-0-138-202 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 02:19:38.689616 ip-10-0-138-202 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 22 02:19:38.689616 ip-10-0-138-202 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 02:19:38.689616 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.681643    2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 22 02:19:38.689616 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687765    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 02:19:38.689616 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687781    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 02:19:38.689616 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687785    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 02:19:38.689616 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687788    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 02:19:38.689616 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687792    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 02:19:38.689616 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687795    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 02:19:38.689616 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687803    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 02:19:38.689616 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687806    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 02:19:38.704850 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687809    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 02:19:38.704850 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687812    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 02:19:38.704850 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687815    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 02:19:38.704850 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687822    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 02:19:38.704850 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687825    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 02:19:38.704850 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687828    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 02:19:38.704850 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687832    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 02:19:38.704850 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687840    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 02:19:38.704850 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687847    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 02:19:38.704850 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687850    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 02:19:38.704850 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687853    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 02:19:38.704850 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687856    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 02:19:38.704850 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687864    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 02:19:38.704850 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687867    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 02:19:38.704850 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687869    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 02:19:38.704850 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687872    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 02:19:38.704850 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687875    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 02:19:38.704850 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687877    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 02:19:38.704850 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687884    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 02:19:38.704850 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687887    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 02:19:38.708919 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687889    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 02:19:38.708919 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687892    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 02:19:38.708919 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687895    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 02:19:38.708919 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687897    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 02:19:38.708919 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687904    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 02:19:38.708919 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687906    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 02:19:38.708919 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687909    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 02:19:38.708919 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687912    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 02:19:38.708919 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687914    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 02:19:38.708919 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687917    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 02:19:38.708919 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687920    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 02:19:38.708919 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687926    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 02:19:38.708919 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687930    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 02:19:38.708919 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687932    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 02:19:38.708919 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687937    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 02:19:38.708919 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687941    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 02:19:38.708919 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687947    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 02:19:38.708919 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687950    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 02:19:38.708919 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687953    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 02:19:38.711476 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687957    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 02:19:38.711476 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687959    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 02:19:38.711476 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687962    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 02:19:38.711476 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687969    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 02:19:38.711476 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687973    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 02:19:38.711476 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687976    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 02:19:38.711476 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687978    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 02:19:38.711476 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687981    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 02:19:38.711476 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687984    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 02:19:38.711476 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687990    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 02:19:38.711476 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687993    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 02:19:38.711476 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687995    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 02:19:38.711476 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.687997    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 02:19:38.711476 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688000    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 02:19:38.711476 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688003    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 02:19:38.711476 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688006    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 02:19:38.711476 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688012    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 02:19:38.711476 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688015    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 02:19:38.711476 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688018    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 02:19:38.711476 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688021    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 02:19:38.866631 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688024    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 02:19:38.866631 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688026    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 02:19:38.866631 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688033    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 02:19:38.866631 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688036    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 02:19:38.866631 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688039    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 02:19:38.866631 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688041    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 02:19:38.866631 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688044    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 02:19:38.866631 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688046    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 02:19:38.866631 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688049    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 02:19:38.866631 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688055    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 02:19:38.866631 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688058    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 02:19:38.866631 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688063    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 02:19:38.866631 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688068    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 02:19:38.866631 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688071    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 02:19:38.866631 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688078    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 02:19:38.866631 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688081    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 02:19:38.866631 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688084    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 02:19:38.866631 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688087    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 02:19:38.866631 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688090    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 02:19:38.866631 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688096    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 02:19:38.752700 ip-10-0-138-202 systemd[1]: Started Kubernetes Kubelet.
Jul 22 02:19:38.885749 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688099    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 02:19:38.885749 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688102    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 02:19:38.885749 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688105    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 02:19:38.885749 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688107    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 02:19:38.885749 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688111    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 02:19:38.885749 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688115    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 02:19:38.885749 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688118    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 02:19:38.885749 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688120    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 02:19:38.885749 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688127    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 02:19:38.885749 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688130    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 02:19:38.885749 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688133    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 02:19:38.885749 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.688136    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 02:19:38.885749 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689040    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 02:19:38.885749 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689048    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 02:19:38.885749 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689053    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 02:19:38.885749 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689056    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 02:19:38.885749 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689063    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 02:19:38.885749 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689066    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 02:19:38.885749 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689069    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 02:19:38.888228 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689072    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 02:19:38.888228 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689075    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 02:19:38.888228 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689077    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 02:19:38.888228 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689084    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 02:19:38.888228 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689087    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 02:19:38.888228 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689089    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 02:19:38.888228 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689093    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 02:19:38.888228 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689096    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 02:19:38.888228 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689099    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 02:19:38.888228 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689106    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 02:19:38.888228 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689119    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 02:19:38.888228 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689127    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 02:19:38.888228 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689131    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 02:19:38.888228 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689134    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 02:19:38.888228 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689137    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 02:19:38.888228 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689140    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 02:19:38.888228 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689143    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 02:19:38.888228 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689150    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 02:19:38.888228 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689152    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 02:19:38.888228 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689155    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 02:19:38.891510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689157    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 02:19:38.891510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689162    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 02:19:38.891510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689165    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 02:19:38.891510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689171    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 02:19:38.891510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689175    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 02:19:38.891510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689177    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 02:19:38.891510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689180    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 02:19:38.891510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689182    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 02:19:38.891510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689190    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 02:19:38.891510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689194    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 02:19:38.891510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689197    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 02:19:38.891510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689200    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 02:19:38.891510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689203    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 02:19:38.891510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689206    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 02:19:38.891510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689210    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 02:19:38.891510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689216    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 02:19:38.891510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689218    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 02:19:38.891510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689221    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 02:19:38.891510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689224    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 02:19:38.891510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689228    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 02:19:38.894087 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689231    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 02:19:38.894087 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689238    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 02:19:38.894087 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689241    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 02:19:38.894087 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689244    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 02:19:38.894087 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689247    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 02:19:38.894087 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689250    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 02:19:38.894087 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689253    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 02:19:38.894087 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689261    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 02:19:38.894087 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689264    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 02:19:38.894087 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689267    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 02:19:38.894087 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689270    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 02:19:38.894087 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689277    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 02:19:38.894087 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689281    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 02:19:38.894087 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689284    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 02:19:38.894087 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689288    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 02:19:38.894087 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689290    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 02:19:38.894087 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689293    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 02:19:38.894087 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689300    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 02:19:38.894087 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689303    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 02:19:38.895890 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689305    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 02:19:38.895890 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689308    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 02:19:38.895890 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689310    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 02:19:38.895890 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689313    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 02:19:38.895890 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689319    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 02:19:38.895890 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689322    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 02:19:38.895890 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689325    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 02:19:38.895890 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689327    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 02:19:38.895890 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689330    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 02:19:38.895890 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689332    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 02:19:38.895890 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689335    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 02:19:38.895890 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689337    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 02:19:38.895890 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689344    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 02:19:38.895890 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689347    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 02:19:38.895890 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689349    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 02:19:38.895890 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689352    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 02:19:38.895890 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689354    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 02:19:38.895890 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689357    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 02:19:38.895890 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689363    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 02:19:38.895890 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689366    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 02:19:38.898354 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689369    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 02:19:38.898354 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689375    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 02:19:38.898354 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689382    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 02:19:38.898354 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689384    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 02:19:38.898354 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689387    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 02:19:38.898354 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689390    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 02:19:38.898354 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689393    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 02:19:38.898354 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689395    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 02:19:38.898354 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689398    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 02:19:38.898354 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689405    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 02:19:38.898354 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689408    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 02:19:38.898354 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689410    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 02:19:38.898354 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689413    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 02:19:38.898354 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689848    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 02:19:38.898354 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689854    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 02:19:38.898354 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689857    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 02:19:38.898354 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689859    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 02:19:38.898354 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689862    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 02:19:38.898354 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689865    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 02:19:38.898354 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689867    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 02:19:38.900014 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689870    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 02:19:38.900014 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689873    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 02:19:38.900014 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689875    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 02:19:38.900014 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689878    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 02:19:38.900014 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689882    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 02:19:38.900014 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689887    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 02:19:38.900014 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689890    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 02:19:38.900014 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689892    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 02:19:38.900014 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689895    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 02:19:38.900014 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689897    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 02:19:38.900014 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689900    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 02:19:38.900014 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689903    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 02:19:38.900014 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689905    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 02:19:38.900014 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689912    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 02:19:38.900014 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689914    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 02:19:38.900014 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689917    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 02:19:38.900014 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689919    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 02:19:38.900014 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689922    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 02:19:38.900014 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689924    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 02:19:38.902419 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689927    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 02:19:38.902419 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689929    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 02:19:38.902419 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689932    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 02:19:38.902419 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689935    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 02:19:38.902419 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689937    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 02:19:38.902419 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689940    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 02:19:38.902419 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689942    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 02:19:38.902419 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689945    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 02:19:38.902419 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689948    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 02:19:38.902419 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689950    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 02:19:38.902419 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689952    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 02:19:38.902419 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689956    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 02:19:38.902419 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689958    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 02:19:38.902419 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689961    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 02:19:38.902419 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689963    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 02:19:38.902419 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689966    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 02:19:38.902419 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689968    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 02:19:38.902419 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689971    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 02:19:38.902419 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689973    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 02:19:38.902419 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689976    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 02:19:38.904717 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689979    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 02:19:38.904717 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689983    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 02:19:38.904717 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689986    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 02:19:38.904717 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689990    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 02:19:38.904717 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689992    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 02:19:38.904717 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689995    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 02:19:38.904717 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.689999    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 02:19:38.904717 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690002    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 02:19:38.904717 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690004    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 02:19:38.904717 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690006    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 02:19:38.904717 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690009    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 02:19:38.904717 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690012    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 02:19:38.904717 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690015    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 02:19:38.904717 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690017    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 02:19:38.904717 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690020    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 02:19:38.904717 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690023    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 02:19:38.904717 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690025    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 02:19:38.904717 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690029    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 02:19:38.904717 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690032    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 02:19:38.905493 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690034    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 02:19:38.905493 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690037    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 02:19:38.905493 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690040    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 02:19:38.905493 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690043    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 02:19:38.905493 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690045    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 02:19:38.905493 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690047    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 02:19:38.905493 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690050    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 02:19:38.905493 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690052    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 02:19:38.905493 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690055    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 02:19:38.905493 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690057    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 02:19:38.905493 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690060    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 02:19:38.905493 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690062    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 02:19:38.905493 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690065    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 02:19:38.905493 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690067    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 02:19:38.905493 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690070    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 02:19:38.905493 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690073    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 02:19:38.905493 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690075    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 02:19:38.905493 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690078    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 02:19:38.905493 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690080    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 02:19:38.905493 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690084    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 02:19:38.906539 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690087    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 02:19:38.906539 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690089    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 02:19:38.906539 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690092    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 02:19:38.906539 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690094    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 02:19:38.906539 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690097    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 02:19:38.906539 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690099    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 02:19:38.906539 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690102    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 02:19:38.906539 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690105    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 02:19:38.906539 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690107    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 02:19:38.906539 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690109    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 02:19:38.906539 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690115    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 02:19:38.906539 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690118    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 02:19:38.906539 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690122    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 02:19:38.906539 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690125    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 02:19:38.906539 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690193    2581 flags.go:64] FLAG: --address="0.0.0.0"
Jul 22 02:19:38.906539 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690200    2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 22 02:19:38.906539 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690206    2581 flags.go:64] FLAG: --anonymous-auth="true"
Jul 22 02:19:38.906539 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690210    2581 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 22 02:19:38.906539 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690214    2581 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 22 02:19:38.906539 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690217    2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 22 02:19:38.906539 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690221    2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 22 02:19:38.907244 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690225    2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 22 02:19:38.907244 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690228    2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 22 02:19:38.907244 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690231    2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 22 02:19:38.907244 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690235    2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 22 02:19:38.907244 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690238    2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 22 02:19:38.907244 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690242    2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 22 02:19:38.907244 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690244    2581 flags.go:64] FLAG: --cgroup-root=""
Jul 22 02:19:38.907244 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690247    2581 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 22 02:19:38.907244 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690250    2581 flags.go:64] FLAG: --client-ca-file=""
Jul 22 02:19:38.907244 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690253    2581 flags.go:64] FLAG: --cloud-provider="external"
Jul 22 02:19:38.907244 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690257    2581 flags.go:64] FLAG: --cluster-dns="[]"
Jul 22 02:19:38.907244 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690265    2581 flags.go:64] FLAG: --cluster-domain=""
Jul 22 02:19:38.907244 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690268    2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 22 02:19:38.907244 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690271    2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 22 02:19:38.907244 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690274    2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 22 02:19:38.907244 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690277    2581 flags.go:64] FLAG: --container-log-max-files="5"
Jul 22 02:19:38.907244 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690281    2581 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 22 02:19:38.907244 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690284    2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 22 02:19:38.907244 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690287    2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 22 02:19:38.907244 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690291    2581 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 22 02:19:38.907244 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690294    2581 flags.go:64] FLAG: --contention-profiling="false"
Jul 22 02:19:38.907244 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690298    2581 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 22 02:19:38.907244 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690301    2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 22 02:19:38.907244 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690304    2581 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 22 02:19:38.907956 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690307    2581 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 22 02:19:38.907956 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690316    2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 22 02:19:38.907956 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690318    2581 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 22 02:19:38.907956 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690322    2581 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 22 02:19:38.907956 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690324    2581 flags.go:64] FLAG: --enable-load-reader="false"
Jul 22 02:19:38.907956 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690327    2581 flags.go:64] FLAG: --enable-server="true"
Jul 22 02:19:38.907956 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690331    2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 22 02:19:38.907956 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690335    2581 flags.go:64] FLAG: --event-burst="100"
Jul 22 02:19:38.907956 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690338    2581 flags.go:64] FLAG: --event-qps="50"
Jul 22 02:19:38.907956 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690341    2581 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 22 02:19:38.907956 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690344    2581 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 22 02:19:38.907956 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690346    2581 flags.go:64] FLAG: --eviction-hard=""
Jul 22 02:19:38.907956 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690350    2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 22 02:19:38.907956 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690353    2581 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 22 02:19:38.907956 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690356    2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 22 02:19:38.907956 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690359    2581 flags.go:64] FLAG: --eviction-soft=""
Jul 22 02:19:38.907956 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690362    2581 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 22 02:19:38.907956 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690365    2581 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 22 02:19:38.907956 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690369    2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 22 02:19:38.907956 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690372    2581 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 22 02:19:38.907956 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690375    2581 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 22 02:19:38.907956 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690378    2581 flags.go:64] FLAG: --fail-swap-on="true"
Jul 22 02:19:38.907956 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690380    2581 flags.go:64] FLAG: --feature-gates=""
Jul 22 02:19:38.907956 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690384    2581 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 22 02:19:38.907956 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690387    2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 22 02:19:38.908651 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690390    2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 22 02:19:38.908651 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690394    2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 22 02:19:38.908651 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690397    2581 flags.go:64] FLAG: --healthz-port="10248"
Jul 22 02:19:38.908651 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690400    2581 flags.go:64] FLAG: --help="false"
Jul 22 02:19:38.908651 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690404    2581 flags.go:64] FLAG: --hostname-override="ip-10-0-138-202.ec2.internal"
Jul 22 02:19:38.908651 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690407    2581 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 22 02:19:38.908651 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690410    2581 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 22 02:19:38.908651 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690413    2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 22 02:19:38.908651 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690416    2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 22 02:19:38.908651 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690420    2581 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 22 02:19:38.908651 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690423    2581 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 22 02:19:38.908651 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690425    2581 flags.go:64] FLAG: --image-service-endpoint=""
Jul 22 02:19:38.908651 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690428    2581 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 22 02:19:38.908651 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690431    2581 flags.go:64] FLAG: --kube-api-burst="100"
Jul 22 02:19:38.908651 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690434    2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 22 02:19:38.908651 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690437    2581 flags.go:64] FLAG: --kube-api-qps="50"
Jul 22 02:19:38.908651 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690440    2581 flags.go:64] FLAG: --kube-reserved=""
Jul 22 02:19:38.908651 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690443    2581 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 22 02:19:38.908651 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690445    2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 22 02:19:38.908651 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690448    2581 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 22 02:19:38.908651 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690451    2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 22 02:19:38.908651 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690460    2581 flags.go:64] FLAG: --lock-file=""
Jul 22 02:19:38.908651 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690464    2581 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 22 02:19:38.908651 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690467    2581 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 22 02:19:38.909408 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690469    2581 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 22 02:19:38.909408 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690476    2581 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 22 02:19:38.909408 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690479    2581 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 22 02:19:38.909408 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690482    2581 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 22 02:19:38.909408 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690485    2581 flags.go:64] FLAG: --logging-format="text"
Jul 22 02:19:38.909408 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690488    2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 22 02:19:38.909408 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690491    2581 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 22 02:19:38.909408 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690494    2581 flags.go:64] FLAG: --manifest-url=""
Jul 22 02:19:38.909408 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690497    2581 flags.go:64] FLAG: --manifest-url-header=""
Jul 22 02:19:38.909408 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690504    2581 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 22 02:19:38.909408 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690507    2581 flags.go:64] FLAG: --max-open-files="1000000"
Jul 22 02:19:38.909408 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690511    2581 flags.go:64] FLAG: --max-pods="110"
Jul 22 02:19:38.909408 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690516    2581 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 22 02:19:38.909408 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690519    2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 22 02:19:38.909408 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690522    2581 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 22 02:19:38.909408 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690525    2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 22 02:19:38.909408 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690531    2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 22 02:19:38.909408 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690535    2581 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 22 02:19:38.909408 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690538    2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 22 02:19:38.909408 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690545    2581 flags.go:64] FLAG: --node-status-max-images="50"
Jul 22 02:19:38.909408 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690548    2581 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 22 02:19:38.909408 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690555    2581 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 22 02:19:38.909408 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690558    2581 flags.go:64] FLAG: --pod-cidr=""
Jul 22 02:19:38.910037 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690561    2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 22 02:19:38.910037 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690566    2581 flags.go:64] FLAG: --pod-manifest-path=""
Jul 22 02:19:38.910037 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690569    2581 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 22 02:19:38.910037 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690575    2581 flags.go:64] FLAG: --pods-per-core="0"
Jul 22 02:19:38.910037 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690579    2581 flags.go:64] FLAG: --port="10250"
Jul 22 02:19:38.910037 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690587    2581 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 22 02:19:38.910037 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690591    2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e1c5783bd36480ec"
Jul 22 02:19:38.910037 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690594    2581 flags.go:64] FLAG: --qos-reserved=""
Jul 22 02:19:38.910037 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690602    2581 flags.go:64] FLAG: --read-only-port="10255"
Jul 22 02:19:38.910037 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690605    2581 flags.go:64] FLAG: --register-node="true"
Jul 22 02:19:38.910037 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690608    2581 flags.go:64] FLAG: --register-with-taints=""
Jul 22 02:19:38.910037 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690613    2581 flags.go:64] FLAG: --registry-burst="10"
Jul 22 02:19:38.910037 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690620    2581 flags.go:64] FLAG: --registry-qps="5"
Jul 22 02:19:38.910037 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690623    2581 flags.go:64] FLAG: --reserved-cpus=""
Jul 22 02:19:38.910037 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690626    2581 flags.go:64] FLAG: --reserved-memory=""
Jul 22 02:19:38.910037 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690630    2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 22 02:19:38.910037 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690633    2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 22 02:19:38.910037 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690639    2581 flags.go:64] FLAG: --rotate-certificates="false"
Jul 22 02:19:38.910037 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690642    2581 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 22 02:19:38.910037 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690645    2581 flags.go:64] FLAG: --runonce="false"
Jul 22 02:19:38.910037 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690648    2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 22 02:19:38.910037 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690653    2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 22 02:19:38.910037 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690656    2581 flags.go:64] FLAG: --seccomp-default="false"
Jul 22 02:19:38.910037 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690664    2581 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 22 02:19:38.910037 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690667    2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 22 02:19:38.910753 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690670    2581 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 22 02:19:38.910753 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690673    2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 22 02:19:38.910753 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690676    2581 flags.go:64] FLAG: --storage-driver-password="root"
Jul 22 02:19:38.910753 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690683    2581 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 22 02:19:38.910753 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690686    2581 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 22 02:19:38.910753 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690689    2581 flags.go:64] FLAG: --storage-driver-user="root"
Jul 22 02:19:38.910753 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690692    2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 22 02:19:38.910753 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690695    2581 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 22 02:19:38.910753 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690698    2581 flags.go:64] FLAG: --system-cgroups=""
Jul 22 02:19:38.910753 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690705    2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 22 02:19:38.910753 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690711    2581 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 22 02:19:38.910753 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690714    2581 flags.go:64] FLAG: --tls-cert-file=""
Jul 22 02:19:38.910753 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690716    2581 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 22 02:19:38.910753 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690721    2581 flags.go:64] FLAG: --tls-min-version=""
Jul 22 02:19:38.910753 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690728    2581 flags.go:64] FLAG: --tls-private-key-file=""
Jul 22 02:19:38.910753 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690731    2581 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 22 02:19:38.910753 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690750    2581 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 22 02:19:38.910753 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690753    2581 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 22 02:19:38.910753 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690757    2581 flags.go:64] FLAG: --v="2"
Jul 22 02:19:38.910753 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690762    2581 flags.go:64] FLAG: --version="false"
Jul 22 02:19:38.910753 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690766    2581 flags.go:64] FLAG: --vmodule=""
Jul 22 02:19:38.910753 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690773    2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 22 02:19:38.910753 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.690777    2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 22 02:19:38.910753 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690910    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 02:19:38.910753 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690918    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 02:19:38.911510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690921    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 02:19:38.911510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690924    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 02:19:38.911510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690927    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 02:19:38.911510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690932    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 02:19:38.911510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690938    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 02:19:38.911510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690941    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 02:19:38.911510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690944    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 02:19:38.911510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690947    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 02:19:38.911510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690949    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 02:19:38.911510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690952    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 02:19:38.911510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690954    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 02:19:38.911510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690961    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 02:19:38.911510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690964    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 02:19:38.911510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690966    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 02:19:38.911510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690969    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 02:19:38.911510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690971    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 02:19:38.911510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690974    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 02:19:38.911510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690976    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 02:19:38.911510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690983    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 02:19:38.911510 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690985    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 02:19:38.912092 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690988    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 02:19:38.912092 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690992    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 02:19:38.912092 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.690996    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 02:19:38.912092 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691003    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 02:19:38.912092 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691007    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 02:19:38.912092 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691010    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 02:19:38.912092 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691013    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 02:19:38.912092 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691016    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 02:19:38.912092 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691018    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 02:19:38.912092 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691026    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 02:19:38.912092 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691030    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 02:19:38.912092 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691033    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 02:19:38.912092 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691035    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 02:19:38.912092 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691038    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 02:19:38.912092 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691045    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 02:19:38.912092 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691049    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 02:19:38.912092 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691052    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 02:19:38.912092 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691054    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 02:19:38.912589 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691057    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 02:19:38.912589 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691060    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 02:19:38.912589 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691066    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 02:19:38.912589 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691069    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 02:19:38.912589 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691072    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 02:19:38.912589 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691074    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 02:19:38.912589 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691077    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 02:19:38.912589 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691079    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 02:19:38.912589 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691087    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 02:19:38.912589 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691091    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 02:19:38.912589 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691094    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 02:19:38.912589 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691096    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 02:19:38.912589 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691099    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 02:19:38.912589 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691102    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 02:19:38.912589 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691109    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 02:19:38.912589 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691113    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 02:19:38.912589 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691115    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 02:19:38.912589 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691118    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 02:19:38.912589 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691120    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 02:19:38.912589 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691125    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 02:19:38.913168 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691131    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 02:19:38.913168 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691134    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 02:19:38.913168 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691137    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 02:19:38.913168 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691139    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 02:19:38.913168 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691142    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 02:19:38.913168 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691144    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 02:19:38.913168 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691147    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 02:19:38.913168 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691154    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 02:19:38.913168 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691156    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 02:19:38.913168 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691160    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 02:19:38.913168 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691163    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 02:19:38.913168 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691166    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 02:19:38.913168 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691168    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 02:19:38.913168 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691175    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 02:19:38.913168 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691177    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 02:19:38.913168 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691180    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 02:19:38.913168 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691183    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 02:19:38.913168 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691185    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 02:19:38.913168 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691188    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 02:19:38.913168 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691190    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 02:19:38.913786 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691197    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 02:19:38.913786 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691199    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 02:19:38.913786 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691202    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 02:19:38.913786 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691205    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 02:19:38.913786 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691207    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 02:19:38.913786 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691210    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 02:19:38.913786 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691216    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 02:19:38.913786 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691219    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 02:19:38.913786 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691222    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 02:19:38.913786 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691224    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 02:19:38.913786 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691227    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 02:19:38.913786 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691231    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 02:19:38.913786 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691234    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 02:19:38.913786 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691240    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 02:19:38.913786 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691243    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 02:19:38.913786 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691245    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 02:19:38.913786 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691247    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 02:19:38.913786 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691250    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 02:19:38.913786 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.691253    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 02:19:38.914323 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.691813    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 02:19:38.914323 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.700575    2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 22 02:19:38.914323 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.700597    2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 22 02:19:38.914323 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.700615    2581 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 22 02:19:38.914323 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.700623    2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 22 02:19:38.914323 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700700    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 02:19:38.914323 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700708    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 02:19:38.914323 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700713    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 02:19:38.914323 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700717    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 02:19:38.914323 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700721    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 02:19:38.914323 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700725    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 02:19:38.914323 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700729    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 02:19:38.914323 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700732    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 02:19:38.914323 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700755    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 02:19:38.914323 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700759    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 02:19:38.914842 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700763    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 02:19:38.914842 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700766    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 02:19:38.914842 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700770    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 02:19:38.914842 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700774    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 02:19:38.914842 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700778    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 02:19:38.914842 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700782    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 02:19:38.914842 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700786    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 02:19:38.914842 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700789    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 02:19:38.914842 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700793    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 02:19:38.914842 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700797    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 02:19:38.914842 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700802    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 02:19:38.914842 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700806    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 02:19:38.914842 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700812    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 02:19:38.914842 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700818    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 02:19:38.914842 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700824    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 02:19:38.914842 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700829    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 02:19:38.914842 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700833    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 02:19:38.914842 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700838    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 02:19:38.915367 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700841    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 02:19:38.915367 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700846    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 02:19:38.915367 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700850    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 02:19:38.915367 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700854    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 02:19:38.915367 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700858    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 02:19:38.915367 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700862    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 02:19:38.915367 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700866    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 02:19:38.915367 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700870    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 02:19:38.915367 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700874    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 02:19:38.915367 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700877    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 02:19:38.915367 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700881    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 02:19:38.915367 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700886    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 02:19:38.915367 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700890    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 02:19:38.915367 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700894    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 02:19:38.915367 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700899    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 02:19:38.915367 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700902    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 02:19:38.915367 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700906    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 02:19:38.915367 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700910    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 02:19:38.915367 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700914    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 02:19:38.915367 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700918    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 02:19:38.915960 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700922    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 02:19:38.915960 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700926    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 02:19:38.915960 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700930    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 02:19:38.915960 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700934    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 02:19:38.915960 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700940    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 02:19:38.915960 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700944    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 02:19:38.915960 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700949    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 02:19:38.915960 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700953    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 02:19:38.915960 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700957    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 02:19:38.915960 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700961    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 02:19:38.915960 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700965    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 02:19:38.915960 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700969    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 02:19:38.915960 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700973    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 02:19:38.915960 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700978    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 02:19:38.915960 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700982    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 02:19:38.915960 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700986    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 02:19:38.915960 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700990    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 02:19:38.915960 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700994    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 02:19:38.915960 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.700997    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 02:19:38.916525 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701001    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 02:19:38.916525 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701005    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 02:19:38.916525 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701009    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 02:19:38.916525 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701013    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 02:19:38.916525 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701018    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 02:19:38.916525 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701022    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 02:19:38.916525 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701026    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 02:19:38.916525 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701029    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 02:19:38.916525 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701033    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 02:19:38.916525 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701037    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 02:19:38.916525 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701040    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 02:19:38.916525 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701044    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 02:19:38.916525 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701048    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 02:19:38.916525 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701052    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 02:19:38.916525 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701056    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 02:19:38.916525 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701060    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 02:19:38.916525 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701063    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 02:19:38.916525 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701070    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 02:19:38.916525 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701074    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 02:19:38.916525 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701078    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 02:19:39.121664 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701081    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 02:19:39.121664 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701085    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 02:19:39.121664 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701089    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 02:19:39.121664 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701093    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 02:19:39.121664 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701097    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 02:19:39.121664 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701101    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 02:19:39.121664 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701104    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 02:19:39.121664 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701107    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 02:19:39.121664 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701112    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 02:19:39.121664 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701116    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 02:19:39.121664 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701120    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 02:19:39.121664 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701123    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 02:19:39.121664 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.701131    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 02:19:39.121664 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701297    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 02:19:39.121664 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701305    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 02:19:39.122558 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701309    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 02:19:39.122558 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701314    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 02:19:39.122558 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701320    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 02:19:39.122558 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701324    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 02:19:39.122558 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701329    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 02:19:39.122558 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701332    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 02:19:39.122558 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701336    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 02:19:39.122558 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701342    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 02:19:39.122558 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701349    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 02:19:39.122558 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701353    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 02:19:39.122558 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701357    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 02:19:39.122558 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701361    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 02:19:39.122558 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701365    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 02:19:39.122558 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701368    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 02:19:39.122558 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701375    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 02:19:39.122558 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701379    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 02:19:39.122558 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701383    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 02:19:39.122558 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701386    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 02:19:39.122558 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701390    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 02:19:39.122558 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701394    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 02:19:39.123239 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701398    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 02:19:39.123239 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701404    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 02:19:39.123239 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701409    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 02:19:39.123239 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701413    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 02:19:39.123239 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701417    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 02:19:39.123239 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701421    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 02:19:39.123239 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701426    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 02:19:39.123239 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701431    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 02:19:39.123239 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701435    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 02:19:39.123239 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701438    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 02:19:39.123239 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701442    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 02:19:39.123239 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701446    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 02:19:39.123239 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701450    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 02:19:39.123239 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701454    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 02:19:39.123239 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701460    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 02:19:39.123239 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701463    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 02:19:39.123239 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701467    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 02:19:39.123239 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701471    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 02:19:39.123239 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701475    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 02:19:39.123802 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701479    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 02:19:39.123802 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701483    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 02:19:39.123802 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701487    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 02:19:39.123802 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701491    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 02:19:39.123802 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701494    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 02:19:39.123802 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701498    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 02:19:39.123802 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701502    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 02:19:39.123802 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701507    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 02:19:39.123802 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701511    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 02:19:39.123802 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701515    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 02:19:39.123802 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701519    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 02:19:39.123802 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701523    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 02:19:39.123802 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701526    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 02:19:39.123802 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701530    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 02:19:39.123802 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701534    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 02:19:39.123802 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701538    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 02:19:39.123802 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701542    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 02:19:39.123802 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701545    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 02:19:39.123802 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701549    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 02:19:39.123802 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701553    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 02:19:39.124458 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701558    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 02:19:39.124458 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701561    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 02:19:39.124458 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701565    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 02:19:39.124458 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701569    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 02:19:39.124458 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701572    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 02:19:39.124458 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701576    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 02:19:39.124458 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701580    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 02:19:39.124458 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701585    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 02:19:39.124458 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701589    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 02:19:39.124458 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701593    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 02:19:39.124458 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701597    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 02:19:39.124458 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701600    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 02:19:39.124458 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701604    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 02:19:39.124458 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701608    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 02:19:39.124458 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701612    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 02:19:39.124458 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701615    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 02:19:39.124458 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701619    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 02:19:39.124458 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701623    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 02:19:39.124458 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701627    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 02:19:39.124458 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701632    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 02:19:39.125029 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701636    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 02:19:39.125029 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701640    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 02:19:39.125029 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701643    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 02:19:39.125029 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701646    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 02:19:39.125029 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701651    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 02:19:39.125029 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701654    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 02:19:39.125029 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701658    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 02:19:39.125029 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701662    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 02:19:39.125029 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701665    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 02:19:39.125029 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701669    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 02:19:39.125029 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701673    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 02:19:39.125029 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701676    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 02:19:39.125029 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701680    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 02:19:39.125029 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701685    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 02:19:39.125029 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701691    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 02:19:39.125029 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701696    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 02:19:39.125029 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701700    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 02:19:39.125029 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:38.701704    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 02:19:39.125489 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.701711    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 02:19:39.125489 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.701878    2581 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 22 02:19:39.125489 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.704372    2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 22 02:19:39.125489 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.705784    2581 server.go:1019] "Starting client certificate rotation"
Jul 22 02:19:39.125489 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.705893    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 22 02:19:39.125489 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.705938    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 22 02:19:39.125489 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.708622    2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 22 02:19:39.125489 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.712253    2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 22 02:19:39.125489 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.722308    2581 log.go:25] "Validated CRI v1 runtime API"
Jul 22 02:19:39.125489 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.724512    2581 log.go:25] "Validated CRI v1 image API"
Jul 22 02:19:39.125489 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.725337    2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 22 02:19:39.125489 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.727284    2581 fs.go:135] Filesystem UUIDs: map[3c770eab-1ef9-43c3-ac7b-ac6c2aee339e:/dev/nvme0n1p4 4c952fbc-fd02-40da-a767-aff011d63d87:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Jul 22 02:19:39.125880 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.727299    2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 22 02:19:39.126009 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.732098    2581 manager.go:216] Machine: {Timestamp:2026-07-22 02:19:38.731044693 +0000 UTC m=+0.271838331 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100991 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24f8aac8844e6c964248d35ce89fc6 SystemUUID:ec24f8aa-c884-4e6c-9642-48d35ce89fc6 BootID:d17780b2-556b-4503-b897-97d08d080656 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d7:07:43:17:a7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d7:07:43:17:a7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e6:08:ea:1b:a3:91 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 22 02:19:39.126009 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.732203    2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 22 02:19:39.126009 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.732287    2581 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 22 02:19:39.126009 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.734965    2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 22 02:19:39.126009 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.734998    2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-202.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 22 02:19:39.126009 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.735728    2581 topology_manager.go:138] "Creating topology manager with none policy"
Jul 22 02:19:39.126009 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.735755    2581 container_manager_linux.go:309] "Creating device plugin manager"
Jul 22 02:19:39.126009 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.735766    2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.736804    2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.737965    2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.738159    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.741828    2581 state_mem.go:36] "Initialized new in-memory state store"
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.741983    2581 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.742017    2581 kubelet.go:486] "Attempting to sync node with API server"
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.742032    2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.742046    2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.742058    2581 kubelet.go:398] "Adding apiserver pod source"
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.742070    2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.742670    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.742720    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.744717    2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.746523    2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.746564    2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.747697    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.747711    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.747716    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.747722    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.747731    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.747751    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.747761    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.747767    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.747773    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.747779    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.747788    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.747793    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.747816    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.747822    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.751432    2581 server.go:1268] "Started kubelet"
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.751554    2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.751619    2581 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.751729    2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.751840    2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-202.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.755633    2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:38.756575    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:38.756680    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-202.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.757540    2581 server.go:310] "Adding debug handlers to kubelet server"
Jul 22 02:19:39.126309 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:38.758397    2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 22 02:19:39.127417 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.759071    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rvshn"
Jul 22 02:19:39.127417 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:38.760909    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-202.ec2.internal.18c47b2b6acdbf46  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-202.ec2.internal,UID:ip-10-0-138-202.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-202.ec2.internal,},FirstTimestamp:2026-07-22 02:19:38.751237958 +0000 UTC m=+0.292031575,LastTimestamp:2026-07-22 02:19:38.751237958 +0000 UTC m=+0.292031575,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-202.ec2.internal,}"
Jul 22 02:19:39.127417 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.763571    2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 22 02:19:39.127417 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.763573    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 22 02:19:39.127417 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:38.763781    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-202.ec2.internal\" not found"
Jul 22 02:19:39.127417 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.764106    2581 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 22 02:19:39.127417 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.764123    2581 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 22 02:19:39.127417 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.764228    2581 reconstruct.go:97] "Volume reconstruction finished"
Jul 22 02:19:39.127417 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.764238    2581 reconciler.go:29] "Reconciler: start to sync state"
Jul 22 02:19:39.127417 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.764287    2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 22 02:19:39.127417 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.764328    2581 factory.go:55] Registering systemd factory
Jul 22 02:19:39.127417 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.764344    2581 factory.go:223] Registration of the systemd container factory successfully
Jul 22 02:19:39.127417 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.764543    2581 factory.go:153] Registering CRI-O factory
Jul 22 02:19:39.127417 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.764553    2581 factory.go:223] Registration of the crio container factory successfully
Jul 22 02:19:39.127417 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.764607    2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 22 02:19:39.127417 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.764633    2581 factory.go:103] Registering Raw factory
Jul 22 02:19:39.127417 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.764648    2581 manager.go:1196] Started watching for new ooms in manager
Jul 22 02:19:39.127417 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.765097    2581 manager.go:349] Starting recovery of all containers
Jul 22 02:19:39.127417 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:38.765936    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 22 02:19:39.127417 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:38.766024    2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-202.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 22 02:19:39.127417 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.767562    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rvshn"
Jul 22 02:19:39.127417 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.773511    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 22 02:19:39.127417 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.775310    2581 manager.go:354] Recovery completed
Jul 22 02:19:39.127417 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.780779    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 02:19:39.127417 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.784184    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-202.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 02:19:39.127417 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.784215    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-202.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 02:19:39.127417 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.784228    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-202.ec2.internal" event="NodeHasSufficientPID"
Jul 22 02:19:39.127417 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.785071    2581 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 22 02:19:39.127417 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.785079    2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 22 02:19:39.127417 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.785093    2581 state_mem.go:36] "Initialized new in-memory state store"
Jul 22 02:19:39.127417 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.785138    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 02:19:39.127417 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.786083    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-202.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 02:19:39.127417 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.786107    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-202.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 02:19:39.128353 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.786120    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-202.ec2.internal" event="NodeHasSufficientPID"
Jul 22 02:19:39.128353 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.787172    2581 policy_none.go:49] "None policy: Start"
Jul 22 02:19:39.128353 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.787185    2581 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 22 02:19:39.128353 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.787194    2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 22 02:19:39.128353 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.789281    2581 policy_none.go:47] "Start"
Jul 22 02:19:39.128353 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.819848    2581 manager.go:337] "Starting Device Plugin manager"
Jul 22 02:19:39.128353 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:38.819876    2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 22 02:19:39.128353 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.819885    2581 server.go:87] "Starting device plugin registration server"
Jul 22 02:19:39.128353 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.820194    2581 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 22 02:19:39.128353 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.820204    2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 22 02:19:39.128353 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.820315    2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 22 02:19:39.128353 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.820420    2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 22 02:19:39.128353 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.820430    2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 22 02:19:39.128353 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:38.821375    2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 22 02:19:39.128353 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:38.821425    2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-202.ec2.internal\" not found"
Jul 22 02:19:39.128353 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.896932    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 22 02:19:39.128353 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.896977    2581 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 22 02:19:39.128353 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.897000    2581 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 22 02:19:39.128353 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:38.897038    2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 22 02:19:39.128353 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.899074    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 02:19:39.128353 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.920770    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 02:19:39.128353 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.921595    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-202.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 02:19:39.128353 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.921620    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-202.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 02:19:39.128353 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.921633    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-202.ec2.internal" event="NodeHasSufficientPID"
Jul 22 02:19:39.128353 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.921660    2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-202.ec2.internal"
Jul 22 02:19:39.128353 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.930529    2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-202.ec2.internal"
Jul 22 02:19:39.128353 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:38.930548    2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-202.ec2.internal\": node \"ip-10-0-138-202.ec2.internal\" not found"
Jul 22 02:19:39.128353 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:38.944475    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-202.ec2.internal\" not found"
Jul 22 02:19:39.128353 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.998035    2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-202.ec2.internal"]
Jul 22 02:19:39.128353 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.998123    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 02:19:39.128353 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.999385    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-202.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 02:19:39.128353 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.999413    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-202.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 02:19:39.128353 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:38.999425    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-202.ec2.internal" event="NodeHasSufficientPID"
Jul 22 02:19:39.128353 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:39.001714    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 02:19:39.128353 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:39.001845    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.ec2.internal"
Jul 22 02:19:39.128353 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:39.001875    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 02:19:39.128353 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:39.002391    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-202.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 02:19:39.128353 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:39.002413    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-202.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 02:19:39.129379 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:39.002423    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-202.ec2.internal" event="NodeHasSufficientPID"
Jul 22 02:19:39.129379 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:39.002393    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-202.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 02:19:39.129379 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:39.002482    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-202.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 02:19:39.129379 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:39.002499    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-202.ec2.internal" event="NodeHasSufficientPID"
Jul 22 02:19:39.129379 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:39.004676    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-202.ec2.internal"
Jul 22 02:19:39.129379 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:39.004700    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 02:19:39.129379 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:39.005376    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-202.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 02:19:39.129379 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:39.005405    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-202.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 02:19:39.129379 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:39.005415    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-202.ec2.internal" event="NodeHasSufficientPID"
Jul 22 02:19:39.129379 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:39.030320    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-202.ec2.internal\" not found" node="ip-10-0-138-202.ec2.internal"
Jul 22 02:19:39.129379 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:39.033851    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-202.ec2.internal\" not found" node="ip-10-0-138-202.ec2.internal"
Jul 22 02:19:39.129379 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:39.045532    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-202.ec2.internal\" not found"
Jul 22 02:19:39.129379 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:39.065450    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c6e00b7f032530804168912eb043de53-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-202.ec2.internal\" (UID: \"c6e00b7f032530804168912eb043de53\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.ec2.internal"
Jul 22 02:19:39.129379 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:39.065473    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c6e00b7f032530804168912eb043de53-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-202.ec2.internal\" (UID: \"c6e00b7f032530804168912eb043de53\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.ec2.internal"
Jul 22 02:19:39.129379 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:39.065493    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/03f847ed505e198ceec414e6c1b66d63-config\") pod \"kube-apiserver-proxy-ip-10-0-138-202.ec2.internal\" (UID: \"03f847ed505e198ceec414e6c1b66d63\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-202.ec2.internal"
Jul 22 02:19:39.335073 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:39.146143    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-202.ec2.internal\" not found"
Jul 22 02:19:39.335073 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:39.166499    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c6e00b7f032530804168912eb043de53-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-202.ec2.internal\" (UID: \"c6e00b7f032530804168912eb043de53\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.ec2.internal"
Jul 22 02:19:39.335073 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:39.166539    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c6e00b7f032530804168912eb043de53-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-202.ec2.internal\" (UID: \"c6e00b7f032530804168912eb043de53\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.ec2.internal"
Jul 22 02:19:39.335073 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:39.166549    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c6e00b7f032530804168912eb043de53-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-202.ec2.internal\" (UID: \"c6e00b7f032530804168912eb043de53\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.ec2.internal"
Jul 22 02:19:39.335073 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:39.166595    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/03f847ed505e198ceec414e6c1b66d63-config\") pod \"kube-apiserver-proxy-ip-10-0-138-202.ec2.internal\" (UID: \"03f847ed505e198ceec414e6c1b66d63\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-202.ec2.internal"
Jul 22 02:19:39.335073 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:39.166557    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/03f847ed505e198ceec414e6c1b66d63-config\") pod \"kube-apiserver-proxy-ip-10-0-138-202.ec2.internal\" (UID: \"03f847ed505e198ceec414e6c1b66d63\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-202.ec2.internal"
Jul 22 02:19:39.335073 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:39.167068    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c6e00b7f032530804168912eb043de53-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-202.ec2.internal\" (UID: \"c6e00b7f032530804168912eb043de53\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.ec2.internal"
Jul 22 02:19:39.335073 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:39.246908    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-202.ec2.internal\" not found"
Jul 22 02:19:39.335073 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:39.333432    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.ec2.internal"
Jul 22 02:19:39.336689 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:39.336563    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-202.ec2.internal"
Jul 22 02:19:39.569868 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:39.347694    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-202.ec2.internal\" not found"
Jul 22 02:19:39.569868 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:39.448197    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-202.ec2.internal\" not found"
Jul 22 02:19:39.569868 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:39.548558    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-202.ec2.internal\" not found"
Jul 22 02:19:39.801782 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:39.648906    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-202.ec2.internal\" not found"
Jul 22 02:19:39.801782 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:39.705313    2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 22 02:19:39.801782 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:39.705453    2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 22 02:19:39.801782 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:39.749615    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-202.ec2.internal\" not found"
Jul 22 02:19:39.801782 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:39.773609    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-21 02:14:38 +0000 UTC" deadline="2027-12-30 00:22:08.253852902 +0000 UTC"
Jul 22 02:19:39.801782 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:39.773645    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12622h2m28.480211142s"
Jul 22 02:19:40.032035 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:39.819887    2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 02:19:40.032035 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:39.851064    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-202.ec2.internal\" not found"
Jul 22 02:19:40.032035 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:39.945820    2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 02:19:40.032035 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:39.952047    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-202.ec2.internal\" not found"
Jul 22 02:19:40.233493 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:40.052452    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-202.ec2.internal\" not found"
Jul 22 02:19:40.233493 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:40.091418    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6e00b7f032530804168912eb043de53.slice/crio-709474d5ebfc0203313e1cef9f3fefe9c880685555fe7238a643a120cbc57e7e WatchSource:0}: Error finding container 709474d5ebfc0203313e1cef9f3fefe9c880685555fe7238a643a120cbc57e7e: Status 404 returned error can't find the container with id 709474d5ebfc0203313e1cef9f3fefe9c880685555fe7238a643a120cbc57e7e
Jul 22 02:19:40.233493 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:40.091884    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03f847ed505e198ceec414e6c1b66d63.slice/crio-5fcfa95f5e6519b1ad1e4aaa225e63ebaf043822c7df43c7c57e656a22251820 WatchSource:0}: Error finding container 5fcfa95f5e6519b1ad1e4aaa225e63ebaf043822c7df43c7c57e656a22251820: Status 404 returned error can't find the container with id 5fcfa95f5e6519b1ad1e4aaa225e63ebaf043822c7df43c7c57e656a22251820
Jul 22 02:19:40.233493 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.102099    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 02:19:40.233493 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:40.153462    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-202.ec2.internal\" not found"
Jul 22 02:19:40.233493 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.208372    2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 02:19:40.439752 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.263855    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.ec2.internal"
Jul 22 02:19:40.439752 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.274580    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 22 02:19:40.439752 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.275409    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-202.ec2.internal"
Jul 22 02:19:40.439752 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.284153    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 22 02:19:40.726757 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.726561    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 02:19:40.743251 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.742995    2581 apiserver.go:52] "Watching apiserver"
Jul 22 02:19:40.753113 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.752916    2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 22 02:19:40.753819 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.753315    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-nsq6n","openshift-ovn-kubernetes/ovnkube-node-b78fw","kube-system/global-pull-secret-syncer-gvv6m","kube-system/konnectivity-agent-6wxzz","kube-system/kube-apiserver-proxy-ip-10-0-138-202.ec2.internal","openshift-cluster-node-tuning-operator/tuned-6z7wn","openshift-image-registry/node-ca-xfq7z","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.ec2.internal","openshift-multus/multus-additional-cni-plugins-5wkbl","openshift-multus/network-metrics-daemon-6bsw8","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j8m9","openshift-dns/node-resolver-r5qtx","openshift-multus/multus-24vpj","openshift-network-diagnostics/network-check-target-bsnsc"]
Jul 22 02:19:40.756016 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.755991    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:19:40.756125 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:40.756095    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bsw8" podUID="fa01c319-94d5-4113-9457-6b808c24dee5"
Jul 22 02:19:40.758293 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.758269    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.760631 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.760548    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:19:40.760631 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:40.760625    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsnsc" podUID="cd5d3980-4fea-40a0-b7fd-e5a28d013ef3"
Jul 22 02:19:40.765147 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.763040    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 22 02:19:40.765147 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.763260    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 22 02:19:40.765147 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.763565    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qcdzd\""
Jul 22 02:19:40.765147 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.763794    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 22 02:19:40.765147 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.763904    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 22 02:19:40.765147 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.763992    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 22 02:19:40.765147 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.764100    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 22 02:19:40.765147 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.764157    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6wxzz"
Jul 22 02:19:40.765147 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.764513    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 22 02:19:40.765147 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.765085    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-r5qtx"
Jul 22 02:19:40.766827 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.766805    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 22 02:19:40.768225 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.767040    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 22 02:19:40.768225 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.767289    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-b4x5k\""
Jul 22 02:19:40.768225 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.767709    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 22 02:19:40.768225 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.767889    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.768225 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.767960    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 22 02:19:40.768514 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.768316    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4fwmz\""
Jul 22 02:19:40.769879 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.769859    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5wkbl"
Jul 22 02:19:40.770509 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.770489    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 22 02:19:40.770913 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.770773    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 22 02:19:40.770913 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.770778    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-p7f5f\""
Jul 22 02:19:40.772775 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.772183    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j8m9"
Jul 22 02:19:40.772775 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.772277    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 22 02:19:40.772775 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.772606    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 22 02:19:40.772972 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.772910    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 22 02:19:40.772972 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.772916    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-h2zsd\""
Jul 22 02:19:40.773072 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.773038    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 22 02:19:40.773157 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.773140    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 22 02:19:40.773759 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.773702    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tchtq\" (UniqueName: \"kubernetes.io/projected/59f7f98a-9f53-4c16-a176-d944826dcb68-kube-api-access-tchtq\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.774202 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.773865    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fc01079-d37c-4352-b0fd-5d7e7e09a713-tmp-dir\") pod \"node-resolver-r5qtx\" (UID: \"5fc01079-d37c-4352-b0fd-5d7e7e09a713\") " pod="openshift-dns/node-resolver-r5qtx"
Jul 22 02:19:40.774202 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.773904    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/59f7f98a-9f53-4c16-a176-d944826dcb68-ovn-node-metrics-cert\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.774202 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.773928    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/59f7f98a-9f53-4c16-a176-d944826dcb68-ovnkube-script-lib\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.774202 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.773977    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-host-run-netns\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.774202 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.774014    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-run-systemd\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.774202 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.774042    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.774202 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.774067    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/59f7f98a-9f53-4c16-a176-d944826dcb68-ovnkube-config\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.774202 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.774092    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hplh\" (UniqueName: \"kubernetes.io/projected/fa01c319-94d5-4113-9457-6b808c24dee5-kube-api-access-5hplh\") pod \"network-metrics-daemon-6bsw8\" (UID: \"fa01c319-94d5-4113-9457-6b808c24dee5\") " pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:19:40.774202 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.774127    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-run-ovn\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.774202 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.774170    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzz6v\" (UniqueName: \"kubernetes.io/projected/cd5d3980-4fea-40a0-b7fd-e5a28d013ef3-kube-api-access-hzz6v\") pod \"network-check-target-bsnsc\" (UID: \"cd5d3980-4fea-40a0-b7fd-e5a28d013ef3\") " pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:19:40.774866 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.774211    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7fd1ec82-dd3b-4712-87dc-b63028464820-konnectivity-ca\") pod \"konnectivity-agent-6wxzz\" (UID: \"7fd1ec82-dd3b-4712-87dc-b63028464820\") " pod="kube-system/konnectivity-agent-6wxzz"
Jul 22 02:19:40.774866 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.774243    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5fc01079-d37c-4352-b0fd-5d7e7e09a713-hosts-file\") pod \"node-resolver-r5qtx\" (UID: \"5fc01079-d37c-4352-b0fd-5d7e7e09a713\") " pod="openshift-dns/node-resolver-r5qtx"
Jul 22 02:19:40.774866 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.774274    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa01c319-94d5-4113-9457-6b808c24dee5-metrics-certs\") pod \"network-metrics-daemon-6bsw8\" (UID: \"fa01c319-94d5-4113-9457-6b808c24dee5\") " pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:19:40.774866 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.774305    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-host-kubelet\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.774866 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.774333    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-systemd-units\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.774866 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.774361    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7fd1ec82-dd3b-4712-87dc-b63028464820-agent-certs\") pod \"konnectivity-agent-6wxzz\" (UID: \"7fd1ec82-dd3b-4712-87dc-b63028464820\") " pod="kube-system/konnectivity-agent-6wxzz"
Jul 22 02:19:40.774866 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.774391    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-run-openvswitch\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.774866 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.774417    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-node-log\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.774866 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.774443    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-log-socket\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.774866 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.774475    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-host-run-ovn-kubernetes\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.774866 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.774519    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-host-cni-bin\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.774866 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.774573    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-host-cni-netd\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.774866 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.774606    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqj29\" (UniqueName: \"kubernetes.io/projected/5fc01079-d37c-4352-b0fd-5d7e7e09a713-kube-api-access-hqj29\") pod \"node-resolver-r5qtx\" (UID: \"5fc01079-d37c-4352-b0fd-5d7e7e09a713\") " pod="openshift-dns/node-resolver-r5qtx"
Jul 22 02:19:40.774866 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.774633    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-var-lib-openvswitch\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.775654 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.774656    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-etc-openvswitch\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.775654 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.774677    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/59f7f98a-9f53-4c16-a176-d944826dcb68-env-overrides\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.775654 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.774705    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-host-slash\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.775654 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.774929    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 22 02:19:40.778180 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.776283    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dbj7m\""
Jul 22 02:19:40.778180 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.776478    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 22 02:19:40.778180 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.777971    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 22 02:19:40.788477 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.787426    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 22 02:19:40.788477 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.787462    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.790689 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.790496    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nsq6n"
Jul 22 02:19:40.791886 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.791869    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-ddlq6\""
Jul 22 02:19:40.792103 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.791881    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 22 02:19:40.793275 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.793001    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xfq7z"
Jul 22 02:19:40.794108 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.794086    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 22 02:19:40.794199 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.794170    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-zd75h\""
Jul 22 02:19:40.794621 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.794374    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 22 02:19:40.794621 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.794407    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 22 02:19:40.795458 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.795348    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 22 02:19:40.795640 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.795597    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:19:40.795716 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:40.795659    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gvv6m" podUID="adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a"
Jul 22 02:19:40.796427 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.796020    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 22 02:19:40.796427 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.796233    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 22 02:19:40.796865 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.796689    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ll872\""
Jul 22 02:19:40.808162 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.807877    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-mjm7r"
Jul 22 02:19:40.814179 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.814154    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-mjm7r"
Jul 22 02:19:40.865457 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.865342    2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 22 02:19:40.874982 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.874908    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-host-slash\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.874982 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.874950    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-host-var-lib-cni-multus\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.875172 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.875009    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-host-slash\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.875172 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.875043    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8dc86394-9528-4b72-9d62-c64c03585758-host-slash\") pod \"iptables-alerter-nsq6n\" (UID: \"8dc86394-9528-4b72-9d62-c64c03585758\") " pod="openshift-network-operator/iptables-alerter-nsq6n"
Jul 22 02:19:40.875172 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.875066    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4f6a3dfe-18b9-4040-901a-74e50f0cf243-host\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.875172 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.875089    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j75d8\" (UniqueName: \"kubernetes.io/projected/9facab3f-4b58-4168-b73e-ff7535771473-kube-api-access-j75d8\") pod \"node-ca-xfq7z\" (UID: \"9facab3f-4b58-4168-b73e-ff7535771473\") " pod="openshift-image-registry/node-ca-xfq7z"
Jul 22 02:19:40.875172 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.875121    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/59f7f98a-9f53-4c16-a176-d944826dcb68-ovnkube-config\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.875172 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.875144    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8cd528d0-41ba-4a80-ad89-94b6fe93b2af-device-dir\") pod \"aws-ebs-csi-driver-node-2j8m9\" (UID: \"8cd528d0-41ba-4a80-ad89-94b6fe93b2af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j8m9"
Jul 22 02:19:40.875172 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.875170    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hplh\" (UniqueName: \"kubernetes.io/projected/fa01c319-94d5-4113-9457-6b808c24dee5-kube-api-access-5hplh\") pod \"network-metrics-daemon-6bsw8\" (UID: \"fa01c319-94d5-4113-9457-6b808c24dee5\") " pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:19:40.875544 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.875195    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-run-ovn\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.875544 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.875218    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5fc01079-d37c-4352-b0fd-5d7e7e09a713-hosts-file\") pod \"node-resolver-r5qtx\" (UID: \"5fc01079-d37c-4352-b0fd-5d7e7e09a713\") " pod="openshift-dns/node-resolver-r5qtx"
Jul 22 02:19:40.875544 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.875254    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-multus-cni-dir\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.875544 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.875278    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4f6a3dfe-18b9-4040-901a-74e50f0cf243-etc-modprobe-d\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.875544 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.875301    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4f6a3dfe-18b9-4040-901a-74e50f0cf243-sys\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.875544 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.875324    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4f6a3dfe-18b9-4040-901a-74e50f0cf243-lib-modules\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.875544 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.875365    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7a87ca25-67e4-47cd-a391-ff3f1035448f-os-release\") pod \"multus-additional-cni-plugins-5wkbl\" (UID: \"7a87ca25-67e4-47cd-a391-ff3f1035448f\") " pod="openshift-multus/multus-additional-cni-plugins-5wkbl"
Jul 22 02:19:40.875544 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.875404    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9facab3f-4b58-4168-b73e-ff7535771473-host\") pod \"node-ca-xfq7z\" (UID: \"9facab3f-4b58-4168-b73e-ff7535771473\") " pod="openshift-image-registry/node-ca-xfq7z"
Jul 22 02:19:40.875544 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.875427    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8cd528d0-41ba-4a80-ad89-94b6fe93b2af-socket-dir\") pod \"aws-ebs-csi-driver-node-2j8m9\" (UID: \"8cd528d0-41ba-4a80-ad89-94b6fe93b2af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j8m9"
Jul 22 02:19:40.875544 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.875453    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-host-kubelet\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.875544 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.875475    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-systemd-units\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.875544 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.875510    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7fd1ec82-dd3b-4712-87dc-b63028464820-agent-certs\") pod \"konnectivity-agent-6wxzz\" (UID: \"7fd1ec82-dd3b-4712-87dc-b63028464820\") " pod="kube-system/konnectivity-agent-6wxzz"
Jul 22 02:19:40.875544 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.875536    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-multus-conf-dir\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.876193 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.875558    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-host-run-multus-certs\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.876193 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.875584    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8cd528d0-41ba-4a80-ad89-94b6fe93b2af-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2j8m9\" (UID: \"8cd528d0-41ba-4a80-ad89-94b6fe93b2af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j8m9"
Jul 22 02:19:40.876193 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.875613    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-run-openvswitch\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.876193 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.875638    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-host-cni-bin\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.876193 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.875663    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqj29\" (UniqueName: \"kubernetes.io/projected/5fc01079-d37c-4352-b0fd-5d7e7e09a713-kube-api-access-hqj29\") pod \"node-resolver-r5qtx\" (UID: \"5fc01079-d37c-4352-b0fd-5d7e7e09a713\") " pod="openshift-dns/node-resolver-r5qtx"
Jul 22 02:19:40.876193 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.875688    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4f6a3dfe-18b9-4040-901a-74e50f0cf243-etc-sysctl-conf\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.876193 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.875711    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7a87ca25-67e4-47cd-a391-ff3f1035448f-system-cni-dir\") pod \"multus-additional-cni-plugins-5wkbl\" (UID: \"7a87ca25-67e4-47cd-a391-ff3f1035448f\") " pod="openshift-multus/multus-additional-cni-plugins-5wkbl"
Jul 22 02:19:40.876193 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.875751    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-var-lib-openvswitch\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.876193 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.875768    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/59f7f98a-9f53-4c16-a176-d944826dcb68-ovnkube-config\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.876193 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.875782    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-etc-openvswitch\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.876193 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.875826    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/59f7f98a-9f53-4c16-a176-d944826dcb68-env-overrides\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.876193 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.875858    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-host-kubelet\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.876193 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.875868    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-cnibin\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.876193 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.875895    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-multus-socket-dir-parent\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.876867 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.875918    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4f6a3dfe-18b9-4040-901a-74e50f0cf243-run\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.876867 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.875941    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7a87ca25-67e4-47cd-a391-ff3f1035448f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5wkbl\" (UID: \"7a87ca25-67e4-47cd-a391-ff3f1035448f\") " pod="openshift-multus/multus-additional-cni-plugins-5wkbl"
Jul 22 02:19:40.876867 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.875976    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8dc86394-9528-4b72-9d62-c64c03585758-iptables-alerter-script\") pod \"iptables-alerter-nsq6n\" (UID: \"8dc86394-9528-4b72-9d62-c64c03585758\") " pod="openshift-network-operator/iptables-alerter-nsq6n"
Jul 22 02:19:40.876867 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.876000    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4f6a3dfe-18b9-4040-901a-74e50f0cf243-etc-tuned\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.876867 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.876025    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7a87ca25-67e4-47cd-a391-ff3f1035448f-cni-binary-copy\") pod \"multus-additional-cni-plugins-5wkbl\" (UID: \"7a87ca25-67e4-47cd-a391-ff3f1035448f\") " pod="openshift-multus/multus-additional-cni-plugins-5wkbl"
Jul 22 02:19:40.876867 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.876055    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h62w2\" (UniqueName: \"kubernetes.io/projected/7a87ca25-67e4-47cd-a391-ff3f1035448f-kube-api-access-h62w2\") pod \"multus-additional-cni-plugins-5wkbl\" (UID: \"7a87ca25-67e4-47cd-a391-ff3f1035448f\") " pod="openshift-multus/multus-additional-cni-plugins-5wkbl"
Jul 22 02:19:40.876867 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.876083    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tchtq\" (UniqueName: \"kubernetes.io/projected/59f7f98a-9f53-4c16-a176-d944826dcb68-kube-api-access-tchtq\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.876867 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.876124    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fc01079-d37c-4352-b0fd-5d7e7e09a713-tmp-dir\") pod \"node-resolver-r5qtx\" (UID: \"5fc01079-d37c-4352-b0fd-5d7e7e09a713\") " pod="openshift-dns/node-resolver-r5qtx"
Jul 22 02:19:40.876867 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.876127    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-run-ovn\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.876867 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.876161    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-os-release\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.876867 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.876188    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-host-run-k8s-cni-cncf-io\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.876867 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.876218    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pksfw\" (UniqueName: \"kubernetes.io/projected/f332116e-038b-48e4-b0d6-ee8cb1847e7c-kube-api-access-pksfw\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.876867 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.876243    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7a87ca25-67e4-47cd-a391-ff3f1035448f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5wkbl\" (UID: \"7a87ca25-67e4-47cd-a391-ff3f1035448f\") " pod="openshift-multus/multus-additional-cni-plugins-5wkbl"
Jul 22 02:19:40.876867 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.876265    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-hostroot\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.877565 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.876292    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/59f7f98a-9f53-4c16-a176-d944826dcb68-ovn-node-metrics-cert\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.877565 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.876321    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/59f7f98a-9f53-4c16-a176-d944826dcb68-ovnkube-script-lib\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.877565 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.876345    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-host-run-netns\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.877565 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.876372    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-run-systemd\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.877565 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.876398    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.877565 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.876417    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fc01079-d37c-4352-b0fd-5d7e7e09a713-tmp-dir\") pod \"node-resolver-r5qtx\" (UID: \"5fc01079-d37c-4352-b0fd-5d7e7e09a713\") " pod="openshift-dns/node-resolver-r5qtx"
Jul 22 02:19:40.877565 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.876666    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-host-run-netns\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.877565 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.876909    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-system-cni-dir\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.877565 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.876950    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-host-run-netns\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.877565 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.876991    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzz6v\" (UniqueName: \"kubernetes.io/projected/cd5d3980-4fea-40a0-b7fd-e5a28d013ef3-kube-api-access-hzz6v\") pod \"network-check-target-bsnsc\" (UID: \"cd5d3980-4fea-40a0-b7fd-e5a28d013ef3\") " pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:19:40.877565 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.876996    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-run-openvswitch\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.877565 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.877019    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7fd1ec82-dd3b-4712-87dc-b63028464820-konnectivity-ca\") pod \"konnectivity-agent-6wxzz\" (UID: \"7fd1ec82-dd3b-4712-87dc-b63028464820\") " pod="kube-system/konnectivity-agent-6wxzz"
Jul 22 02:19:40.877565 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.877023    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/59f7f98a-9f53-4c16-a176-d944826dcb68-ovnkube-script-lib\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.877565 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.877037    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/59f7f98a-9f53-4c16-a176-d944826dcb68-env-overrides\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.878337 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.877043    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-host-var-lib-cni-bin\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.878337 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.877066    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-run-systemd\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.878337 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.877070    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f332116e-038b-48e4-b0d6-ee8cb1847e7c-multus-daemon-config\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.878337 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.877043    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-host-cni-bin\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.878337 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.877092    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-etc-openvswitch\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.878337 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.877086    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-var-lib-openvswitch\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.878337 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.877137    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.878337 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.877159    2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 22 02:19:40.878337 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.877175    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-etc-kubernetes\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.878337 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.877202    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pr5gp\" (UniqueName: \"kubernetes.io/projected/8dc86394-9528-4b72-9d62-c64c03585758-kube-api-access-pr5gp\") pod \"iptables-alerter-nsq6n\" (UID: \"8dc86394-9528-4b72-9d62-c64c03585758\") " pod="openshift-network-operator/iptables-alerter-nsq6n"
Jul 22 02:19:40.878337 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.877249    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4f6a3dfe-18b9-4040-901a-74e50f0cf243-var-lib-kubelet\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.878337 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.877276    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9fvp\" (UniqueName: \"kubernetes.io/projected/8cd528d0-41ba-4a80-ad89-94b6fe93b2af-kube-api-access-p9fvp\") pod \"aws-ebs-csi-driver-node-2j8m9\" (UID: \"8cd528d0-41ba-4a80-ad89-94b6fe93b2af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j8m9"
Jul 22 02:19:40.878337 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.877302    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f332116e-038b-48e4-b0d6-ee8cb1847e7c-cni-binary-copy\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.878337 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.877142    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5fc01079-d37c-4352-b0fd-5d7e7e09a713-hosts-file\") pod \"node-resolver-r5qtx\" (UID: \"5fc01079-d37c-4352-b0fd-5d7e7e09a713\") " pod="openshift-dns/node-resolver-r5qtx"
Jul 22 02:19:40.879101 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.877348    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9facab3f-4b58-4168-b73e-ff7535771473-serviceca\") pod \"node-ca-xfq7z\" (UID: \"9facab3f-4b58-4168-b73e-ff7535771473\") " pod="openshift-image-registry/node-ca-xfq7z"
Jul 22 02:19:40.879101 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.877381    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f6a3dfe-18b9-4040-901a-74e50f0cf243-tmp\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.879101 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.877457    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-systemd-units\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.879101 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.877463    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8cd528d0-41ba-4a80-ad89-94b6fe93b2af-sys-fs\") pod \"aws-ebs-csi-driver-node-2j8m9\" (UID: \"8cd528d0-41ba-4a80-ad89-94b6fe93b2af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j8m9"
Jul 22 02:19:40.879101 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.877514    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa01c319-94d5-4113-9457-6b808c24dee5-metrics-certs\") pod \"network-metrics-daemon-6bsw8\" (UID: \"fa01c319-94d5-4113-9457-6b808c24dee5\") " pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:19:40.879101 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.877549    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-host-var-lib-kubelet\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.879101 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.877976    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7fd1ec82-dd3b-4712-87dc-b63028464820-konnectivity-ca\") pod \"konnectivity-agent-6wxzz\" (UID: \"7fd1ec82-dd3b-4712-87dc-b63028464820\") " pod="kube-system/konnectivity-agent-6wxzz"
Jul 22 02:19:40.879101 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:40.878119    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 02:19:40.879101 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.878163    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a-dbus\") pod \"global-pull-secret-syncer-gvv6m\" (UID: \"adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a\") " pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:19:40.879101 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.878190    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8cd528d0-41ba-4a80-ad89-94b6fe93b2af-etc-selinux\") pod \"aws-ebs-csi-driver-node-2j8m9\" (UID: \"8cd528d0-41ba-4a80-ad89-94b6fe93b2af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j8m9"
Jul 22 02:19:40.879101 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.878217    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-node-log\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.879101 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.878246    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-log-socket\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.879101 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.878272    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-host-run-ovn-kubernetes\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.879101 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.878292    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-host-cni-netd\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.879101 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.878311    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a-kubelet-config\") pod \"global-pull-secret-syncer-gvv6m\" (UID: \"adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a\") " pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:19:40.879991 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.878332    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4f6a3dfe-18b9-4040-901a-74e50f0cf243-etc-sysconfig\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.879991 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.878350    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4f6a3dfe-18b9-4040-901a-74e50f0cf243-etc-sysctl-d\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.879991 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.878400    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4f6a3dfe-18b9-4040-901a-74e50f0cf243-etc-systemd\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.879991 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.878418    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4f6a3dfe-18b9-4040-901a-74e50f0cf243-etc-kubernetes\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.879991 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.878439    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7a87ca25-67e4-47cd-a391-ff3f1035448f-cnibin\") pod \"multus-additional-cni-plugins-5wkbl\" (UID: \"7a87ca25-67e4-47cd-a391-ff3f1035448f\") " pod="openshift-multus/multus-additional-cni-plugins-5wkbl"
Jul 22 02:19:40.879991 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.878457    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8cd528d0-41ba-4a80-ad89-94b6fe93b2af-registration-dir\") pod \"aws-ebs-csi-driver-node-2j8m9\" (UID: \"8cd528d0-41ba-4a80-ad89-94b6fe93b2af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j8m9"
Jul 22 02:19:40.879991 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.878477    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a-original-pull-secret\") pod \"global-pull-secret-syncer-gvv6m\" (UID: \"adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a\") " pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:19:40.879991 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.878498    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w597g\" (UniqueName: \"kubernetes.io/projected/4f6a3dfe-18b9-4040-901a-74e50f0cf243-kube-api-access-w597g\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.879991 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.878518    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7a87ca25-67e4-47cd-a391-ff3f1035448f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5wkbl\" (UID: \"7a87ca25-67e4-47cd-a391-ff3f1035448f\") " pod="openshift-multus/multus-additional-cni-plugins-5wkbl"
Jul 22 02:19:40.879991 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.878584    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-node-log\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.879991 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:40.878663    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa01c319-94d5-4113-9457-6b808c24dee5-metrics-certs podName:fa01c319-94d5-4113-9457-6b808c24dee5 nodeName:}" failed. No retries permitted until 2026-07-22 02:19:41.378632829 +0000 UTC m=+2.919426449 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fa01c319-94d5-4113-9457-6b808c24dee5-metrics-certs") pod "network-metrics-daemon-6bsw8" (UID: "fa01c319-94d5-4113-9457-6b808c24dee5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 02:19:40.879991 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.878782    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-log-socket\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.879991 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.878813    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-host-run-ovn-kubernetes\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.881521 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.878840    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/59f7f98a-9f53-4c16-a176-d944826dcb68-host-cni-netd\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.884982 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.883112    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7fd1ec82-dd3b-4712-87dc-b63028464820-agent-certs\") pod \"konnectivity-agent-6wxzz\" (UID: \"7fd1ec82-dd3b-4712-87dc-b63028464820\") " pod="kube-system/konnectivity-agent-6wxzz"
Jul 22 02:19:40.884982 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:40.883393    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 02:19:40.884982 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:40.883412    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 02:19:40.884982 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:40.883432    2581 projected.go:196] Error preparing data for projected volume kube-api-access-hzz6v for pod openshift-network-diagnostics/network-check-target-bsnsc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 02:19:40.884982 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:40.883506    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd5d3980-4fea-40a0-b7fd-e5a28d013ef3-kube-api-access-hzz6v podName:cd5d3980-4fea-40a0-b7fd-e5a28d013ef3 nodeName:}" failed. No retries permitted until 2026-07-22 02:19:41.383481867 +0000 UTC m=+2.924275486 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-hzz6v" (UniqueName: "kubernetes.io/projected/cd5d3980-4fea-40a0-b7fd-e5a28d013ef3-kube-api-access-hzz6v") pod "network-check-target-bsnsc" (UID: "cd5d3980-4fea-40a0-b7fd-e5a28d013ef3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 02:19:40.884982 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.883582    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/59f7f98a-9f53-4c16-a176-d944826dcb68-ovn-node-metrics-cert\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.886335 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.885480    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hplh\" (UniqueName: \"kubernetes.io/projected/fa01c319-94d5-4113-9457-6b808c24dee5-kube-api-access-5hplh\") pod \"network-metrics-daemon-6bsw8\" (UID: \"fa01c319-94d5-4113-9457-6b808c24dee5\") " pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:19:40.886335 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.886204    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tchtq\" (UniqueName: \"kubernetes.io/projected/59f7f98a-9f53-4c16-a176-d944826dcb68-kube-api-access-tchtq\") pod \"ovnkube-node-b78fw\" (UID: \"59f7f98a-9f53-4c16-a176-d944826dcb68\") " pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:40.886619 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.886586    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqj29\" (UniqueName: \"kubernetes.io/projected/5fc01079-d37c-4352-b0fd-5d7e7e09a713-kube-api-access-hqj29\") pod \"node-resolver-r5qtx\" (UID: \"5fc01079-d37c-4352-b0fd-5d7e7e09a713\") " pod="openshift-dns/node-resolver-r5qtx"
Jul 22 02:19:40.904560 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.904509    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.ec2.internal" event={"ID":"c6e00b7f032530804168912eb043de53","Type":"ContainerStarted","Data":"709474d5ebfc0203313e1cef9f3fefe9c880685555fe7238a643a120cbc57e7e"}
Jul 22 02:19:40.906575 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.906547    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-202.ec2.internal" event={"ID":"03f847ed505e198ceec414e6c1b66d63","Type":"ContainerStarted","Data":"5fcfa95f5e6519b1ad1e4aaa225e63ebaf043822c7df43c7c57e656a22251820"}
Jul 22 02:19:40.980455 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.978803    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4f6a3dfe-18b9-4040-901a-74e50f0cf243-etc-sysctl-conf\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.980455 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.978841    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7a87ca25-67e4-47cd-a391-ff3f1035448f-system-cni-dir\") pod \"multus-additional-cni-plugins-5wkbl\" (UID: \"7a87ca25-67e4-47cd-a391-ff3f1035448f\") " pod="openshift-multus/multus-additional-cni-plugins-5wkbl"
Jul 22 02:19:40.980455 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.978870    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-cnibin\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.980455 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.978895    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-multus-socket-dir-parent\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.980455 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.978916    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4f6a3dfe-18b9-4040-901a-74e50f0cf243-run\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.980455 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.978925    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7a87ca25-67e4-47cd-a391-ff3f1035448f-system-cni-dir\") pod \"multus-additional-cni-plugins-5wkbl\" (UID: \"7a87ca25-67e4-47cd-a391-ff3f1035448f\") " pod="openshift-multus/multus-additional-cni-plugins-5wkbl"
Jul 22 02:19:40.980455 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.978938    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7a87ca25-67e4-47cd-a391-ff3f1035448f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5wkbl\" (UID: \"7a87ca25-67e4-47cd-a391-ff3f1035448f\") " pod="openshift-multus/multus-additional-cni-plugins-5wkbl"
Jul 22 02:19:40.980455 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.978971    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8dc86394-9528-4b72-9d62-c64c03585758-iptables-alerter-script\") pod \"iptables-alerter-nsq6n\" (UID: \"8dc86394-9528-4b72-9d62-c64c03585758\") " pod="openshift-network-operator/iptables-alerter-nsq6n"
Jul 22 02:19:40.980455 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.978997    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4f6a3dfe-18b9-4040-901a-74e50f0cf243-etc-tuned\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.980455 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979018    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7a87ca25-67e4-47cd-a391-ff3f1035448f-cni-binary-copy\") pod \"multus-additional-cni-plugins-5wkbl\" (UID: \"7a87ca25-67e4-47cd-a391-ff3f1035448f\") " pod="openshift-multus/multus-additional-cni-plugins-5wkbl"
Jul 22 02:19:40.980455 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979018    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4f6a3dfe-18b9-4040-901a-74e50f0cf243-etc-sysctl-conf\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.980455 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979042    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h62w2\" (UniqueName: \"kubernetes.io/projected/7a87ca25-67e4-47cd-a391-ff3f1035448f-kube-api-access-h62w2\") pod \"multus-additional-cni-plugins-5wkbl\" (UID: \"7a87ca25-67e4-47cd-a391-ff3f1035448f\") " pod="openshift-multus/multus-additional-cni-plugins-5wkbl"
Jul 22 02:19:40.980455 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979068    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-os-release\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.980455 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979091    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-host-run-k8s-cni-cncf-io\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.981261 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979102    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-cnibin\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.981261 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979115    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pksfw\" (UniqueName: \"kubernetes.io/projected/f332116e-038b-48e4-b0d6-ee8cb1847e7c-kube-api-access-pksfw\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.981261 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979138    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7a87ca25-67e4-47cd-a391-ff3f1035448f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5wkbl\" (UID: \"7a87ca25-67e4-47cd-a391-ff3f1035448f\") " pod="openshift-multus/multus-additional-cni-plugins-5wkbl"
Jul 22 02:19:40.981261 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979161    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-hostroot\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.981261 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979186    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-system-cni-dir\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.981261 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979208    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-host-run-netns\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.981261 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979226    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4f6a3dfe-18b9-4040-901a-74e50f0cf243-run\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.981261 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979245    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-host-var-lib-cni-bin\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.981261 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979269    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f332116e-038b-48e4-b0d6-ee8cb1847e7c-multus-daemon-config\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.981261 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979314    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-etc-kubernetes\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.981261 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979337    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pr5gp\" (UniqueName: \"kubernetes.io/projected/8dc86394-9528-4b72-9d62-c64c03585758-kube-api-access-pr5gp\") pod \"iptables-alerter-nsq6n\" (UID: \"8dc86394-9528-4b72-9d62-c64c03585758\") " pod="openshift-network-operator/iptables-alerter-nsq6n"
Jul 22 02:19:40.981261 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979360    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4f6a3dfe-18b9-4040-901a-74e50f0cf243-var-lib-kubelet\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.981261 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979383    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9fvp\" (UniqueName: \"kubernetes.io/projected/8cd528d0-41ba-4a80-ad89-94b6fe93b2af-kube-api-access-p9fvp\") pod \"aws-ebs-csi-driver-node-2j8m9\" (UID: \"8cd528d0-41ba-4a80-ad89-94b6fe93b2af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j8m9"
Jul 22 02:19:40.981261 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979406    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f332116e-038b-48e4-b0d6-ee8cb1847e7c-cni-binary-copy\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.981261 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979430    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9facab3f-4b58-4168-b73e-ff7535771473-serviceca\") pod \"node-ca-xfq7z\" (UID: \"9facab3f-4b58-4168-b73e-ff7535771473\") " pod="openshift-image-registry/node-ca-xfq7z"
Jul 22 02:19:40.981980 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979459    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f6a3dfe-18b9-4040-901a-74e50f0cf243-tmp\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.981980 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979481    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8cd528d0-41ba-4a80-ad89-94b6fe93b2af-sys-fs\") pod \"aws-ebs-csi-driver-node-2j8m9\" (UID: \"8cd528d0-41ba-4a80-ad89-94b6fe93b2af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j8m9"
Jul 22 02:19:40.981980 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979517    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-host-var-lib-kubelet\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.981980 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979557    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a-dbus\") pod \"global-pull-secret-syncer-gvv6m\" (UID: \"adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a\") " pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:19:40.981980 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979580    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8cd528d0-41ba-4a80-ad89-94b6fe93b2af-etc-selinux\") pod \"aws-ebs-csi-driver-node-2j8m9\" (UID: \"8cd528d0-41ba-4a80-ad89-94b6fe93b2af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j8m9"
Jul 22 02:19:40.981980 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979612    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a-kubelet-config\") pod \"global-pull-secret-syncer-gvv6m\" (UID: \"adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a\") " pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:19:40.981980 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979636    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4f6a3dfe-18b9-4040-901a-74e50f0cf243-etc-sysconfig\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.981980 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979659    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4f6a3dfe-18b9-4040-901a-74e50f0cf243-etc-sysctl-d\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.981980 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979682    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4f6a3dfe-18b9-4040-901a-74e50f0cf243-etc-systemd\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.981980 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979713    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4f6a3dfe-18b9-4040-901a-74e50f0cf243-etc-kubernetes\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.981980 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979759    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7a87ca25-67e4-47cd-a391-ff3f1035448f-cnibin\") pod \"multus-additional-cni-plugins-5wkbl\" (UID: \"7a87ca25-67e4-47cd-a391-ff3f1035448f\") " pod="openshift-multus/multus-additional-cni-plugins-5wkbl"
Jul 22 02:19:40.981980 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979810    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8cd528d0-41ba-4a80-ad89-94b6fe93b2af-registration-dir\") pod \"aws-ebs-csi-driver-node-2j8m9\" (UID: \"8cd528d0-41ba-4a80-ad89-94b6fe93b2af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j8m9"
Jul 22 02:19:40.981980 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979829    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4f6a3dfe-18b9-4040-901a-74e50f0cf243-var-lib-kubelet\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.981980 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979848    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a-original-pull-secret\") pod \"global-pull-secret-syncer-gvv6m\" (UID: \"adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a\") " pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:19:40.982612 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979861    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7a87ca25-67e4-47cd-a391-ff3f1035448f-cni-binary-copy\") pod \"multus-additional-cni-plugins-5wkbl\" (UID: \"7a87ca25-67e4-47cd-a391-ff3f1035448f\") " pod="openshift-multus/multus-additional-cni-plugins-5wkbl"
Jul 22 02:19:40.982612 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979876    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w597g\" (UniqueName: \"kubernetes.io/projected/4f6a3dfe-18b9-4040-901a-74e50f0cf243-kube-api-access-w597g\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.982612 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979878    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7a87ca25-67e4-47cd-a391-ff3f1035448f-cnibin\") pod \"multus-additional-cni-plugins-5wkbl\" (UID: \"7a87ca25-67e4-47cd-a391-ff3f1035448f\") " pod="openshift-multus/multus-additional-cni-plugins-5wkbl"
Jul 22 02:19:40.982612 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979903    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7a87ca25-67e4-47cd-a391-ff3f1035448f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5wkbl\" (UID: \"7a87ca25-67e4-47cd-a391-ff3f1035448f\") " pod="openshift-multus/multus-additional-cni-plugins-5wkbl"
Jul 22 02:19:40.982612 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979926    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-multus-socket-dir-parent\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.982612 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979933    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-host-var-lib-cni-multus\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.982612 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979928    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-hostroot\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.982612 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979977    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-os-release\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.982612 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.980029    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-host-run-k8s-cni-cncf-io\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.982612 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.980035    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-system-cni-dir\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.982612 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.980059    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8dc86394-9528-4b72-9d62-c64c03585758-host-slash\") pod \"iptables-alerter-nsq6n\" (UID: \"8dc86394-9528-4b72-9d62-c64c03585758\") " pod="openshift-network-operator/iptables-alerter-nsq6n"
Jul 22 02:19:40.982612 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.980085    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4f6a3dfe-18b9-4040-901a-74e50f0cf243-host\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.982612 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.980107    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j75d8\" (UniqueName: \"kubernetes.io/projected/9facab3f-4b58-4168-b73e-ff7535771473-kube-api-access-j75d8\") pod \"node-ca-xfq7z\" (UID: \"9facab3f-4b58-4168-b73e-ff7535771473\") " pod="openshift-image-registry/node-ca-xfq7z"
Jul 22 02:19:40.982612 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.980135    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8cd528d0-41ba-4a80-ad89-94b6fe93b2af-device-dir\") pod \"aws-ebs-csi-driver-node-2j8m9\" (UID: \"8cd528d0-41ba-4a80-ad89-94b6fe93b2af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j8m9"
Jul 22 02:19:40.982612 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.980165    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-multus-cni-dir\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.983325 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.980189    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4f6a3dfe-18b9-4040-901a-74e50f0cf243-etc-modprobe-d\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.983325 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.980212    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4f6a3dfe-18b9-4040-901a-74e50f0cf243-sys\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.983325 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.980237    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4f6a3dfe-18b9-4040-901a-74e50f0cf243-lib-modules\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.983325 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.980262    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7a87ca25-67e4-47cd-a391-ff3f1035448f-os-release\") pod \"multus-additional-cni-plugins-5wkbl\" (UID: \"7a87ca25-67e4-47cd-a391-ff3f1035448f\") " pod="openshift-multus/multus-additional-cni-plugins-5wkbl"
Jul 22 02:19:40.983325 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.980289    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9facab3f-4b58-4168-b73e-ff7535771473-host\") pod \"node-ca-xfq7z\" (UID: \"9facab3f-4b58-4168-b73e-ff7535771473\") " pod="openshift-image-registry/node-ca-xfq7z"
Jul 22 02:19:40.983325 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.980312    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8cd528d0-41ba-4a80-ad89-94b6fe93b2af-socket-dir\") pod \"aws-ebs-csi-driver-node-2j8m9\" (UID: \"8cd528d0-41ba-4a80-ad89-94b6fe93b2af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j8m9"
Jul 22 02:19:40.983325 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.980341    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-multus-conf-dir\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.983325 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.980368    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-host-run-multus-certs\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.983325 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.980392    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8cd528d0-41ba-4a80-ad89-94b6fe93b2af-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2j8m9\" (UID: \"8cd528d0-41ba-4a80-ad89-94b6fe93b2af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j8m9"
Jul 22 02:19:40.983325 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.980406    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4f6a3dfe-18b9-4040-901a-74e50f0cf243-host\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.983325 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.980424    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9facab3f-4b58-4168-b73e-ff7535771473-serviceca\") pod \"node-ca-xfq7z\" (UID: \"9facab3f-4b58-4168-b73e-ff7535771473\") " pod="openshift-image-registry/node-ca-xfq7z"
Jul 22 02:19:40.983325 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.980477    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-host-var-lib-cni-bin\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.983325 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.980484    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8cd528d0-41ba-4a80-ad89-94b6fe93b2af-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2j8m9\" (UID: \"8cd528d0-41ba-4a80-ad89-94b6fe93b2af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j8m9"
Jul 22 02:19:40.983325 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.980484    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f332116e-038b-48e4-b0d6-ee8cb1847e7c-cni-binary-copy\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.983325 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.980511    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-host-run-netns\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.984052 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.980544    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-host-var-lib-cni-multus\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.984052 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.980725    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8cd528d0-41ba-4a80-ad89-94b6fe93b2af-device-dir\") pod \"aws-ebs-csi-driver-node-2j8m9\" (UID: \"8cd528d0-41ba-4a80-ad89-94b6fe93b2af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j8m9"
Jul 22 02:19:40.984052 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.979115    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7a87ca25-67e4-47cd-a391-ff3f1035448f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5wkbl\" (UID: \"7a87ca25-67e4-47cd-a391-ff3f1035448f\") " pod="openshift-multus/multus-additional-cni-plugins-5wkbl"
Jul 22 02:19:40.984052 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.980804    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-multus-cni-dir\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.984052 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.980898    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4f6a3dfe-18b9-4040-901a-74e50f0cf243-etc-modprobe-d\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.984052 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.980915    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f332116e-038b-48e4-b0d6-ee8cb1847e7c-multus-daemon-config\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.984052 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.980948    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4f6a3dfe-18b9-4040-901a-74e50f0cf243-sys\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.984052 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.980979    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-etc-kubernetes\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.984052 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.981009    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7a87ca25-67e4-47cd-a391-ff3f1035448f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5wkbl\" (UID: \"7a87ca25-67e4-47cd-a391-ff3f1035448f\") " pod="openshift-multus/multus-additional-cni-plugins-5wkbl"
Jul 22 02:19:40.984052 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.981028    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4f6a3dfe-18b9-4040-901a-74e50f0cf243-lib-modules\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.984052 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.981086    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7a87ca25-67e4-47cd-a391-ff3f1035448f-os-release\") pod \"multus-additional-cni-plugins-5wkbl\" (UID: \"7a87ca25-67e4-47cd-a391-ff3f1035448f\") " pod="openshift-multus/multus-additional-cni-plugins-5wkbl"
Jul 22 02:19:40.984052 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.981124    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9facab3f-4b58-4168-b73e-ff7535771473-host\") pod \"node-ca-xfq7z\" (UID: \"9facab3f-4b58-4168-b73e-ff7535771473\") " pod="openshift-image-registry/node-ca-xfq7z"
Jul 22 02:19:40.984052 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.981128    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8cd528d0-41ba-4a80-ad89-94b6fe93b2af-socket-dir\") pod \"aws-ebs-csi-driver-node-2j8m9\" (UID: \"8cd528d0-41ba-4a80-ad89-94b6fe93b2af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j8m9"
Jul 22 02:19:40.984052 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.981170    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-multus-conf-dir\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.984052 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.981194    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8dc86394-9528-4b72-9d62-c64c03585758-host-slash\") pod \"iptables-alerter-nsq6n\" (UID: \"8dc86394-9528-4b72-9d62-c64c03585758\") " pod="openshift-network-operator/iptables-alerter-nsq6n"
Jul 22 02:19:40.984747 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.981208    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-host-run-multus-certs\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.984747 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.981255    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8cd528d0-41ba-4a80-ad89-94b6fe93b2af-registration-dir\") pod \"aws-ebs-csi-driver-node-2j8m9\" (UID: \"8cd528d0-41ba-4a80-ad89-94b6fe93b2af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j8m9"
Jul 22 02:19:40.984747 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:40.981270    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 02:19:40.984747 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.981277    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a-kubelet-config\") pod \"global-pull-secret-syncer-gvv6m\" (UID: \"adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a\") " pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:19:40.984747 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.981294    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f332116e-038b-48e4-b0d6-ee8cb1847e7c-host-var-lib-kubelet\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:40.984747 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:40.981337    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a-original-pull-secret podName:adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a nodeName:}" failed. No retries permitted until 2026-07-22 02:19:41.481319479 +0000 UTC m=+3.022113101 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a-original-pull-secret") pod "global-pull-secret-syncer-gvv6m" (UID: "adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a") : object "kube-system"/"original-pull-secret" not registered
Jul 22 02:19:40.984747 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.981344    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8cd528d0-41ba-4a80-ad89-94b6fe93b2af-sys-fs\") pod \"aws-ebs-csi-driver-node-2j8m9\" (UID: \"8cd528d0-41ba-4a80-ad89-94b6fe93b2af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j8m9"
Jul 22 02:19:40.984747 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.981348    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a-dbus\") pod \"global-pull-secret-syncer-gvv6m\" (UID: \"adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a\") " pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:19:40.984747 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.981421    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4f6a3dfe-18b9-4040-901a-74e50f0cf243-etc-sysctl-d\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.984747 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.981423    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8cd528d0-41ba-4a80-ad89-94b6fe93b2af-etc-selinux\") pod \"aws-ebs-csi-driver-node-2j8m9\" (UID: \"8cd528d0-41ba-4a80-ad89-94b6fe93b2af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j8m9"
Jul 22 02:19:40.984747 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.981464    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4f6a3dfe-18b9-4040-901a-74e50f0cf243-etc-sysconfig\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.984747 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.981486    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4f6a3dfe-18b9-4040-901a-74e50f0cf243-etc-systemd\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.984747 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.981529    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4f6a3dfe-18b9-4040-901a-74e50f0cf243-etc-kubernetes\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.984747 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.981844    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8dc86394-9528-4b72-9d62-c64c03585758-iptables-alerter-script\") pod \"iptables-alerter-nsq6n\" (UID: \"8dc86394-9528-4b72-9d62-c64c03585758\") " pod="openshift-network-operator/iptables-alerter-nsq6n"
Jul 22 02:19:40.985533 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.982080    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4f6a3dfe-18b9-4040-901a-74e50f0cf243-etc-tuned\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.985533 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.983015    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f6a3dfe-18b9-4040-901a-74e50f0cf243-tmp\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.989130 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.988770    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h62w2\" (UniqueName: \"kubernetes.io/projected/7a87ca25-67e4-47cd-a391-ff3f1035448f-kube-api-access-h62w2\") pod \"multus-additional-cni-plugins-5wkbl\" (UID: \"7a87ca25-67e4-47cd-a391-ff3f1035448f\") " pod="openshift-multus/multus-additional-cni-plugins-5wkbl"
Jul 22 02:19:40.989501 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.989305    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7a87ca25-67e4-47cd-a391-ff3f1035448f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5wkbl\" (UID: \"7a87ca25-67e4-47cd-a391-ff3f1035448f\") " pod="openshift-multus/multus-additional-cni-plugins-5wkbl"
Jul 22 02:19:40.990238 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.989852    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pr5gp\" (UniqueName: \"kubernetes.io/projected/8dc86394-9528-4b72-9d62-c64c03585758-kube-api-access-pr5gp\") pod \"iptables-alerter-nsq6n\" (UID: \"8dc86394-9528-4b72-9d62-c64c03585758\") " pod="openshift-network-operator/iptables-alerter-nsq6n"
Jul 22 02:19:40.990238 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.990199    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9fvp\" (UniqueName: \"kubernetes.io/projected/8cd528d0-41ba-4a80-ad89-94b6fe93b2af-kube-api-access-p9fvp\") pod \"aws-ebs-csi-driver-node-2j8m9\" (UID: \"8cd528d0-41ba-4a80-ad89-94b6fe93b2af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j8m9"
Jul 22 02:19:40.990238 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.990217    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w597g\" (UniqueName: \"kubernetes.io/projected/4f6a3dfe-18b9-4040-901a-74e50f0cf243-kube-api-access-w597g\") pod \"tuned-6z7wn\" (UID: \"4f6a3dfe-18b9-4040-901a-74e50f0cf243\") " pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:40.991213 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.990975    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j75d8\" (UniqueName: \"kubernetes.io/projected/9facab3f-4b58-4168-b73e-ff7535771473-kube-api-access-j75d8\") pod \"node-ca-xfq7z\" (UID: \"9facab3f-4b58-4168-b73e-ff7535771473\") " pod="openshift-image-registry/node-ca-xfq7z"
Jul 22 02:19:40.991213 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:40.991175    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pksfw\" (UniqueName: \"kubernetes.io/projected/f332116e-038b-48e4-b0d6-ee8cb1847e7c-kube-api-access-pksfw\") pod \"multus-24vpj\" (UID: \"f332116e-038b-48e4-b0d6-ee8cb1847e7c\") " pod="openshift-multus/multus-24vpj"
Jul 22 02:19:41.074412 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:41.074385    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:19:41.093596 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:41.093568    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6wxzz"
Jul 22 02:19:41.103412 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:41.103391    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-r5qtx"
Jul 22 02:19:41.111028 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:41.111009    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6z7wn"
Jul 22 02:19:41.120704 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:41.120643    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5wkbl"
Jul 22 02:19:41.127459 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:41.127438    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j8m9"
Jul 22 02:19:41.133203 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:41.133176    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-24vpj"
Jul 22 02:19:41.143830 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:41.143810    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nsq6n"
Jul 22 02:19:41.150456 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:41.150429    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xfq7z"
Jul 22 02:19:41.383860 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:41.383780    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzz6v\" (UniqueName: \"kubernetes.io/projected/cd5d3980-4fea-40a0-b7fd-e5a28d013ef3-kube-api-access-hzz6v\") pod \"network-check-target-bsnsc\" (UID: \"cd5d3980-4fea-40a0-b7fd-e5a28d013ef3\") " pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:19:41.383860 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:41.383815    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa01c319-94d5-4113-9457-6b808c24dee5-metrics-certs\") pod \"network-metrics-daemon-6bsw8\" (UID: \"fa01c319-94d5-4113-9457-6b808c24dee5\") " pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:19:41.384064 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:41.383957    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 02:19:41.384064 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:41.383988    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 02:19:41.384064 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:41.384000    2581 projected.go:196] Error preparing data for projected volume kube-api-access-hzz6v for pod openshift-network-diagnostics/network-check-target-bsnsc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 02:19:41.384064 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:41.383960    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 02:19:41.384064 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:41.384059    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd5d3980-4fea-40a0-b7fd-e5a28d013ef3-kube-api-access-hzz6v podName:cd5d3980-4fea-40a0-b7fd-e5a28d013ef3 nodeName:}" failed. No retries permitted until 2026-07-22 02:19:42.384041065 +0000 UTC m=+3.924834673 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-hzz6v" (UniqueName: "kubernetes.io/projected/cd5d3980-4fea-40a0-b7fd-e5a28d013ef3-kube-api-access-hzz6v") pod "network-check-target-bsnsc" (UID: "cd5d3980-4fea-40a0-b7fd-e5a28d013ef3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 02:19:41.384355 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:41.384079    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa01c319-94d5-4113-9457-6b808c24dee5-metrics-certs podName:fa01c319-94d5-4113-9457-6b808c24dee5 nodeName:}" failed. No retries permitted until 2026-07-22 02:19:42.384067487 +0000 UTC m=+3.924861092 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fa01c319-94d5-4113-9457-6b808c24dee5-metrics-certs") pod "network-metrics-daemon-6bsw8" (UID: "fa01c319-94d5-4113-9457-6b808c24dee5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 02:19:41.484195 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:41.484164    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a-original-pull-secret\") pod \"global-pull-secret-syncer-gvv6m\" (UID: \"adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a\") " pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:19:41.484370 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:41.484310    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 02:19:41.484426 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:41.484376    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a-original-pull-secret podName:adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a nodeName:}" failed. No retries permitted until 2026-07-22 02:19:42.484360466 +0000 UTC m=+4.025154070 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a-original-pull-secret") pod "global-pull-secret-syncer-gvv6m" (UID: "adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a") : object "kube-system"/"original-pull-secret" not registered
Jul 22 02:19:41.817919 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:41.817858    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 02:14:40 +0000 UTC" deadline="2028-03-29 09:38:31.513906484 +0000 UTC"
Jul 22 02:19:41.817919 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:41.817913    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14791h18m49.69599792s"
Jul 22 02:19:42.048788 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:42.048761    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fd1ec82_dd3b_4712_87dc_b63028464820.slice/crio-ea605798ca3b21aac87fe14e102f72112585ddeafe3e1e8cee052f2d34b906cc WatchSource:0}: Error finding container ea605798ca3b21aac87fe14e102f72112585ddeafe3e1e8cee052f2d34b906cc: Status 404 returned error can't find the container with id ea605798ca3b21aac87fe14e102f72112585ddeafe3e1e8cee052f2d34b906cc
Jul 22 02:19:42.050214 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:42.050172    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f6a3dfe_18b9_4040_901a_74e50f0cf243.slice/crio-5dcbe3b13382146776370550f11a8608cbc722b248150b6ed1be2bd8da84ecd0 WatchSource:0}: Error finding container 5dcbe3b13382146776370550f11a8608cbc722b248150b6ed1be2bd8da84ecd0: Status 404 returned error can't find the container with id 5dcbe3b13382146776370550f11a8608cbc722b248150b6ed1be2bd8da84ecd0
Jul 22 02:19:42.052115 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:42.052081    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fc01079_d37c_4352_b0fd_5d7e7e09a713.slice/crio-e2bbaf51ae34e1344b3b8df1c6b401bad0d0661efd5751deb165f9014f3a3089 WatchSource:0}: Error finding container e2bbaf51ae34e1344b3b8df1c6b401bad0d0661efd5751deb165f9014f3a3089: Status 404 returned error can't find the container with id e2bbaf51ae34e1344b3b8df1c6b401bad0d0661efd5751deb165f9014f3a3089
Jul 22 02:19:42.055623 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:42.055513    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cd528d0_41ba_4a80_ad89_94b6fe93b2af.slice/crio-325755b9e1a62886de4d5d8f8e35ff68372aae67b3f66baa63a72f29900c0d99 WatchSource:0}: Error finding container 325755b9e1a62886de4d5d8f8e35ff68372aae67b3f66baa63a72f29900c0d99: Status 404 returned error can't find the container with id 325755b9e1a62886de4d5d8f8e35ff68372aae67b3f66baa63a72f29900c0d99
Jul 22 02:19:42.055841 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:42.055811    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59f7f98a_9f53_4c16_a176_d944826dcb68.slice/crio-31ca99fde2dc18a64ad69aff88eb0c323baa67e46149302fc63b92ada8c2671b WatchSource:0}: Error finding container 31ca99fde2dc18a64ad69aff88eb0c323baa67e46149302fc63b92ada8c2671b: Status 404 returned error can't find the container with id 31ca99fde2dc18a64ad69aff88eb0c323baa67e46149302fc63b92ada8c2671b
Jul 22 02:19:42.057174 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:19:42.057044    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a87ca25_67e4_47cd_a391_ff3f1035448f.slice/crio-0c5570f04528e1207f3ccac52e37d3b1ae02c88e8438fb7021f43d4fb3b1c031 WatchSource:0}: Error finding container 0c5570f04528e1207f3ccac52e37d3b1ae02c88e8438fb7021f43d4fb3b1c031: Status 404 returned error can't find the container with id 0c5570f04528e1207f3ccac52e37d3b1ae02c88e8438fb7021f43d4fb3b1c031
Jul 22 02:19:42.390189 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:42.390096    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzz6v\" (UniqueName: \"kubernetes.io/projected/cd5d3980-4fea-40a0-b7fd-e5a28d013ef3-kube-api-access-hzz6v\") pod \"network-check-target-bsnsc\" (UID: \"cd5d3980-4fea-40a0-b7fd-e5a28d013ef3\") " pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:19:42.390189 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:42.390142    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa01c319-94d5-4113-9457-6b808c24dee5-metrics-certs\") pod \"network-metrics-daemon-6bsw8\" (UID: \"fa01c319-94d5-4113-9457-6b808c24dee5\") " pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:19:42.390367 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:42.390247    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 02:19:42.390367 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:42.390267    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 02:19:42.390367 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:42.390291    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 02:19:42.390367 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:42.390301    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa01c319-94d5-4113-9457-6b808c24dee5-metrics-certs podName:fa01c319-94d5-4113-9457-6b808c24dee5 nodeName:}" failed. No retries permitted until 2026-07-22 02:19:44.390287045 +0000 UTC m=+5.931080651 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fa01c319-94d5-4113-9457-6b808c24dee5-metrics-certs") pod "network-metrics-daemon-6bsw8" (UID: "fa01c319-94d5-4113-9457-6b808c24dee5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 02:19:42.390367 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:42.390303    2581 projected.go:196] Error preparing data for projected volume kube-api-access-hzz6v for pod openshift-network-diagnostics/network-check-target-bsnsc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 02:19:42.390367 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:42.390343    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd5d3980-4fea-40a0-b7fd-e5a28d013ef3-kube-api-access-hzz6v podName:cd5d3980-4fea-40a0-b7fd-e5a28d013ef3 nodeName:}" failed. No retries permitted until 2026-07-22 02:19:44.39033164 +0000 UTC m=+5.931125245 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-hzz6v" (UniqueName: "kubernetes.io/projected/cd5d3980-4fea-40a0-b7fd-e5a28d013ef3-kube-api-access-hzz6v") pod "network-check-target-bsnsc" (UID: "cd5d3980-4fea-40a0-b7fd-e5a28d013ef3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 02:19:42.491547 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:42.491505    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a-original-pull-secret\") pod \"global-pull-secret-syncer-gvv6m\" (UID: \"adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a\") " pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:19:42.492663 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:42.491717    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 02:19:42.492663 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:42.491849    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a-original-pull-secret podName:adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a nodeName:}" failed. No retries permitted until 2026-07-22 02:19:44.491829492 +0000 UTC m=+6.032623101 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a-original-pull-secret") pod "global-pull-secret-syncer-gvv6m" (UID: "adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a") : object "kube-system"/"original-pull-secret" not registered
Jul 22 02:19:42.818165 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:42.818103    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 02:14:40 +0000 UTC" deadline="2028-04-26 02:21:44.024880884 +0000 UTC"
Jul 22 02:19:42.818165 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:42.818162    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15456h2m1.206722355s"
Jul 22 02:19:42.900502 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:42.899647    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:19:42.900502 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:42.899790    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsnsc" podUID="cd5d3980-4fea-40a0-b7fd-e5a28d013ef3"
Jul 22 02:19:42.900502 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:42.900212    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:19:42.900502 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:42.900323    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bsw8" podUID="fa01c319-94d5-4113-9457-6b808c24dee5"
Jul 22 02:19:42.900502 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:42.900387    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:19:42.900502 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:42.900458    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gvv6m" podUID="adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a"
Jul 22 02:19:42.926301 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:42.926266    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nsq6n" event={"ID":"8dc86394-9528-4b72-9d62-c64c03585758","Type":"ContainerStarted","Data":"16bad2d6a974cb56780ceff2ae72cb8665da9b9e3922a5fafe6d3dede9a0cc2c"}
Jul 22 02:19:42.929041 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:42.929006    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5wkbl" event={"ID":"7a87ca25-67e4-47cd-a391-ff3f1035448f","Type":"ContainerStarted","Data":"0c5570f04528e1207f3ccac52e37d3b1ae02c88e8438fb7021f43d4fb3b1c031"}
Jul 22 02:19:42.943959 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:42.943928    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b78fw" event={"ID":"59f7f98a-9f53-4c16-a176-d944826dcb68","Type":"ContainerStarted","Data":"31ca99fde2dc18a64ad69aff88eb0c323baa67e46149302fc63b92ada8c2671b"}
Jul 22 02:19:42.954291 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:42.954263    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j8m9" event={"ID":"8cd528d0-41ba-4a80-ad89-94b6fe93b2af","Type":"ContainerStarted","Data":"325755b9e1a62886de4d5d8f8e35ff68372aae67b3f66baa63a72f29900c0d99"}
Jul 22 02:19:42.977244 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:42.977194    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6z7wn" event={"ID":"4f6a3dfe-18b9-4040-901a-74e50f0cf243","Type":"ContainerStarted","Data":"5dcbe3b13382146776370550f11a8608cbc722b248150b6ed1be2bd8da84ecd0"}
Jul 22 02:19:42.990530 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:42.990497    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xfq7z" event={"ID":"9facab3f-4b58-4168-b73e-ff7535771473","Type":"ContainerStarted","Data":"be6814b9313a41a6492bb28a7221642c2efd426e324a4929b590447bc252af41"}
Jul 22 02:19:42.994102 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:42.994073    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-24vpj" event={"ID":"f332116e-038b-48e4-b0d6-ee8cb1847e7c","Type":"ContainerStarted","Data":"5655f919a187d6d0ff7a26df0a5d931938687c48cf84b96a32b1cc79d7b6ed97"}
Jul 22 02:19:42.997693 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:42.997622    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-r5qtx" event={"ID":"5fc01079-d37c-4352-b0fd-5d7e7e09a713","Type":"ContainerStarted","Data":"e2bbaf51ae34e1344b3b8df1c6b401bad0d0661efd5751deb165f9014f3a3089"}
Jul 22 02:19:43.021304 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:43.021226    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6wxzz" event={"ID":"7fd1ec82-dd3b-4712-87dc-b63028464820","Type":"ContainerStarted","Data":"ea605798ca3b21aac87fe14e102f72112585ddeafe3e1e8cee052f2d34b906cc"}
Jul 22 02:19:43.025042 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:43.025009    2581 generic.go:356] "Generic (PLEG): container finished" podID="c6e00b7f032530804168912eb043de53" containerID="635e3358e1a8b4a8aa647059be4d61fce8e0e73eda86d0797de38ee3654421e4" exitCode=0
Jul 22 02:19:43.025191 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:43.025103    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.ec2.internal" event={"ID":"c6e00b7f032530804168912eb043de53","Type":"ContainerDied","Data":"635e3358e1a8b4a8aa647059be4d61fce8e0e73eda86d0797de38ee3654421e4"}
Jul 22 02:19:43.035255 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:43.034751    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-202.ec2.internal" event={"ID":"03f847ed505e198ceec414e6c1b66d63","Type":"ContainerStarted","Data":"476131b58517d13896fc27706616191051d9e7b606b4a9888e4e7dae905da82b"}
Jul 22 02:19:44.045859 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:44.045792    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.ec2.internal" event={"ID":"c6e00b7f032530804168912eb043de53","Type":"ContainerStarted","Data":"42997605bb2b9fe710efc53f391241349afc6be6de2c9f5fdda3c08b7ce035e0"}
Jul 22 02:19:44.064592 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:44.063951    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-202.ec2.internal" podStartSLOduration=4.063929695 podStartE2EDuration="4.063929695s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:19:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:19:43.058489304 +0000 UTC m=+4.599282934" watchObservedRunningTime="2026-07-22 02:19:44.063929695 +0000 UTC m=+5.604723303"
Jul 22 02:19:44.407227 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:44.406615    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzz6v\" (UniqueName: \"kubernetes.io/projected/cd5d3980-4fea-40a0-b7fd-e5a28d013ef3-kube-api-access-hzz6v\") pod \"network-check-target-bsnsc\" (UID: \"cd5d3980-4fea-40a0-b7fd-e5a28d013ef3\") " pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:19:44.407227 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:44.406922    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa01c319-94d5-4113-9457-6b808c24dee5-metrics-certs\") pod \"network-metrics-daemon-6bsw8\" (UID: \"fa01c319-94d5-4113-9457-6b808c24dee5\") " pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:19:44.407227 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:44.406951    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 02:19:44.407227 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:44.406983    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 02:19:44.407227 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:44.406998    2581 projected.go:196] Error preparing data for projected volume kube-api-access-hzz6v for pod openshift-network-diagnostics/network-check-target-bsnsc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 02:19:44.407227 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:44.407061    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd5d3980-4fea-40a0-b7fd-e5a28d013ef3-kube-api-access-hzz6v podName:cd5d3980-4fea-40a0-b7fd-e5a28d013ef3 nodeName:}" failed. No retries permitted until 2026-07-22 02:19:48.407042783 +0000 UTC m=+9.947836390 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-hzz6v" (UniqueName: "kubernetes.io/projected/cd5d3980-4fea-40a0-b7fd-e5a28d013ef3-kube-api-access-hzz6v") pod "network-check-target-bsnsc" (UID: "cd5d3980-4fea-40a0-b7fd-e5a28d013ef3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 02:19:44.407227 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:44.407085    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 02:19:44.407227 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:44.407138    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa01c319-94d5-4113-9457-6b808c24dee5-metrics-certs podName:fa01c319-94d5-4113-9457-6b808c24dee5 nodeName:}" failed. No retries permitted until 2026-07-22 02:19:48.407122129 +0000 UTC m=+9.947915739 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fa01c319-94d5-4113-9457-6b808c24dee5-metrics-certs") pod "network-metrics-daemon-6bsw8" (UID: "fa01c319-94d5-4113-9457-6b808c24dee5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 02:19:44.507701 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:44.507661    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a-original-pull-secret\") pod \"global-pull-secret-syncer-gvv6m\" (UID: \"adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a\") " pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:19:44.507914 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:44.507829    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 02:19:44.507914 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:44.507894    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a-original-pull-secret podName:adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a nodeName:}" failed. No retries permitted until 2026-07-22 02:19:48.507873789 +0000 UTC m=+10.048667408 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a-original-pull-secret") pod "global-pull-secret-syncer-gvv6m" (UID: "adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a") : object "kube-system"/"original-pull-secret" not registered
Jul 22 02:19:44.900436 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:44.899605    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:19:44.900436 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:44.899756    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bsw8" podUID="fa01c319-94d5-4113-9457-6b808c24dee5"
Jul 22 02:19:44.900436 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:44.899828    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:19:44.900436 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:44.899909    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gvv6m" podUID="adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a"
Jul 22 02:19:44.900436 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:44.899947    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:19:44.900436 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:44.900015    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsnsc" podUID="cd5d3980-4fea-40a0-b7fd-e5a28d013ef3"
Jul 22 02:19:46.898773 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:46.898608    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:19:46.898773 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:46.898757    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsnsc" podUID="cd5d3980-4fea-40a0-b7fd-e5a28d013ef3"
Jul 22 02:19:46.899317 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:46.898820    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:19:46.899317 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:46.898821    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:19:46.899317 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:46.898943    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bsw8" podUID="fa01c319-94d5-4113-9457-6b808c24dee5"
Jul 22 02:19:46.899317 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:46.899018    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gvv6m" podUID="adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a"
Jul 22 02:19:48.443161 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:48.443125    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzz6v\" (UniqueName: \"kubernetes.io/projected/cd5d3980-4fea-40a0-b7fd-e5a28d013ef3-kube-api-access-hzz6v\") pod \"network-check-target-bsnsc\" (UID: \"cd5d3980-4fea-40a0-b7fd-e5a28d013ef3\") " pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:19:48.443161 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:48.443166    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa01c319-94d5-4113-9457-6b808c24dee5-metrics-certs\") pod \"network-metrics-daemon-6bsw8\" (UID: \"fa01c319-94d5-4113-9457-6b808c24dee5\") " pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:19:48.443681 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:48.443278    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 02:19:48.443681 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:48.443295    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 02:19:48.443681 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:48.443311    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 02:19:48.443681 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:48.443324    2581 projected.go:196] Error preparing data for projected volume kube-api-access-hzz6v for pod openshift-network-diagnostics/network-check-target-bsnsc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 02:19:48.443681 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:48.443338    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa01c319-94d5-4113-9457-6b808c24dee5-metrics-certs podName:fa01c319-94d5-4113-9457-6b808c24dee5 nodeName:}" failed. No retries permitted until 2026-07-22 02:19:56.443320656 +0000 UTC m=+17.984114263 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fa01c319-94d5-4113-9457-6b808c24dee5-metrics-certs") pod "network-metrics-daemon-6bsw8" (UID: "fa01c319-94d5-4113-9457-6b808c24dee5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 02:19:48.443681 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:48.443360    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd5d3980-4fea-40a0-b7fd-e5a28d013ef3-kube-api-access-hzz6v podName:cd5d3980-4fea-40a0-b7fd-e5a28d013ef3 nodeName:}" failed. No retries permitted until 2026-07-22 02:19:56.443349648 +0000 UTC m=+17.984143259 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-hzz6v" (UniqueName: "kubernetes.io/projected/cd5d3980-4fea-40a0-b7fd-e5a28d013ef3-kube-api-access-hzz6v") pod "network-check-target-bsnsc" (UID: "cd5d3980-4fea-40a0-b7fd-e5a28d013ef3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 02:19:48.544663 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:48.544409    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a-original-pull-secret\") pod \"global-pull-secret-syncer-gvv6m\" (UID: \"adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a\") " pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:19:48.544663 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:48.544580    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 02:19:48.544663 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:48.544643    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a-original-pull-secret podName:adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a nodeName:}" failed. No retries permitted until 2026-07-22 02:19:56.544624875 +0000 UTC m=+18.085418497 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a-original-pull-secret") pod "global-pull-secret-syncer-gvv6m" (UID: "adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a") : object "kube-system"/"original-pull-secret" not registered
Jul 22 02:19:48.898153 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:48.898088    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:19:48.898330 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:48.898216    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gvv6m" podUID="adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a"
Jul 22 02:19:48.898618 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:48.898594    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:19:48.899707 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:48.899674    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsnsc" podUID="cd5d3980-4fea-40a0-b7fd-e5a28d013ef3"
Jul 22 02:19:48.899830 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:48.899770    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:19:48.899926 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:48.899905    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bsw8" podUID="fa01c319-94d5-4113-9457-6b808c24dee5"
Jul 22 02:19:50.898567 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:50.898035    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:19:50.898567 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:50.898183    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bsw8" podUID="fa01c319-94d5-4113-9457-6b808c24dee5"
Jul 22 02:19:50.898567 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:50.898504    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:19:50.899172 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:50.898610    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gvv6m" podUID="adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a"
Jul 22 02:19:50.899172 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:50.898789    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:19:50.899172 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:50.898893    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsnsc" podUID="cd5d3980-4fea-40a0-b7fd-e5a28d013ef3"
Jul 22 02:19:52.898158 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:52.898121    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:19:52.898596 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:52.898121    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:19:52.898596 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:52.898247    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsnsc" podUID="cd5d3980-4fea-40a0-b7fd-e5a28d013ef3"
Jul 22 02:19:52.898596 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:52.898373    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bsw8" podUID="fa01c319-94d5-4113-9457-6b808c24dee5"
Jul 22 02:19:52.898596 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:52.898121    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:19:52.898596 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:52.898474    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gvv6m" podUID="adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a"
Jul 22 02:19:54.897322 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:54.897292    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:19:54.897836 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:54.897292    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:19:54.897836 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:54.897445    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:19:54.897836 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:54.897406    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsnsc" podUID="cd5d3980-4fea-40a0-b7fd-e5a28d013ef3"
Jul 22 02:19:54.897836 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:54.897516    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gvv6m" podUID="adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a"
Jul 22 02:19:54.897836 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:54.897589    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bsw8" podUID="fa01c319-94d5-4113-9457-6b808c24dee5"
Jul 22 02:19:56.504774 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:56.504712    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzz6v\" (UniqueName: \"kubernetes.io/projected/cd5d3980-4fea-40a0-b7fd-e5a28d013ef3-kube-api-access-hzz6v\") pod \"network-check-target-bsnsc\" (UID: \"cd5d3980-4fea-40a0-b7fd-e5a28d013ef3\") " pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:19:56.504774 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:56.504777    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa01c319-94d5-4113-9457-6b808c24dee5-metrics-certs\") pod \"network-metrics-daemon-6bsw8\" (UID: \"fa01c319-94d5-4113-9457-6b808c24dee5\") " pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:19:56.505259 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:56.504870    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 02:19:56.505259 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:56.504935    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa01c319-94d5-4113-9457-6b808c24dee5-metrics-certs podName:fa01c319-94d5-4113-9457-6b808c24dee5 nodeName:}" failed. No retries permitted until 2026-07-22 02:20:12.504919954 +0000 UTC m=+34.045713559 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fa01c319-94d5-4113-9457-6b808c24dee5-metrics-certs") pod "network-metrics-daemon-6bsw8" (UID: "fa01c319-94d5-4113-9457-6b808c24dee5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 02:19:56.505259 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:56.504877    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 02:19:56.505259 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:56.504978    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 02:19:56.505259 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:56.504991    2581 projected.go:196] Error preparing data for projected volume kube-api-access-hzz6v for pod openshift-network-diagnostics/network-check-target-bsnsc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 02:19:56.505259 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:56.505039    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd5d3980-4fea-40a0-b7fd-e5a28d013ef3-kube-api-access-hzz6v podName:cd5d3980-4fea-40a0-b7fd-e5a28d013ef3 nodeName:}" failed. No retries permitted until 2026-07-22 02:20:12.505024719 +0000 UTC m=+34.045818342 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-hzz6v" (UniqueName: "kubernetes.io/projected/cd5d3980-4fea-40a0-b7fd-e5a28d013ef3-kube-api-access-hzz6v") pod "network-check-target-bsnsc" (UID: "cd5d3980-4fea-40a0-b7fd-e5a28d013ef3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 02:19:56.605441 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:56.605406    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a-original-pull-secret\") pod \"global-pull-secret-syncer-gvv6m\" (UID: \"adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a\") " pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:19:56.605594 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:56.605529    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 02:19:56.605594 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:56.605588    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a-original-pull-secret podName:adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a nodeName:}" failed. No retries permitted until 2026-07-22 02:20:12.605569998 +0000 UTC m=+34.146363615 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a-original-pull-secret") pod "global-pull-secret-syncer-gvv6m" (UID: "adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a") : object "kube-system"/"original-pull-secret" not registered
Jul 22 02:19:56.898282 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:56.898194    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:19:56.898282 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:56.898222    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:19:56.898515 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:56.898356    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bsw8" podUID="fa01c319-94d5-4113-9457-6b808c24dee5"
Jul 22 02:19:56.898515 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:56.898391    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsnsc" podUID="cd5d3980-4fea-40a0-b7fd-e5a28d013ef3"
Jul 22 02:19:56.898515 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:56.898431    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:19:56.898515 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:56.898511    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gvv6m" podUID="adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a"
Jul 22 02:19:58.898305 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:58.898271    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:19:58.898730 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:58.898380    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:19:58.898730 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:19:58.898408    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:19:58.898730 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:58.898380    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsnsc" podUID="cd5d3980-4fea-40a0-b7fd-e5a28d013ef3"
Jul 22 02:19:58.898730 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:58.898520    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bsw8" podUID="fa01c319-94d5-4113-9457-6b808c24dee5"
Jul 22 02:19:58.898730 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:19:58.898624    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gvv6m" podUID="adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a"
Jul 22 02:20:00.898975 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:00.898860    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:20:00.899440 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:00.898994    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gvv6m" podUID="adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a"
Jul 22 02:20:00.899693 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:00.899681    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:20:00.900095 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:00.899798    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsnsc" podUID="cd5d3980-4fea-40a0-b7fd-e5a28d013ef3"
Jul 22 02:20:00.900095 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:00.899949    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:20:00.900095 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:00.900049    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bsw8" podUID="fa01c319-94d5-4113-9457-6b808c24dee5"
Jul 22 02:20:01.084785 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:01.084530    2581 generic.go:356] "Generic (PLEG): container finished" podID="7a87ca25-67e4-47cd-a391-ff3f1035448f" containerID="5f578a23cd90fb0d3f4b4c6ffabaf81b7f48d533fde01a65e0a40ab74f9ec89d" exitCode=0
Jul 22 02:20:01.084941 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:01.084782    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5wkbl" event={"ID":"7a87ca25-67e4-47cd-a391-ff3f1035448f","Type":"ContainerDied","Data":"5f578a23cd90fb0d3f4b4c6ffabaf81b7f48d533fde01a65e0a40ab74f9ec89d"}
Jul 22 02:20:01.087577 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:01.087547    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b78fw" event={"ID":"59f7f98a-9f53-4c16-a176-d944826dcb68","Type":"ContainerStarted","Data":"f604cad3851732b6590c417f84d753f28ef567bc4ac1728a2aa0d9fe63d992a3"}
Jul 22 02:20:01.087683 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:01.087641    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b78fw" event={"ID":"59f7f98a-9f53-4c16-a176-d944826dcb68","Type":"ContainerStarted","Data":"4d17234c5f4855984b358231217a64cbaf6cf41538e847bbae335884bded8096"}
Jul 22 02:20:01.089205 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:01.089130    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j8m9" event={"ID":"8cd528d0-41ba-4a80-ad89-94b6fe93b2af","Type":"ContainerStarted","Data":"5456e5cd895850ff5cd6d896684d9e84dc20afa3e5b2b9ab1be0e2c3e31f7622"}
Jul 22 02:20:01.090620 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:01.090591    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6z7wn" event={"ID":"4f6a3dfe-18b9-4040-901a-74e50f0cf243","Type":"ContainerStarted","Data":"bea42c0d1c79b59470a807c2a032d7d053e7a9d2be91b2c2961f74cd039ec32c"}
Jul 22 02:20:01.092177 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:01.092150    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xfq7z" event={"ID":"9facab3f-4b58-4168-b73e-ff7535771473","Type":"ContainerStarted","Data":"b68cdae7b710195b115cc60c0a2ef9ea373aca75d9bdf84f18837d4ba153e89e"}
Jul 22 02:20:01.093728 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:01.093705    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-24vpj" event={"ID":"f332116e-038b-48e4-b0d6-ee8cb1847e7c","Type":"ContainerStarted","Data":"8430bd894956c9e778b6af76218b72f850e6b895561cdcc96105ca89d7f6a2a7"}
Jul 22 02:20:01.095165 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:01.095139    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-r5qtx" event={"ID":"5fc01079-d37c-4352-b0fd-5d7e7e09a713","Type":"ContainerStarted","Data":"eafbcf771ed784c3b17571bf52253541a4d569da41b11886bd42e739bcd2d105"}
Jul 22 02:20:01.096434 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:01.096411    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6wxzz" event={"ID":"7fd1ec82-dd3b-4712-87dc-b63028464820","Type":"ContainerStarted","Data":"01c7185c108cdb116ddde588c4858b17f6ab682194cdbe240cce160e3b0e1c6a"}
Jul 22 02:20:01.109983 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:01.109936    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.ec2.internal" podStartSLOduration=21.109923566 podStartE2EDuration="21.109923566s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:19:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:19:44.064571424 +0000 UTC m=+5.605365052" watchObservedRunningTime="2026-07-22 02:20:01.109923566 +0000 UTC m=+22.650717192"
Jul 22 02:20:01.144350 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:01.144307    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6wxzz" podStartSLOduration=8.647025367 podStartE2EDuration="23.144292868s" totalImagesPullingTime="14.497267501s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:19:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:01.144091569 +0000 UTC m=+22.684885210" watchObservedRunningTime="2026-07-22 02:20:01.144292868 +0000 UTC m=+22.685086495"
Jul 22 02:20:01.144535 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:01.144510    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xfq7z" podStartSLOduration=3.468139579 podStartE2EDuration="22.144502261s" totalImagesPullingTime="18.676362682s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:19:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:01.12556276 +0000 UTC m=+22.666356387" watchObservedRunningTime="2026-07-22 02:20:01.144502261 +0000 UTC m=+22.685295896"
Jul 22 02:20:01.166518 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:01.166478    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-r5qtx" podStartSLOduration=4.443971865 podStartE2EDuration="23.166463148s" totalImagesPullingTime="18.722491283s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:19:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:01.166153007 +0000 UTC m=+22.706946634" watchObservedRunningTime="2026-07-22 02:20:01.166463148 +0000 UTC m=+22.707256775"
Jul 22 02:20:01.187644 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:01.187346    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-6z7wn" podStartSLOduration=4.46350635 podStartE2EDuration="23.187323789s" totalImagesPullingTime="18.723817439s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:19:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:01.187239428 +0000 UTC m=+22.728033054" watchObservedRunningTime="2026-07-22 02:20:01.187323789 +0000 UTC m=+22.728117417"
Jul 22 02:20:01.235664 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:01.235636    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:01.352999 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:01.352971    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:01.452939 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:01.452913    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:01.852351 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:01.852067    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6wxzz"
Jul 22 02:20:01.852967 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:01.852947    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6wxzz"
Jul 22 02:20:01.870439 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:01.870392    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-24vpj" podStartSLOduration=4.100839247 podStartE2EDuration="22.870372343s" totalImagesPullingTime="18.769533096s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:19:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:01.205290697 +0000 UTC m=+22.746084323" watchObservedRunningTime="2026-07-22 02:20:01.870372343 +0000 UTC m=+23.411165972"
Jul 22 02:20:02.099958 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:02.099922    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nsq6n" event={"ID":"8dc86394-9528-4b72-9d62-c64c03585758","Type":"ContainerStarted","Data":"2a0f96318529557884c07e24277c49edff8ca24c03715b6579a62107e7f72aaf"}
Jul 22 02:20:02.102828 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:02.102797    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b78fw" event={"ID":"59f7f98a-9f53-4c16-a176-d944826dcb68","Type":"ContainerStarted","Data":"04eac416f75b9639e7c5f38833f2375b7f1a6f67149c11f97b3e9ba59d855d05"}
Jul 22 02:20:02.102939 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:02.102873    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b78fw" event={"ID":"59f7f98a-9f53-4c16-a176-d944826dcb68","Type":"ContainerStarted","Data":"0e2f2671236b25b2b9adc41109e5b67fff3bcb1cfc0fc3847d294e7d461e68cd"}
Jul 22 02:20:02.102939 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:02.102905    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b78fw" event={"ID":"59f7f98a-9f53-4c16-a176-d944826dcb68","Type":"ContainerStarted","Data":"88face3c0fa8fae4020a3fcfbe1511f78037a208ffab740757b19e01d138fe8f"}
Jul 22 02:20:02.103020 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:02.102941    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b78fw" event={"ID":"59f7f98a-9f53-4c16-a176-d944826dcb68","Type":"ContainerStarted","Data":"d1fc2dc9319ac181fab1633564d8c215dd6ab509bdacb73f9b71af739d92176f"}
Jul 22 02:20:02.115899 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:02.115854    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nsq6n" podStartSLOduration=4.439714419 podStartE2EDuration="23.115838622s" totalImagesPullingTime="18.676124203s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:19:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:02.115242842 +0000 UTC m=+23.656036469" watchObservedRunningTime="2026-07-22 02:20:02.115838622 +0000 UTC m=+23.656632259"
Jul 22 02:20:02.233400 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:02.233367    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:02.335029 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:02.334999    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:02.384184 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:02.384157    2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 22 02:20:02.836652 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:02.836553    2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-22T02:20:02.384179646Z","UUID":"b8f802aa-2614-443c-a93f-5b73056cae07","Handler":null,"Name":"","Endpoint":""}
Jul 22 02:20:02.838189 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:02.838162    2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 22 02:20:02.838189 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:02.838195    2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 22 02:20:02.897722 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:02.897690    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:20:02.897919 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:02.897751    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:20:02.897919 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:02.897688    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:20:02.897919 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:02.897845    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gvv6m" podUID="adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a"
Jul 22 02:20:02.898089 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:02.897992    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsnsc" podUID="cd5d3980-4fea-40a0-b7fd-e5a28d013ef3"
Jul 22 02:20:02.898145 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:02.898110    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bsw8" podUID="fa01c319-94d5-4113-9457-6b808c24dee5"
Jul 22 02:20:03.107444 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:03.107373    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j8m9" event={"ID":"8cd528d0-41ba-4a80-ad89-94b6fe93b2af","Type":"ContainerStarted","Data":"3f18e75f9080255a91f77a120580b0eaf5735a8362fc816167ebd64107f4bc47"}
Jul 22 02:20:03.107444 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:03.107428    2581 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 22 02:20:04.114396 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:04.114357    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b78fw" event={"ID":"59f7f98a-9f53-4c16-a176-d944826dcb68","Type":"ContainerStarted","Data":"715659d85199ea4faef3bf3e90e5e29a251b3a43f9c5e39f5c0983c023251b91"}
Jul 22 02:20:04.116883 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:04.116852    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j8m9" event={"ID":"8cd528d0-41ba-4a80-ad89-94b6fe93b2af","Type":"ContainerStarted","Data":"eb483dc7f1eba3623f3569f8b9d991b5b55ff2f0e8ed90dc2e48989ba091a886"}
Jul 22 02:20:04.135099 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:04.135054    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j8m9" podStartSLOduration=5.090689609 podStartE2EDuration="26.135036427s" totalImagesPullingTime="21.044346818s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:19:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:04.134617281 +0000 UTC m=+25.675410908" watchObservedRunningTime="2026-07-22 02:20:04.135036427 +0000 UTC m=+25.675830054"
Jul 22 02:20:04.897508 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:04.897458    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:20:04.897508 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:04.897510    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:20:04.897784 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:04.897458    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:20:04.897784 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:04.897588    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsnsc" podUID="cd5d3980-4fea-40a0-b7fd-e5a28d013ef3"
Jul 22 02:20:04.897784 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:04.897685    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bsw8" podUID="fa01c319-94d5-4113-9457-6b808c24dee5"
Jul 22 02:20:04.897784 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:04.897772    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gvv6m" podUID="adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a"
Jul 22 02:20:06.122811 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:06.122573    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:06.125692 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:06.125626    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5wkbl" event={"ID":"7a87ca25-67e4-47cd-a391-ff3f1035448f","Type":"ContainerStarted","Data":"bf5b2545058f9962038666420322747484d19fe744f7ff0217f139805f27591a"}
Jul 22 02:20:06.129765 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:06.129717    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b78fw" event={"ID":"59f7f98a-9f53-4c16-a176-d944826dcb68","Type":"ContainerStarted","Data":"909acb78d150299a5e7dbb8fc396e29937017f651807928e19df916cddeef829"}
Jul 22 02:20:06.130054 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:06.130015    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:20:06.145702 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:06.145308    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:20:06.181866 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:06.181814    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-b78fw" podStartSLOduration=9.38867667 podStartE2EDuration="28.181796121s" totalImagesPullingTime="18.793119451s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:19:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:06.179610503 +0000 UTC m=+27.720404130" watchObservedRunningTime="2026-07-22 02:20:06.181796121 +0000 UTC m=+27.722589755"
Jul 22 02:20:06.201159 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:06.201130    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:06.263700 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:06.263668    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:06.841335 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:06.841304    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:06.897492 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:06.897457    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:20:06.897492 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:06.897489    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:20:06.897649 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:06.897558    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsnsc" podUID="cd5d3980-4fea-40a0-b7fd-e5a28d013ef3"
Jul 22 02:20:06.897649 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:06.897598    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:20:06.897731 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:06.897709    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bsw8" podUID="fa01c319-94d5-4113-9457-6b808c24dee5"
Jul 22 02:20:06.897790 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:06.897768    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gvv6m" podUID="adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a"
Jul 22 02:20:07.141783 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:07.137028    2581 generic.go:356] "Generic (PLEG): container finished" podID="7a87ca25-67e4-47cd-a391-ff3f1035448f" containerID="bf5b2545058f9962038666420322747484d19fe744f7ff0217f139805f27591a" exitCode=0
Jul 22 02:20:07.141783 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:07.137385    2581 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 22 02:20:07.141783 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:07.139464    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5wkbl" event={"ID":"7a87ca25-67e4-47cd-a391-ff3f1035448f","Type":"ContainerDied","Data":"bf5b2545058f9962038666420322747484d19fe744f7ff0217f139805f27591a"}
Jul 22 02:20:07.141783 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:07.140637    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:20:07.161270 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:07.160773    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:20:07.874963 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:07.874782    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:07.889317 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:07.889286    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gvv6m"]
Jul 22 02:20:07.889477 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:07.889403    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:20:07.889523 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:07.889493    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gvv6m" podUID="adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a"
Jul 22 02:20:07.893000 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:07.892972    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6bsw8"]
Jul 22 02:20:07.893169 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:07.893092    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:20:07.893233 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:07.893195    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bsw8" podUID="fa01c319-94d5-4113-9457-6b808c24dee5"
Jul 22 02:20:07.910056 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:07.909955    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bsnsc"]
Jul 22 02:20:07.910056 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:07.910056    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:20:07.910245 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:07.910136    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsnsc" podUID="cd5d3980-4fea-40a0-b7fd-e5a28d013ef3"
Jul 22 02:20:07.975670 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:07.975626    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:08.140678 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:08.140651    2581 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 22 02:20:08.246516 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:08.246484    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:08.345977 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:08.345946    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:08.401863 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:08.401840    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-r5qtx_5fc01079-d37c-4352-b0fd-5d7e7e09a713/dns-node-resolver/0.log"
Jul 22 02:20:08.447982 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:08.447944    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:08.501596 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:08.501568    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:20:08.626581 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:08.626552    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:08.732913 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:08.732881    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:08.855855 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:08.855773    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6wxzz"
Jul 22 02:20:08.855998 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:08.855898    2581 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 22 02:20:08.856403 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:08.856388    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6wxzz"
Jul 22 02:20:08.925312 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:08.925279    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:09.147081 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:09.146998    2581 generic.go:356] "Generic (PLEG): container finished" podID="7a87ca25-67e4-47cd-a391-ff3f1035448f" containerID="205daa9a05bc400c95dece1c226bd1608059863e94fed70ec33e74b148f82744" exitCode=0
Jul 22 02:20:09.147235 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:09.147081    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5wkbl" event={"ID":"7a87ca25-67e4-47cd-a391-ff3f1035448f","Type":"ContainerDied","Data":"205daa9a05bc400c95dece1c226bd1608059863e94fed70ec33e74b148f82744"}
Jul 22 02:20:09.601537 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:09.601513    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xfq7z_9facab3f-4b58-4168-b73e-ff7535771473/node-ca/0.log"
Jul 22 02:20:09.898335 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:09.898244    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:20:09.898335 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:09.898286    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:20:09.898508 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:09.898369    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsnsc" podUID="cd5d3980-4fea-40a0-b7fd-e5a28d013ef3"
Jul 22 02:20:09.898508 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:09.898462    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bsw8" podUID="fa01c319-94d5-4113-9457-6b808c24dee5"
Jul 22 02:20:09.898585 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:09.898539    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:20:09.898637 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:09.898621    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gvv6m" podUID="adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a"
Jul 22 02:20:10.039579 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:10.039538    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:10.111943 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:10.111915    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:10.165772 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:10.165667    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-b78fw" podUID="59f7f98a-9f53-4c16-a176-d944826dcb68" containerName="ovnkube-controller" probeResult="failure" output=""
Jul 22 02:20:10.210947 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:10.210912    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:10.312113 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:10.312087    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:10.520016 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:10.519987    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:10.622902 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:10.622867    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:10.715283 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:10.715248    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:11.153551 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:11.153521    2581 generic.go:356] "Generic (PLEG): container finished" podID="7a87ca25-67e4-47cd-a391-ff3f1035448f" containerID="6be7d7dfebf35c2aff4eaa88103c3a29f135a8ea740d04afa79c5a2e1b8cf734" exitCode=0
Jul 22 02:20:11.153713 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:11.153576    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5wkbl" event={"ID":"7a87ca25-67e4-47cd-a391-ff3f1035448f","Type":"ContainerDied","Data":"6be7d7dfebf35c2aff4eaa88103c3a29f135a8ea740d04afa79c5a2e1b8cf734"}
Jul 22 02:20:11.458383 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:11.458295    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:11.897466 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:11.897425    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:20:11.898000 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:11.897475    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:20:11.898000 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:11.897475    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:20:11.898000 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:11.897545    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsnsc" podUID="cd5d3980-4fea-40a0-b7fd-e5a28d013ef3"
Jul 22 02:20:11.898000 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:11.897648    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gvv6m" podUID="adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a"
Jul 22 02:20:11.898000 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:11.897760    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bsw8" podUID="fa01c319-94d5-4113-9457-6b808c24dee5"
Jul 22 02:20:12.170335 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:12.170255    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:12.270024 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:12.269994    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:12.535712 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:12.535670    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzz6v\" (UniqueName: \"kubernetes.io/projected/cd5d3980-4fea-40a0-b7fd-e5a28d013ef3-kube-api-access-hzz6v\") pod \"network-check-target-bsnsc\" (UID: \"cd5d3980-4fea-40a0-b7fd-e5a28d013ef3\") " pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:20:12.535712 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:12.535717    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa01c319-94d5-4113-9457-6b808c24dee5-metrics-certs\") pod \"network-metrics-daemon-6bsw8\" (UID: \"fa01c319-94d5-4113-9457-6b808c24dee5\") " pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:20:12.535951 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:12.535869    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 02:20:12.535951 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:12.535896    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 02:20:12.535951 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:12.535908    2581 projected.go:196] Error preparing data for projected volume kube-api-access-hzz6v for pod openshift-network-diagnostics/network-check-target-bsnsc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 02:20:12.536055 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:12.535955    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd5d3980-4fea-40a0-b7fd-e5a28d013ef3-kube-api-access-hzz6v podName:cd5d3980-4fea-40a0-b7fd-e5a28d013ef3 nodeName:}" failed. No retries permitted until 2026-07-22 02:20:44.535942059 +0000 UTC m=+66.076735664 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-hzz6v" (UniqueName: "kubernetes.io/projected/cd5d3980-4fea-40a0-b7fd-e5a28d013ef3-kube-api-access-hzz6v") pod "network-check-target-bsnsc" (UID: "cd5d3980-4fea-40a0-b7fd-e5a28d013ef3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 02:20:12.536055 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:12.535895    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 02:20:12.536055 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:12.536009    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa01c319-94d5-4113-9457-6b808c24dee5-metrics-certs podName:fa01c319-94d5-4113-9457-6b808c24dee5 nodeName:}" failed. No retries permitted until 2026-07-22 02:20:44.535994403 +0000 UTC m=+66.076788009 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fa01c319-94d5-4113-9457-6b808c24dee5-metrics-certs") pod "network-metrics-daemon-6bsw8" (UID: "fa01c319-94d5-4113-9457-6b808c24dee5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 02:20:12.636917 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:12.636863    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a-original-pull-secret\") pod \"global-pull-secret-syncer-gvv6m\" (UID: \"adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a\") " pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:20:12.637122 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:12.637030    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 02:20:12.637122 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:12.637111    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a-original-pull-secret podName:adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a nodeName:}" failed. No retries permitted until 2026-07-22 02:20:44.637092008 +0000 UTC m=+66.177885630 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a-original-pull-secret") pod "global-pull-secret-syncer-gvv6m" (UID: "adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a") : object "kube-system"/"original-pull-secret" not registered
Jul 22 02:20:13.897349 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:13.897317    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:20:13.897349 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:13.897351    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:20:13.897819 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:13.897439    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bsw8" podUID="fa01c319-94d5-4113-9457-6b808c24dee5"
Jul 22 02:20:13.897819 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:13.897472    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:20:13.897819 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:13.897558    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gvv6m" podUID="adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a"
Jul 22 02:20:13.897819 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:13.897654    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsnsc" podUID="cd5d3980-4fea-40a0-b7fd-e5a28d013ef3"
Jul 22 02:20:14.482522 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:14.482478    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:14.628238 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:14.628192    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:14.870178 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:14.870129    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:14.970586 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:14.970555    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:15.072270 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:15.072233    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:15.220914 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:15.220838    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:15.374346 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:15.374307    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:15.846305 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:15.846123    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:15.897300 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:15.897222    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:20:15.897300 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:15.897255    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:20:15.897300 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:15.897257    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:20:15.897504 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:15.897347    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bsw8" podUID="fa01c319-94d5-4113-9457-6b808c24dee5"
Jul 22 02:20:15.897504 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:15.897458    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsnsc" podUID="cd5d3980-4fea-40a0-b7fd-e5a28d013ef3"
Jul 22 02:20:15.897574 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:15.897518    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gvv6m" podUID="adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a"
Jul 22 02:20:16.166804 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:16.166717    2581 generic.go:356] "Generic (PLEG): container finished" podID="7a87ca25-67e4-47cd-a391-ff3f1035448f" containerID="319c1e9a4755edf3921ac70e14a8982b8e11c1714543be9d21dfda4d5443f05f" exitCode=0
Jul 22 02:20:16.166804 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:16.166772    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5wkbl" event={"ID":"7a87ca25-67e4-47cd-a391-ff3f1035448f","Type":"ContainerDied","Data":"319c1e9a4755edf3921ac70e14a8982b8e11c1714543be9d21dfda4d5443f05f"}
Jul 22 02:20:16.584980 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:16.584938    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:16.973443 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:16.973354    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:17.172256 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:17.172226    2581 generic.go:356] "Generic (PLEG): container finished" podID="7a87ca25-67e4-47cd-a391-ff3f1035448f" containerID="9743ae51b9f32c983eca7ec1c59d8a285b0a279cb3b25d44b7880b3e8fd6a4ee" exitCode=0
Jul 22 02:20:17.172645 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:17.172283    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5wkbl" event={"ID":"7a87ca25-67e4-47cd-a391-ff3f1035448f","Type":"ContainerDied","Data":"9743ae51b9f32c983eca7ec1c59d8a285b0a279cb3b25d44b7880b3e8fd6a4ee"}
Jul 22 02:20:17.212018 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:17.211983    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:17.385900 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:17.385870    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:17.778398 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:17.778366    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:17.880039 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:17.880004    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:17.898109 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:17.898089    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:20:17.898210 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:17.898179    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:20:17.898210 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:17.898187    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsnsc" podUID="cd5d3980-4fea-40a0-b7fd-e5a28d013ef3"
Jul 22 02:20:17.898210 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:17.898203    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:20:17.898301 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:17.898284    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bsw8" podUID="fa01c319-94d5-4113-9457-6b808c24dee5"
Jul 22 02:20:17.898363 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:17.898347    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gvv6m" podUID="adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a"
Jul 22 02:20:18.177656 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:18.177568    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5wkbl" event={"ID":"7a87ca25-67e4-47cd-a391-ff3f1035448f","Type":"ContainerStarted","Data":"c09df72e8a41c44764f5ad9fd2838ab8b942d08a8ef3d07c73271dd55124fef1"}
Jul 22 02:20:18.192961 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:18.192932    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:18.203202 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:18.203134    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-5wkbl" podStartSLOduration=9.839129606 podStartE2EDuration="40.203117453s" totalImagesPullingTime="30.363987847s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:19:38 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:18.20149757 +0000 UTC m=+39.742291196" watchObservedRunningTime="2026-07-22 02:20:18.203117453 +0000 UTC m=+39.743911087"
Jul 22 02:20:18.380770 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:18.380722    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:18.574201 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:18.574162    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:18.975909 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:18.975826    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:19.251783 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:19.251733    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:19.326061 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:19.326028    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:19.381599 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:19.381564    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:19.589665 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:19.589579    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:19.897915 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:19.897834    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:20:19.897915 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:19.897852    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:20:19.898188 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:19.897845    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:20:19.898188 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:19.897947    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bsw8" podUID="fa01c319-94d5-4113-9457-6b808c24dee5"
Jul 22 02:20:19.898188 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:19.898016    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gvv6m" podUID="adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a"
Jul 22 02:20:19.898188 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:19.898095    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsnsc" podUID="cd5d3980-4fea-40a0-b7fd-e5a28d013ef3"
Jul 22 02:20:20.378101 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:20.378060    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:20.681997 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:20.681920    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:20.783450 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:20.783416    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:21.195991 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:21.195957    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:21.585871 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:21.585833    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:21.898098 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:21.898019    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:20:21.898247 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:21.898019    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:20:21.898247 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:21.898133    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gvv6m" podUID="adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a"
Jul 22 02:20:21.898247 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:21.898019    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:20:21.898343 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:21.898197    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsnsc" podUID="cd5d3980-4fea-40a0-b7fd-e5a28d013ef3"
Jul 22 02:20:21.898343 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:21.898308    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bsw8" podUID="fa01c319-94d5-4113-9457-6b808c24dee5"
Jul 22 02:20:21.977728 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:21.977690    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:22.379063 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:22.379003    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:22.479406 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:22.479376    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:22.792216 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:22.792169    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:23.208359 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:23.208278    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:23.378680 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:23.378621    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:23.785846 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:23.785808    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:23.897754 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:23.897715    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:20:23.898187 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:23.897831    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:20:23.898187 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:23.897938    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsnsc" podUID="cd5d3980-4fea-40a0-b7fd-e5a28d013ef3"
Jul 22 02:20:23.898187 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:23.897833    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bsw8" podUID="fa01c319-94d5-4113-9457-6b808c24dee5"
Jul 22 02:20:23.898187 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:23.897835    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:20:23.898187 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:23.898043    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gvv6m" podUID="adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a"
Jul 22 02:20:24.573487 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:24.573451    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:24.785088 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:24.785055    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:24.979797 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:24.979679    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:25.182377 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:25.182344    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:25.897962 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:25.897929    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:20:25.898163 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:25.898018    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:20:25.898163 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:25.898022    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsnsc" podUID="cd5d3980-4fea-40a0-b7fd-e5a28d013ef3"
Jul 22 02:20:25.898163 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:25.898051    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:20:25.898319 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:25.898158    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bsw8" podUID="fa01c319-94d5-4113-9457-6b808c24dee5"
Jul 22 02:20:25.898319 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:25.898248    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gvv6m" podUID="adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a"
Jul 22 02:20:27.165135 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:27.165106    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:27.265628 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:27.265573    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:27.897535 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:27.897497    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:20:27.897712 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:27.897554    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:20:27.897712 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:27.897497    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:20:27.897712 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:27.897642    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bsw8" podUID="fa01c319-94d5-4113-9457-6b808c24dee5"
Jul 22 02:20:27.897847 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:27.897708    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsnsc" podUID="cd5d3980-4fea-40a0-b7fd-e5a28d013ef3"
Jul 22 02:20:27.897847 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:27.897803    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gvv6m" podUID="adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a"
Jul 22 02:20:29.897905 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:29.897868    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:20:29.898311 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:29.897960    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:20:29.898311 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:29.897971    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:20:29.898311 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:29.897967    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsnsc" podUID="cd5d3980-4fea-40a0-b7fd-e5a28d013ef3"
Jul 22 02:20:29.898311 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:29.898048    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bsw8" podUID="fa01c319-94d5-4113-9457-6b808c24dee5"
Jul 22 02:20:29.898311 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:29.898111    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gvv6m" podUID="adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a"
Jul 22 02:20:31.897448 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:31.897413    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:20:31.897888 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:31.897509    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:20:31.897888 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:31.897519    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:20:31.897888 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:31.897515    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gvv6m" podUID="adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a"
Jul 22 02:20:31.897888 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:31.897583    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsnsc" podUID="cd5d3980-4fea-40a0-b7fd-e5a28d013ef3"
Jul 22 02:20:31.897888 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:31.897652    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bsw8" podUID="fa01c319-94d5-4113-9457-6b808c24dee5"
Jul 22 02:20:33.898085 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:33.898036    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:20:33.898085 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:33.898092    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:20:33.898586 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:33.898061    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:20:33.898586 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:33.898174    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gvv6m" podUID="adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a"
Jul 22 02:20:33.898586 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:33.898215    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsnsc" podUID="cd5d3980-4fea-40a0-b7fd-e5a28d013ef3"
Jul 22 02:20:33.898586 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:33.898294    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bsw8" podUID="fa01c319-94d5-4113-9457-6b808c24dee5"
Jul 22 02:20:35.447228 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:35.447191    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:35.549941 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:35.549893    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:35.648169 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:35.648137    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:35.748491 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:35.748459    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:35.897588 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:35.897552    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:20:35.897825 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:35.897553    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:20:35.897825 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:35.897658    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsnsc" podUID="cd5d3980-4fea-40a0-b7fd-e5a28d013ef3"
Jul 22 02:20:35.897825 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:35.897685    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:20:35.897825 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:35.897808    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gvv6m" podUID="adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a"
Jul 22 02:20:35.898056 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:35.897867    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bsw8" podUID="fa01c319-94d5-4113-9457-6b808c24dee5"
Jul 22 02:20:36.742608 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:36.742581    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-202.ec2.internal" event="NodeReady"
Jul 22 02:20:36.742997 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:36.742707    2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 22 02:20:36.801515 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:36.801478    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2mvpx"]
Jul 22 02:20:36.804361 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:36.804343    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jzzqf"]
Jul 22 02:20:36.804506 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:36.804486    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2mvpx"
Jul 22 02:20:36.807168 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:36.807143    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 22 02:20:36.807275 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:36.807212    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 22 02:20:36.807275 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:36.807230    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jzzqf"
Jul 22 02:20:36.807379 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:36.807229    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 22 02:20:36.807569 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:36.807557    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6sgmx\""
Jul 22 02:20:36.807612 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:36.807585    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 22 02:20:36.810149 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:36.810122    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4k4rg\""
Jul 22 02:20:36.810269 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:36.810122    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 22 02:20:36.810269 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:36.810205    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 22 02:20:36.815726 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:36.815703    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jzzqf"]
Jul 22 02:20:36.820527 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:36.820507    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2mvpx"]
Jul 22 02:20:36.915759 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:36.912928    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/efc56fd3-50cc-4228-8ecf-44eed0f5864e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2mvpx\" (UID: \"efc56fd3-50cc-4228-8ecf-44eed0f5864e\") " pod="openshift-insights/insights-runtime-extractor-2mvpx"
Jul 22 02:20:36.915759 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:36.912990    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqdwg\" (UniqueName: \"kubernetes.io/projected/efc56fd3-50cc-4228-8ecf-44eed0f5864e-kube-api-access-pqdwg\") pod \"insights-runtime-extractor-2mvpx\" (UID: \"efc56fd3-50cc-4228-8ecf-44eed0f5864e\") " pod="openshift-insights/insights-runtime-extractor-2mvpx"
Jul 22 02:20:36.915759 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:36.913048    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/efc56fd3-50cc-4228-8ecf-44eed0f5864e-crio-socket\") pod \"insights-runtime-extractor-2mvpx\" (UID: \"efc56fd3-50cc-4228-8ecf-44eed0f5864e\") " pod="openshift-insights/insights-runtime-extractor-2mvpx"
Jul 22 02:20:36.915759 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:36.913082    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/62cd0167-5561-4aca-8c78-a1f7c526e34e-config-volume\") pod \"dns-default-jzzqf\" (UID: \"62cd0167-5561-4aca-8c78-a1f7c526e34e\") " pod="openshift-dns/dns-default-jzzqf"
Jul 22 02:20:36.915759 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:36.913118    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/efc56fd3-50cc-4228-8ecf-44eed0f5864e-data-volume\") pod \"insights-runtime-extractor-2mvpx\" (UID: \"efc56fd3-50cc-4228-8ecf-44eed0f5864e\") " pod="openshift-insights/insights-runtime-extractor-2mvpx"
Jul 22 02:20:36.915759 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:36.913147    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/62cd0167-5561-4aca-8c78-a1f7c526e34e-metrics-tls\") pod \"dns-default-jzzqf\" (UID: \"62cd0167-5561-4aca-8c78-a1f7c526e34e\") " pod="openshift-dns/dns-default-jzzqf"
Jul 22 02:20:36.915759 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:36.913214    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/efc56fd3-50cc-4228-8ecf-44eed0f5864e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2mvpx\" (UID: \"efc56fd3-50cc-4228-8ecf-44eed0f5864e\") " pod="openshift-insights/insights-runtime-extractor-2mvpx"
Jul 22 02:20:36.915759 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:36.913239    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62cd0167-5561-4aca-8c78-a1f7c526e34e-tmp-dir\") pod \"dns-default-jzzqf\" (UID: \"62cd0167-5561-4aca-8c78-a1f7c526e34e\") " pod="openshift-dns/dns-default-jzzqf"
Jul 22 02:20:36.915759 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:36.913274    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76xnj\" (UniqueName: \"kubernetes.io/projected/62cd0167-5561-4aca-8c78-a1f7c526e34e-kube-api-access-76xnj\") pod \"dns-default-jzzqf\" (UID: \"62cd0167-5561-4aca-8c78-a1f7c526e34e\") " pod="openshift-dns/dns-default-jzzqf"
Jul 22 02:20:36.915759 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:36.913867    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-w5t8r"]
Jul 22 02:20:36.917337 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:36.917320    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-w5t8r"
Jul 22 02:20:36.920230 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:36.920211    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 22 02:20:36.920390 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:36.920373    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 22 02:20:36.920643 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:36.920627    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ptnrq\""
Jul 22 02:20:36.920718 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:36.920632    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 22 02:20:36.928141 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:36.928117    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-w5t8r"]
Jul 22 02:20:36.995898 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:36.995805    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:37.014012 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.013986    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/efc56fd3-50cc-4228-8ecf-44eed0f5864e-crio-socket\") pod \"insights-runtime-extractor-2mvpx\" (UID: \"efc56fd3-50cc-4228-8ecf-44eed0f5864e\") " pod="openshift-insights/insights-runtime-extractor-2mvpx"
Jul 22 02:20:37.014101 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.014018    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/62cd0167-5561-4aca-8c78-a1f7c526e34e-config-volume\") pod \"dns-default-jzzqf\" (UID: \"62cd0167-5561-4aca-8c78-a1f7c526e34e\") " pod="openshift-dns/dns-default-jzzqf"
Jul 22 02:20:37.014101 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.014047    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/efc56fd3-50cc-4228-8ecf-44eed0f5864e-data-volume\") pod \"insights-runtime-extractor-2mvpx\" (UID: \"efc56fd3-50cc-4228-8ecf-44eed0f5864e\") " pod="openshift-insights/insights-runtime-extractor-2mvpx"
Jul 22 02:20:37.014101 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.014069    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/62cd0167-5561-4aca-8c78-a1f7c526e34e-metrics-tls\") pod \"dns-default-jzzqf\" (UID: \"62cd0167-5561-4aca-8c78-a1f7c526e34e\") " pod="openshift-dns/dns-default-jzzqf"
Jul 22 02:20:37.014247 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.014109    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/efc56fd3-50cc-4228-8ecf-44eed0f5864e-crio-socket\") pod \"insights-runtime-extractor-2mvpx\" (UID: \"efc56fd3-50cc-4228-8ecf-44eed0f5864e\") " pod="openshift-insights/insights-runtime-extractor-2mvpx"
Jul 22 02:20:37.014247 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.014118    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0311047b-64b1-4a99-94f6-8cd2571c2d4c-cert\") pod \"ingress-canary-w5t8r\" (UID: \"0311047b-64b1-4a99-94f6-8cd2571c2d4c\") " pod="openshift-ingress-canary/ingress-canary-w5t8r"
Jul 22 02:20:37.014247 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.014178    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/efc56fd3-50cc-4228-8ecf-44eed0f5864e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2mvpx\" (UID: \"efc56fd3-50cc-4228-8ecf-44eed0f5864e\") " pod="openshift-insights/insights-runtime-extractor-2mvpx"
Jul 22 02:20:37.014247 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.014205    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62cd0167-5561-4aca-8c78-a1f7c526e34e-tmp-dir\") pod \"dns-default-jzzqf\" (UID: \"62cd0167-5561-4aca-8c78-a1f7c526e34e\") " pod="openshift-dns/dns-default-jzzqf"
Jul 22 02:20:37.014247 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.014227    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxqfc\" (UniqueName: \"kubernetes.io/projected/0311047b-64b1-4a99-94f6-8cd2571c2d4c-kube-api-access-kxqfc\") pod \"ingress-canary-w5t8r\" (UID: \"0311047b-64b1-4a99-94f6-8cd2571c2d4c\") " pod="openshift-ingress-canary/ingress-canary-w5t8r"
Jul 22 02:20:37.014493 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.014262    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76xnj\" (UniqueName: \"kubernetes.io/projected/62cd0167-5561-4aca-8c78-a1f7c526e34e-kube-api-access-76xnj\") pod \"dns-default-jzzqf\" (UID: \"62cd0167-5561-4aca-8c78-a1f7c526e34e\") " pod="openshift-dns/dns-default-jzzqf"
Jul 22 02:20:37.014493 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.014396    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/efc56fd3-50cc-4228-8ecf-44eed0f5864e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2mvpx\" (UID: \"efc56fd3-50cc-4228-8ecf-44eed0f5864e\") " pod="openshift-insights/insights-runtime-extractor-2mvpx"
Jul 22 02:20:37.014493 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.014437    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqdwg\" (UniqueName: \"kubernetes.io/projected/efc56fd3-50cc-4228-8ecf-44eed0f5864e-kube-api-access-pqdwg\") pod \"insights-runtime-extractor-2mvpx\" (UID: \"efc56fd3-50cc-4228-8ecf-44eed0f5864e\") " pod="openshift-insights/insights-runtime-extractor-2mvpx"
Jul 22 02:20:37.014625 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.014399    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/efc56fd3-50cc-4228-8ecf-44eed0f5864e-data-volume\") pod \"insights-runtime-extractor-2mvpx\" (UID: \"efc56fd3-50cc-4228-8ecf-44eed0f5864e\") " pod="openshift-insights/insights-runtime-extractor-2mvpx"
Jul 22 02:20:37.014625 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.014542    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62cd0167-5561-4aca-8c78-a1f7c526e34e-tmp-dir\") pod \"dns-default-jzzqf\" (UID: \"62cd0167-5561-4aca-8c78-a1f7c526e34e\") " pod="openshift-dns/dns-default-jzzqf"
Jul 22 02:20:37.014776 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.014759    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/efc56fd3-50cc-4228-8ecf-44eed0f5864e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2mvpx\" (UID: \"efc56fd3-50cc-4228-8ecf-44eed0f5864e\") " pod="openshift-insights/insights-runtime-extractor-2mvpx"
Jul 22 02:20:37.014844 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.014772    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/62cd0167-5561-4aca-8c78-a1f7c526e34e-config-volume\") pod \"dns-default-jzzqf\" (UID: \"62cd0167-5561-4aca-8c78-a1f7c526e34e\") " pod="openshift-dns/dns-default-jzzqf"
Jul 22 02:20:37.018260 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.018235    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/62cd0167-5561-4aca-8c78-a1f7c526e34e-metrics-tls\") pod \"dns-default-jzzqf\" (UID: \"62cd0167-5561-4aca-8c78-a1f7c526e34e\") " pod="openshift-dns/dns-default-jzzqf"
Jul 22 02:20:37.018260 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.018235    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/efc56fd3-50cc-4228-8ecf-44eed0f5864e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2mvpx\" (UID: \"efc56fd3-50cc-4228-8ecf-44eed0f5864e\") " pod="openshift-insights/insights-runtime-extractor-2mvpx"
Jul 22 02:20:37.023412 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.023385    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76xnj\" (UniqueName: \"kubernetes.io/projected/62cd0167-5561-4aca-8c78-a1f7c526e34e-kube-api-access-76xnj\") pod \"dns-default-jzzqf\" (UID: \"62cd0167-5561-4aca-8c78-a1f7c526e34e\") " pod="openshift-dns/dns-default-jzzqf"
Jul 22 02:20:37.023552 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.023536    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqdwg\" (UniqueName: \"kubernetes.io/projected/efc56fd3-50cc-4228-8ecf-44eed0f5864e-kube-api-access-pqdwg\") pod \"insights-runtime-extractor-2mvpx\" (UID: \"efc56fd3-50cc-4228-8ecf-44eed0f5864e\") " pod="openshift-insights/insights-runtime-extractor-2mvpx"
Jul 22 02:20:37.062390 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.062359    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:37.114551 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.114514    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2mvpx"
Jul 22 02:20:37.114839 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.114816    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0311047b-64b1-4a99-94f6-8cd2571c2d4c-cert\") pod \"ingress-canary-w5t8r\" (UID: \"0311047b-64b1-4a99-94f6-8cd2571c2d4c\") " pod="openshift-ingress-canary/ingress-canary-w5t8r"
Jul 22 02:20:37.114905 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.114851    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxqfc\" (UniqueName: \"kubernetes.io/projected/0311047b-64b1-4a99-94f6-8cd2571c2d4c-kube-api-access-kxqfc\") pod \"ingress-canary-w5t8r\" (UID: \"0311047b-64b1-4a99-94f6-8cd2571c2d4c\") " pod="openshift-ingress-canary/ingress-canary-w5t8r"
Jul 22 02:20:37.117214 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.117197    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0311047b-64b1-4a99-94f6-8cd2571c2d4c-cert\") pod \"ingress-canary-w5t8r\" (UID: \"0311047b-64b1-4a99-94f6-8cd2571c2d4c\") " pod="openshift-ingress-canary/ingress-canary-w5t8r"
Jul 22 02:20:37.121110 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.120937    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jzzqf"
Jul 22 02:20:37.124986 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.124963    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxqfc\" (UniqueName: \"kubernetes.io/projected/0311047b-64b1-4a99-94f6-8cd2571c2d4c-kube-api-access-kxqfc\") pod \"ingress-canary-w5t8r\" (UID: \"0311047b-64b1-4a99-94f6-8cd2571c2d4c\") " pod="openshift-ingress-canary/ingress-canary-w5t8r"
Jul 22 02:20:37.242014 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.228878    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-w5t8r"
Jul 22 02:20:37.289847 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.289791    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jzzqf"]
Jul 22 02:20:37.294034 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:20:37.294007    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62cd0167_5561_4aca_8c78_a1f7c526e34e.slice/crio-b4ae562393516a68dc1794e1bc46488d0eedcb33f69354d247af18c2a05634fc WatchSource:0}: Error finding container b4ae562393516a68dc1794e1bc46488d0eedcb33f69354d247af18c2a05634fc: Status 404 returned error can't find the container with id b4ae562393516a68dc1794e1bc46488d0eedcb33f69354d247af18c2a05634fc
Jul 22 02:20:37.299886 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.299856    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2mvpx"]
Jul 22 02:20:37.372426 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.372385    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-w5t8r"]
Jul 22 02:20:37.375646 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:20:37.375623    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0311047b_64b1_4a99_94f6_8cd2571c2d4c.slice/crio-8e3c11e8b3a64103bc06e673543b9ad39b2594ca694e093e8db77ab9d71944ca WatchSource:0}: Error finding container 8e3c11e8b3a64103bc06e673543b9ad39b2594ca694e093e8db77ab9d71944ca: Status 404 returned error can't find the container with id 8e3c11e8b3a64103bc06e673543b9ad39b2594ca694e093e8db77ab9d71944ca
Jul 22 02:20:37.898160 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.898119    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:20:37.898602 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.898174    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:20:37.898602 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.898132    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:20:37.901819 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.901620    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 22 02:20:37.903174 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.903152    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 22 02:20:37.903331 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.903309    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 22 02:20:37.903455 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.903435    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t8kv7\""
Jul 22 02:20:37.903577 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.903563    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 22 02:20:37.903668 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:37.903653    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8wzn5\""
Jul 22 02:20:38.228412 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:38.228326    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2mvpx" event={"ID":"efc56fd3-50cc-4228-8ecf-44eed0f5864e","Type":"ContainerStarted","Data":"9f70d7cd8790ab7d8aace5e518762a4bcb57709bf9b075ad8424a0528d73005c"}
Jul 22 02:20:38.230132 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:38.230058    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-w5t8r" event={"ID":"0311047b-64b1-4a99-94f6-8cd2571c2d4c","Type":"ContainerStarted","Data":"8e3c11e8b3a64103bc06e673543b9ad39b2594ca694e093e8db77ab9d71944ca"}
Jul 22 02:20:38.232682 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:38.232646    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jzzqf" event={"ID":"62cd0167-5561-4aca-8c78-a1f7c526e34e","Type":"ContainerStarted","Data":"b4ae562393516a68dc1794e1bc46488d0eedcb33f69354d247af18c2a05634fc"}
Jul 22 02:20:38.990075 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:38.990008    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:39.094632 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:39.094594    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:39.190520 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:39.190444    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:39.264842 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:39.264749    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:39.335133 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:39.335079    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:39.395019 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:39.394989    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:39.798789 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:39.798731    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:39.894908 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:39.894864    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:39.953936 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:39.953867    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-5kbp2"]
Jul 22 02:20:39.972279 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:39.972241    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:39.983344 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:39.983316    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-5kbp2"]
Jul 22 02:20:39.983502 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:39.983416    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-5kbp2"
Jul 22 02:20:39.987441 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:39.987398    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 22 02:20:39.987598 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:39.987548    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-rtw4b\""
Jul 22 02:20:40.037135 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:40.037087    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9edede50-1dd5-4ba2-87af-4b51e52b9ea5-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-5kbp2\" (UID: \"9edede50-1dd5-4ba2-87af-4b51e52b9ea5\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-5kbp2"
Jul 22 02:20:40.138191 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:40.138107    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9edede50-1dd5-4ba2-87af-4b51e52b9ea5-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-5kbp2\" (UID: \"9edede50-1dd5-4ba2-87af-4b51e52b9ea5\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-5kbp2"
Jul 22 02:20:40.138349 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:40.138291    2581 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 22 02:20:40.138420 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:20:40.138393    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9edede50-1dd5-4ba2-87af-4b51e52b9ea5-tls-certificates podName:9edede50-1dd5-4ba2-87af-4b51e52b9ea5 nodeName:}" failed. No retries permitted until 2026-07-22 02:20:40.638370683 +0000 UTC m=+62.179164306 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/9edede50-1dd5-4ba2-87af-4b51e52b9ea5-tls-certificates") pod "prometheus-operator-admission-webhook-85574b5f49-5kbp2" (UID: "9edede50-1dd5-4ba2-87af-4b51e52b9ea5") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 22 02:20:40.161110 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:40.161080    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b78fw"
Jul 22 02:20:40.292305 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:40.292253    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:40.352087 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:40.352051    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:40.469911 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:40.469872    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:40.569698 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:40.569650    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:40.642510 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:40.642478    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9edede50-1dd5-4ba2-87af-4b51e52b9ea5-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-5kbp2\" (UID: \"9edede50-1dd5-4ba2-87af-4b51e52b9ea5\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-5kbp2"
Jul 22 02:20:40.646823 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:40.646388    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9edede50-1dd5-4ba2-87af-4b51e52b9ea5-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-5kbp2\" (UID: \"9edede50-1dd5-4ba2-87af-4b51e52b9ea5\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-5kbp2"
Jul 22 02:20:40.895380 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:40.895344    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-5kbp2"
Jul 22 02:20:42.864687 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:42.864655    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:42.918883 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:42.918843    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:43.201832 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:43.201722    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-5kbp2"]
Jul 22 02:20:43.207638 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:20:43.207603    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9edede50_1dd5_4ba2_87af_4b51e52b9ea5.slice/crio-b0ed4ca22e092f758b21e0af948826bfa0204ec45281911e97c371db4ffd8d06 WatchSource:0}: Error finding container b0ed4ca22e092f758b21e0af948826bfa0204ec45281911e97c371db4ffd8d06: Status 404 returned error can't find the container with id b0ed4ca22e092f758b21e0af948826bfa0204ec45281911e97c371db4ffd8d06
Jul 22 02:20:43.248255 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:43.248218    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-5kbp2" event={"ID":"9edede50-1dd5-4ba2-87af-4b51e52b9ea5","Type":"ContainerStarted","Data":"b0ed4ca22e092f758b21e0af948826bfa0204ec45281911e97c371db4ffd8d06"}
Jul 22 02:20:43.249797 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:43.249767    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2mvpx" event={"ID":"efc56fd3-50cc-4228-8ecf-44eed0f5864e","Type":"ContainerStarted","Data":"7e6ec840d5d9907ff68ceaf90689122fbebca3318b0eaf4ba205c7a215099f9f"}
Jul 22 02:20:43.251151 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:43.251123    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-w5t8r" event={"ID":"0311047b-64b1-4a99-94f6-8cd2571c2d4c","Type":"ContainerStarted","Data":"1dd8f06bc5f2911a6034d91157208f816f7189c01eaebb61234edd59ea27238a"}
Jul 22 02:20:43.253520 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:43.253473    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jzzqf" event={"ID":"62cd0167-5561-4aca-8c78-a1f7c526e34e","Type":"ContainerStarted","Data":"c38e67b2b0d26ca8e31d4c2787cdd5767e2d11ffb6730e345f0da35c693fc206"}
Jul 22 02:20:43.272578 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:43.272519    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-w5t8r" podStartSLOduration=1.5980679709999999 podStartE2EDuration="7.272502371s" totalImagesPullingTime="5.6744344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:20:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:43.272443957 +0000 UTC m=+64.813237584" watchObservedRunningTime="2026-07-22 02:20:43.272502371 +0000 UTC m=+64.813295998"
Jul 22 02:20:44.259856 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:44.259814    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jzzqf" event={"ID":"62cd0167-5561-4aca-8c78-a1f7c526e34e","Type":"ContainerStarted","Data":"cb8d844055fcacd0e9a635d1f7e3afe4eeeefb62aefa889c986c4688d5a679ad"}
Jul 22 02:20:44.260337 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:44.259925    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jzzqf"
Jul 22 02:20:44.278596 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:44.278503    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-jzzqf" podStartSLOduration=2.525524895 podStartE2EDuration="8.278480838s" totalImagesPullingTime="5.752955943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:20:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:44.277853408 +0000 UTC m=+65.818647035" watchObservedRunningTime="2026-07-22 02:20:44.278480838 +0000 UTC m=+65.819274466"
Jul 22 02:20:44.369599 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:44.369528    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:44.470297 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:44.470223    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:44.578052 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:44.577966    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzz6v\" (UniqueName: \"kubernetes.io/projected/cd5d3980-4fea-40a0-b7fd-e5a28d013ef3-kube-api-access-hzz6v\") pod \"network-check-target-bsnsc\" (UID: \"cd5d3980-4fea-40a0-b7fd-e5a28d013ef3\") " pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:20:44.578052 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:44.578021    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa01c319-94d5-4113-9457-6b808c24dee5-metrics-certs\") pod \"network-metrics-daemon-6bsw8\" (UID: \"fa01c319-94d5-4113-9457-6b808c24dee5\") " pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:20:44.581234 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:44.581178    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 22 02:20:44.581364 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:44.581238    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 22 02:20:44.592090 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:44.592060    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa01c319-94d5-4113-9457-6b808c24dee5-metrics-certs\") pod \"network-metrics-daemon-6bsw8\" (UID: \"fa01c319-94d5-4113-9457-6b808c24dee5\") " pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:20:44.596027 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:44.595986    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 22 02:20:44.602403 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:44.602379    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzz6v\" (UniqueName: \"kubernetes.io/projected/cd5d3980-4fea-40a0-b7fd-e5a28d013ef3-kube-api-access-hzz6v\") pod \"network-check-target-bsnsc\" (UID: \"cd5d3980-4fea-40a0-b7fd-e5a28d013ef3\") " pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:20:44.678461 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:44.678421    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a-original-pull-secret\") pod \"global-pull-secret-syncer-gvv6m\" (UID: \"adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a\") " pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:20:44.681707 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:44.681680    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 22 02:20:44.691890 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:44.691868    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a-original-pull-secret\") pod \"global-pull-secret-syncer-gvv6m\" (UID: \"adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a\") " pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:20:44.815934 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:44.815089    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t8kv7\""
Jul 22 02:20:44.822791 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:44.822763    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bsw8"
Jul 22 02:20:44.823311 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:44.823292    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8wzn5\""
Jul 22 02:20:44.828912 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:44.828828    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gvv6m"
Jul 22 02:20:44.831457 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:44.831436    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:20:44.978147 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:44.978115    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6bsw8"]
Jul 22 02:20:44.984056 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:20:44.984027    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa01c319_94d5_4113_9457_6b808c24dee5.slice/crio-6b2492b6ea7ab7214e4fc8aaa78f1ba153b4221a54182b86bff4be7f0e8ece8a WatchSource:0}: Error finding container 6b2492b6ea7ab7214e4fc8aaa78f1ba153b4221a54182b86bff4be7f0e8ece8a: Status 404 returned error can't find the container with id 6b2492b6ea7ab7214e4fc8aaa78f1ba153b4221a54182b86bff4be7f0e8ece8a
Jul 22 02:20:45.002998 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:45.002971    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bsnsc"]
Jul 22 02:20:45.005853 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:20:45.005825    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd5d3980_4fea_40a0_b7fd_e5a28d013ef3.slice/crio-edadf0f7faa426301c0b1d8934430d144924bd59c57780683e296c9306e8f59e WatchSource:0}: Error finding container edadf0f7faa426301c0b1d8934430d144924bd59c57780683e296c9306e8f59e: Status 404 returned error can't find the container with id edadf0f7faa426301c0b1d8934430d144924bd59c57780683e296c9306e8f59e
Jul 22 02:20:45.206507 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:45.206388    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gvv6m"]
Jul 22 02:20:45.263974 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:45.263935    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bsnsc" event={"ID":"cd5d3980-4fea-40a0-b7fd-e5a28d013ef3","Type":"ContainerStarted","Data":"edadf0f7faa426301c0b1d8934430d144924bd59c57780683e296c9306e8f59e"}
Jul 22 02:20:45.265286 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:45.265259    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6bsw8" event={"ID":"fa01c319-94d5-4113-9457-6b808c24dee5","Type":"ContainerStarted","Data":"6b2492b6ea7ab7214e4fc8aaa78f1ba153b4221a54182b86bff4be7f0e8ece8a"}
Jul 22 02:20:45.267119 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:45.267093    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-5kbp2" event={"ID":"9edede50-1dd5-4ba2-87af-4b51e52b9ea5","Type":"ContainerStarted","Data":"4ea5a05d2d1830e3e13096128cccfe479770a3a9fd76eda06ea71ef6821a4544"}
Jul 22 02:20:45.267577 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:45.267527    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-5kbp2"
Jul 22 02:20:45.272754 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:45.272717    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-5kbp2"
Jul 22 02:20:45.283750 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:45.283686    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-5kbp2" podStartSLOduration=4.80614612 podStartE2EDuration="6.283671412s" totalImagesPullingTime="1.477525292s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:20:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:45.282989856 +0000 UTC m=+66.823783484" watchObservedRunningTime="2026-07-22 02:20:45.283671412 +0000 UTC m=+66.824465016"
Jul 22 02:20:45.985535 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:45.985500    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:46.015810 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:46.015773    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-nvxw2"]
Jul 22 02:20:46.082153 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:46.082113    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:46.167064 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:46.167027    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-nvxw2"]
Jul 22 02:20:46.167260 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:46.167154    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-nvxw2"
Jul 22 02:20:46.170358 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:46.170330    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 22 02:20:46.170505 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:46.170460    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 22 02:20:46.170505 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:46.170478    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 22 02:20:46.170629 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:46.170580    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 22 02:20:46.171492 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:46.171477    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 22 02:20:46.171917 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:46.171838    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-5x2zs\""
Jul 22 02:20:46.181941 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:46.181918    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:46.290542 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:46.290452    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64btk\" (UniqueName: \"kubernetes.io/projected/039210ef-e3d8-497e-8553-8760df40860a-kube-api-access-64btk\") pod \"prometheus-operator-5945b5cfd-nvxw2\" (UID: \"039210ef-e3d8-497e-8553-8760df40860a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-nvxw2"
Jul 22 02:20:46.290542 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:46.290496    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/039210ef-e3d8-497e-8553-8760df40860a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-nvxw2\" (UID: \"039210ef-e3d8-497e-8553-8760df40860a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-nvxw2"
Jul 22 02:20:46.290542 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:46.290527    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/039210ef-e3d8-497e-8553-8760df40860a-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-nvxw2\" (UID: \"039210ef-e3d8-497e-8553-8760df40860a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-nvxw2"
Jul 22 02:20:46.291272 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:46.290612    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/039210ef-e3d8-497e-8553-8760df40860a-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-nvxw2\" (UID: \"039210ef-e3d8-497e-8553-8760df40860a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-nvxw2"
Jul 22 02:20:46.391867 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:46.391828    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/039210ef-e3d8-497e-8553-8760df40860a-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-nvxw2\" (UID: \"039210ef-e3d8-497e-8553-8760df40860a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-nvxw2"
Jul 22 02:20:46.392030 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:46.391920    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64btk\" (UniqueName: \"kubernetes.io/projected/039210ef-e3d8-497e-8553-8760df40860a-kube-api-access-64btk\") pod \"prometheus-operator-5945b5cfd-nvxw2\" (UID: \"039210ef-e3d8-497e-8553-8760df40860a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-nvxw2"
Jul 22 02:20:46.392030 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:46.391955    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/039210ef-e3d8-497e-8553-8760df40860a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-nvxw2\" (UID: \"039210ef-e3d8-497e-8553-8760df40860a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-nvxw2"
Jul 22 02:20:46.392030 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:46.391996    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/039210ef-e3d8-497e-8553-8760df40860a-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-nvxw2\" (UID: \"039210ef-e3d8-497e-8553-8760df40860a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-nvxw2"
Jul 22 02:20:46.392641 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:46.392617    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/039210ef-e3d8-497e-8553-8760df40860a-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-nvxw2\" (UID: \"039210ef-e3d8-497e-8553-8760df40860a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-nvxw2"
Jul 22 02:20:46.394214 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:46.394191    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/039210ef-e3d8-497e-8553-8760df40860a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-nvxw2\" (UID: \"039210ef-e3d8-497e-8553-8760df40860a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-nvxw2"
Jul 22 02:20:46.394306 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:46.394270    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/039210ef-e3d8-497e-8553-8760df40860a-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-nvxw2\" (UID: \"039210ef-e3d8-497e-8553-8760df40860a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-nvxw2"
Jul 22 02:20:46.401190 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:46.401166    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64btk\" (UniqueName: \"kubernetes.io/projected/039210ef-e3d8-497e-8553-8760df40860a-kube-api-access-64btk\") pod \"prometheus-operator-5945b5cfd-nvxw2\" (UID: \"039210ef-e3d8-497e-8553-8760df40860a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-nvxw2"
Jul 22 02:20:46.477760 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:46.477704    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-nvxw2"
Jul 22 02:20:46.566134 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:20:46.566055    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadabd7dc_4255_4a59_ab43_a2b2d5aa5a5a.slice/crio-91f6688ea16f1b52801e4b5724e239b2b4885b5e11507f987eecfc549b3b5424 WatchSource:0}: Error finding container 91f6688ea16f1b52801e4b5724e239b2b4885b5e11507f987eecfc549b3b5424: Status 404 returned error can't find the container with id 91f6688ea16f1b52801e4b5724e239b2b4885b5e11507f987eecfc549b3b5424
Jul 22 02:20:47.275540 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:47.275462    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gvv6m" event={"ID":"adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a","Type":"ContainerStarted","Data":"91f6688ea16f1b52801e4b5724e239b2b4885b5e11507f987eecfc549b3b5424"}
Jul 22 02:20:47.656852 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:47.656818    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-nvxw2"]
Jul 22 02:20:47.845455 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:20:47.845365    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod039210ef_e3d8_497e_8553_8760df40860a.slice/crio-e6d5c8338e85506b42792d38245c7be5b556935ecc5b6e5c649bced711200e51 WatchSource:0}: Error finding container e6d5c8338e85506b42792d38245c7be5b556935ecc5b6e5c649bced711200e51: Status 404 returned error can't find the container with id e6d5c8338e85506b42792d38245c7be5b556935ecc5b6e5c649bced711200e51
Jul 22 02:20:48.281470 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:48.281423    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6bsw8" event={"ID":"fa01c319-94d5-4113-9457-6b808c24dee5","Type":"ContainerStarted","Data":"ca6447f23ab13179c75958d234b27b2754e865fe28b4cb51e3eef2eb2b07a3f2"}
Jul 22 02:20:48.281661 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:48.281530    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6bsw8" event={"ID":"fa01c319-94d5-4113-9457-6b808c24dee5","Type":"ContainerStarted","Data":"fa65b9a791bfc7d5e05d526243fd99f9da49265f7eda23cee62247dee64f30b4"}
Jul 22 02:20:48.283403 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:48.283339    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-nvxw2" event={"ID":"039210ef-e3d8-497e-8553-8760df40860a","Type":"ContainerStarted","Data":"e6d5c8338e85506b42792d38245c7be5b556935ecc5b6e5c649bced711200e51"}
Jul 22 02:20:48.285910 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:48.285876    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2mvpx" event={"ID":"efc56fd3-50cc-4228-8ecf-44eed0f5864e","Type":"ContainerStarted","Data":"0a041795b6624d0b4bc746273cc1c0f26cc917f1e40c0ebc602cf2086c400178"}
Jul 22 02:20:48.305182 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:48.305122    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-6bsw8" podStartSLOduration=67.397555259 podStartE2EDuration="1m10.30510286s" totalImagesPullingTime="2.907547601s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:19:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:48.302964832 +0000 UTC m=+69.843758461" watchObservedRunningTime="2026-07-22 02:20:48.30510286 +0000 UTC m=+69.845896488"
Jul 22 02:20:48.426858 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:48.426785    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:48.524729 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:48.524693    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:51.386191 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:51.386157    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:51.448756 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:51.448688    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:51.952732 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:51.952675    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:52.053687 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:52.053646    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:53.774843 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:53.774807    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:54.268999 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:54.268968    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jzzqf"
Jul 22 02:20:54.310458 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:54.310422    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gvv6m" event={"ID":"adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a","Type":"ContainerStarted","Data":"8b837d0901cab067ffa7215697469d376a15a282f48f782af71beac502f12189"}
Jul 22 02:20:54.312502 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:54.312471    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-nvxw2" event={"ID":"039210ef-e3d8-497e-8553-8760df40860a","Type":"ContainerStarted","Data":"cec2c57ed3acc167178a68cd0164f0b2dc517ee588fcaf327d75ab21608f17f6"}
Jul 22 02:20:54.312639 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:54.312530    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-nvxw2" event={"ID":"039210ef-e3d8-497e-8553-8760df40860a","Type":"ContainerStarted","Data":"48c5b63aebe948f75dc82abc0167be5367f1da36586f8d3b54689c36fc26ec75"}
Jul 22 02:20:54.314791 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:54.314768    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2mvpx" event={"ID":"efc56fd3-50cc-4228-8ecf-44eed0f5864e","Type":"ContainerStarted","Data":"1afb70e3864bf5bf65afbded1d88d53a9663bb31731ee3e7cb064d6a71489d9e"}
Jul 22 02:20:54.316245 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:54.316223    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bsnsc" event={"ID":"cd5d3980-4fea-40a0-b7fd-e5a28d013ef3","Type":"ContainerStarted","Data":"17987a4b8c3cda3dfd47e3ab918dae6f33c362cd726a30dc4a1779a141b628c1"}
Jul 22 02:20:54.316386 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:54.316370    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:20:54.328565 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:54.328521    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-gvv6m" podStartSLOduration=68.422551928 podStartE2EDuration="1m15.328507004s" totalImagesPullingTime="6.905955076s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:19:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:54.327338039 +0000 UTC m=+75.868131677" watchObservedRunningTime="2026-07-22 02:20:54.328507004 +0000 UTC m=+75.869300621"
Jul 22 02:20:54.345878 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:54.345833    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-nvxw2" podStartSLOduration=3.732350377 podStartE2EDuration="9.345818621s" totalImagesPullingTime="5.613468244s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:20:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:54.344675894 +0000 UTC m=+75.885469521" watchObservedRunningTime="2026-07-22 02:20:54.345818621 +0000 UTC m=+75.886612248"
Jul 22 02:20:54.361800 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:54.361755    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-bsnsc" podStartSLOduration=67.907118885 podStartE2EDuration="1m16.361719278s" totalImagesPullingTime="8.454600393s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:19:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:54.361267056 +0000 UTC m=+75.902060684" watchObservedRunningTime="2026-07-22 02:20:54.361719278 +0000 UTC m=+75.902512920"
Jul 22 02:20:54.379779 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:54.379707    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2mvpx" podStartSLOduration=2.342650209 podStartE2EDuration="18.37969315s" totalImagesPullingTime="16.037042941s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:20:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:54.378285079 +0000 UTC m=+75.919078707" watchObservedRunningTime="2026-07-22 02:20:54.37969315 +0000 UTC m=+75.920486776"
Jul 22 02:20:54.422035 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:54.422002    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:54.492286 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:54.492256    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:56.138680 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.138631    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:56.228245 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.228206    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-dkd8c"]
Jul 22 02:20:56.231234 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.231210    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-dkd8c"
Jul 22 02:20:56.238273 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.238254    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 22 02:20:56.238387 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.238349    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-qhrbz\""
Jul 22 02:20:56.240170 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.240154    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 22 02:20:56.271079 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.271045    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-dkd8c"]
Jul 22 02:20:56.325955 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.325916    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:56.369991 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.369953    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vmxj\" (UniqueName: \"kubernetes.io/projected/121417f5-c17d-457c-a8f1-6ba982980cec-kube-api-access-4vmxj\") pod \"downloads-5bc67dcd46-dkd8c\" (UID: \"121417f5-c17d-457c-a8f1-6ba982980cec\") " pod="openshift-console/downloads-5bc67dcd46-dkd8c"
Jul 22 02:20:56.471368 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.471284    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vmxj\" (UniqueName: \"kubernetes.io/projected/121417f5-c17d-457c-a8f1-6ba982980cec-kube-api-access-4vmxj\") pod \"downloads-5bc67dcd46-dkd8c\" (UID: \"121417f5-c17d-457c-a8f1-6ba982980cec\") " pod="openshift-console/downloads-5bc67dcd46-dkd8c"
Jul 22 02:20:56.492787 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.492750    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vmxj\" (UniqueName: \"kubernetes.io/projected/121417f5-c17d-457c-a8f1-6ba982980cec-kube-api-access-4vmxj\") pod \"downloads-5bc67dcd46-dkd8c\" (UID: \"121417f5-c17d-457c-a8f1-6ba982980cec\") " pod="openshift-console/downloads-5bc67dcd46-dkd8c"
Jul 22 02:20:56.503537 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.503508    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:56.539535 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.539507    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-dkd8c"
Jul 22 02:20:56.614307 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.614281    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:56.667122 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.667088    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-v78rs"]
Jul 22 02:20:56.671517 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.671472    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-v78rs"
Jul 22 02:20:56.675272 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.675253    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-648ft\""
Jul 22 02:20:56.675403 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.675374    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 22 02:20:56.675585 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.675571    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 22 02:20:56.680819 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.680798    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mp9ts"]
Jul 22 02:20:56.683167 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.683153    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mp9ts"
Jul 22 02:20:56.690244 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.690217    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-dkd8c"]
Jul 22 02:20:56.690535 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.690498    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 22 02:20:56.690710 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.690698    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-d62vw\""
Jul 22 02:20:56.691002 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.690985    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 22 02:20:56.691076 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.691010    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 22 02:20:56.692521 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:20:56.692497    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod121417f5_c17d_457c_a8f1_6ba982980cec.slice/crio-ce5f028e03759d3276f71c42b2b19912f2175962f820a7f4d77d3a214079e3d1 WatchSource:0}: Error finding container ce5f028e03759d3276f71c42b2b19912f2175962f820a7f4d77d3a214079e3d1: Status 404 returned error can't find the container with id ce5f028e03759d3276f71c42b2b19912f2175962f820a7f4d77d3a214079e3d1
Jul 22 02:20:56.701319 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.701299    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-v78rs"]
Jul 22 02:20:56.774907 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.774874    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b8cfc23-4e2b-46cc-9c6d-82e27573d34c-metrics-client-ca\") pod \"node-exporter-mp9ts\" (UID: \"1b8cfc23-4e2b-46cc-9c6d-82e27573d34c\") " pod="openshift-monitoring/node-exporter-mp9ts"
Jul 22 02:20:56.775100 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.774920    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1b8cfc23-4e2b-46cc-9c6d-82e27573d34c-node-exporter-accelerators-collector-config\") pod \"node-exporter-mp9ts\" (UID: \"1b8cfc23-4e2b-46cc-9c6d-82e27573d34c\") " pod="openshift-monitoring/node-exporter-mp9ts"
Jul 22 02:20:56.775100 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.774940    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1b8cfc23-4e2b-46cc-9c6d-82e27573d34c-node-exporter-textfile\") pod \"node-exporter-mp9ts\" (UID: \"1b8cfc23-4e2b-46cc-9c6d-82e27573d34c\") " pod="openshift-monitoring/node-exporter-mp9ts"
Jul 22 02:20:56.775100 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.775001    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg97m\" (UniqueName: \"kubernetes.io/projected/1b8cfc23-4e2b-46cc-9c6d-82e27573d34c-kube-api-access-mg97m\") pod \"node-exporter-mp9ts\" (UID: \"1b8cfc23-4e2b-46cc-9c6d-82e27573d34c\") " pod="openshift-monitoring/node-exporter-mp9ts"
Jul 22 02:20:56.775100 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.775087    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1049839-4275-4715-bea3-230b58d93041-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-v78rs\" (UID: \"d1049839-4275-4715-bea3-230b58d93041\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-v78rs"
Jul 22 02:20:56.775326 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.775121    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1b8cfc23-4e2b-46cc-9c6d-82e27573d34c-sys\") pod \"node-exporter-mp9ts\" (UID: \"1b8cfc23-4e2b-46cc-9c6d-82e27573d34c\") " pod="openshift-monitoring/node-exporter-mp9ts"
Jul 22 02:20:56.775326 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.775146    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1b8cfc23-4e2b-46cc-9c6d-82e27573d34c-node-exporter-tls\") pod \"node-exporter-mp9ts\" (UID: \"1b8cfc23-4e2b-46cc-9c6d-82e27573d34c\") " pod="openshift-monitoring/node-exporter-mp9ts"
Jul 22 02:20:56.775326 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.775214    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qf842\" (UniqueName: \"kubernetes.io/projected/d1049839-4275-4715-bea3-230b58d93041-kube-api-access-qf842\") pod \"openshift-state-metrics-79f6958cdf-v78rs\" (UID: \"d1049839-4275-4715-bea3-230b58d93041\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-v78rs"
Jul 22 02:20:56.775326 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.775241    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d1049839-4275-4715-bea3-230b58d93041-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-v78rs\" (UID: \"d1049839-4275-4715-bea3-230b58d93041\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-v78rs"
Jul 22 02:20:56.775326 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.775268    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1b8cfc23-4e2b-46cc-9c6d-82e27573d34c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mp9ts\" (UID: \"1b8cfc23-4e2b-46cc-9c6d-82e27573d34c\") " pod="openshift-monitoring/node-exporter-mp9ts"
Jul 22 02:20:56.775326 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.775298    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1b8cfc23-4e2b-46cc-9c6d-82e27573d34c-root\") pod \"node-exporter-mp9ts\" (UID: \"1b8cfc23-4e2b-46cc-9c6d-82e27573d34c\") " pod="openshift-monitoring/node-exporter-mp9ts"
Jul 22 02:20:56.775500 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.775341    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1b8cfc23-4e2b-46cc-9c6d-82e27573d34c-node-exporter-wtmp\") pod \"node-exporter-mp9ts\" (UID: \"1b8cfc23-4e2b-46cc-9c6d-82e27573d34c\") " pod="openshift-monitoring/node-exporter-mp9ts"
Jul 22 02:20:56.775500 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.775363    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d1049839-4275-4715-bea3-230b58d93041-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-v78rs\" (UID: \"d1049839-4275-4715-bea3-230b58d93041\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-v78rs"
Jul 22 02:20:56.781852 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.781820    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-f2pg8"]
Jul 22 02:20:56.785238 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.785219    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-f2pg8"
Jul 22 02:20:56.785597 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.785576    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:56.788020 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.787998    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 22 02:20:56.788238 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.788219    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-q4pqt\""
Jul 22 02:20:56.788352 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.788242    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 22 02:20:56.788352 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.788330    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 22 02:20:56.798869 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.798844    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-f2pg8"]
Jul 22 02:20:56.867275 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.867244    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:56.875851 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.875826    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg97m\" (UniqueName: \"kubernetes.io/projected/1b8cfc23-4e2b-46cc-9c6d-82e27573d34c-kube-api-access-mg97m\") pod \"node-exporter-mp9ts\" (UID: \"1b8cfc23-4e2b-46cc-9c6d-82e27573d34c\") " pod="openshift-monitoring/node-exporter-mp9ts"
Jul 22 02:20:56.875930 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.875870    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1049839-4275-4715-bea3-230b58d93041-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-v78rs\" (UID: \"d1049839-4275-4715-bea3-230b58d93041\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-v78rs"
Jul 22 02:20:56.875930 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.875891    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1b8cfc23-4e2b-46cc-9c6d-82e27573d34c-sys\") pod \"node-exporter-mp9ts\" (UID: \"1b8cfc23-4e2b-46cc-9c6d-82e27573d34c\") " pod="openshift-monitoring/node-exporter-mp9ts"
Jul 22 02:20:56.875930 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.875908    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1b8cfc23-4e2b-46cc-9c6d-82e27573d34c-node-exporter-tls\") pod \"node-exporter-mp9ts\" (UID: \"1b8cfc23-4e2b-46cc-9c6d-82e27573d34c\") " pod="openshift-monitoring/node-exporter-mp9ts"
Jul 22 02:20:56.876084 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.875930    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/49ea4211-194d-4d3d-992d-20f3462d03b3-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-f2pg8\" (UID: \"49ea4211-194d-4d3d-992d-20f3462d03b3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-f2pg8"
Jul 22 02:20:56.876084 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.875966    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1b8cfc23-4e2b-46cc-9c6d-82e27573d34c-sys\") pod \"node-exporter-mp9ts\" (UID: \"1b8cfc23-4e2b-46cc-9c6d-82e27573d34c\") " pod="openshift-monitoring/node-exporter-mp9ts"
Jul 22 02:20:56.876084 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.876042    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zp9rd\" (UniqueName: \"kubernetes.io/projected/49ea4211-194d-4d3d-992d-20f3462d03b3-kube-api-access-zp9rd\") pod \"kube-state-metrics-56c5594674-f2pg8\" (UID: \"49ea4211-194d-4d3d-992d-20f3462d03b3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-f2pg8"
Jul 22 02:20:56.876084 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.876070    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qf842\" (UniqueName: \"kubernetes.io/projected/d1049839-4275-4715-bea3-230b58d93041-kube-api-access-qf842\") pod \"openshift-state-metrics-79f6958cdf-v78rs\" (UID: \"d1049839-4275-4715-bea3-230b58d93041\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-v78rs"
Jul 22 02:20:56.876275 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.876106    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d1049839-4275-4715-bea3-230b58d93041-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-v78rs\" (UID: \"d1049839-4275-4715-bea3-230b58d93041\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-v78rs"
Jul 22 02:20:56.876275 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.876144    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1b8cfc23-4e2b-46cc-9c6d-82e27573d34c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mp9ts\" (UID: \"1b8cfc23-4e2b-46cc-9c6d-82e27573d34c\") " pod="openshift-monitoring/node-exporter-mp9ts"
Jul 22 02:20:56.876275 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.876174    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1b8cfc23-4e2b-46cc-9c6d-82e27573d34c-root\") pod \"node-exporter-mp9ts\" (UID: \"1b8cfc23-4e2b-46cc-9c6d-82e27573d34c\") " pod="openshift-monitoring/node-exporter-mp9ts"
Jul 22 02:20:56.876275 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.876205    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1b8cfc23-4e2b-46cc-9c6d-82e27573d34c-node-exporter-wtmp\") pod \"node-exporter-mp9ts\" (UID: \"1b8cfc23-4e2b-46cc-9c6d-82e27573d34c\") " pod="openshift-monitoring/node-exporter-mp9ts"
Jul 22 02:20:56.876275 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.876233    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d1049839-4275-4715-bea3-230b58d93041-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-v78rs\" (UID: \"d1049839-4275-4715-bea3-230b58d93041\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-v78rs"
Jul 22 02:20:56.876275 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.876247    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1b8cfc23-4e2b-46cc-9c6d-82e27573d34c-root\") pod \"node-exporter-mp9ts\" (UID: \"1b8cfc23-4e2b-46cc-9c6d-82e27573d34c\") " pod="openshift-monitoring/node-exporter-mp9ts"
Jul 22 02:20:56.876275 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.876265    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b8cfc23-4e2b-46cc-9c6d-82e27573d34c-metrics-client-ca\") pod \"node-exporter-mp9ts\" (UID: \"1b8cfc23-4e2b-46cc-9c6d-82e27573d34c\") " pod="openshift-monitoring/node-exporter-mp9ts"
Jul 22 02:20:56.876599 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.876378    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1b8cfc23-4e2b-46cc-9c6d-82e27573d34c-node-exporter-wtmp\") pod \"node-exporter-mp9ts\" (UID: \"1b8cfc23-4e2b-46cc-9c6d-82e27573d34c\") " pod="openshift-monitoring/node-exporter-mp9ts"
Jul 22 02:20:56.876599 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.876397    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49ea4211-194d-4d3d-992d-20f3462d03b3-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-f2pg8\" (UID: \"49ea4211-194d-4d3d-992d-20f3462d03b3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-f2pg8"
Jul 22 02:20:56.876599 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.876463    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1b8cfc23-4e2b-46cc-9c6d-82e27573d34c-node-exporter-accelerators-collector-config\") pod \"node-exporter-mp9ts\" (UID: \"1b8cfc23-4e2b-46cc-9c6d-82e27573d34c\") " pod="openshift-monitoring/node-exporter-mp9ts"
Jul 22 02:20:56.876599 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.876499    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/49ea4211-194d-4d3d-992d-20f3462d03b3-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-f2pg8\" (UID: \"49ea4211-194d-4d3d-992d-20f3462d03b3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-f2pg8"
Jul 22 02:20:56.876599 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.876523    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/49ea4211-194d-4d3d-992d-20f3462d03b3-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-f2pg8\" (UID: \"49ea4211-194d-4d3d-992d-20f3462d03b3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-f2pg8"
Jul 22 02:20:56.876599 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.876559    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/49ea4211-194d-4d3d-992d-20f3462d03b3-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-f2pg8\" (UID: \"49ea4211-194d-4d3d-992d-20f3462d03b3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-f2pg8"
Jul 22 02:20:56.876599 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.876586    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1b8cfc23-4e2b-46cc-9c6d-82e27573d34c-node-exporter-textfile\") pod \"node-exporter-mp9ts\" (UID: \"1b8cfc23-4e2b-46cc-9c6d-82e27573d34c\") " pod="openshift-monitoring/node-exporter-mp9ts"
Jul 22 02:20:56.877021 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.876851    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b8cfc23-4e2b-46cc-9c6d-82e27573d34c-metrics-client-ca\") pod \"node-exporter-mp9ts\" (UID: \"1b8cfc23-4e2b-46cc-9c6d-82e27573d34c\") " pod="openshift-monitoring/node-exporter-mp9ts"
Jul 22 02:20:56.877021 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.876883    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d1049839-4275-4715-bea3-230b58d93041-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-v78rs\" (UID: \"d1049839-4275-4715-bea3-230b58d93041\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-v78rs"
Jul 22 02:20:56.877021 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.876990    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1b8cfc23-4e2b-46cc-9c6d-82e27573d34c-node-exporter-textfile\") pod \"node-exporter-mp9ts\" (UID: \"1b8cfc23-4e2b-46cc-9c6d-82e27573d34c\") " pod="openshift-monitoring/node-exporter-mp9ts"
Jul 22 02:20:56.877263 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.877245    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1b8cfc23-4e2b-46cc-9c6d-82e27573d34c-node-exporter-accelerators-collector-config\") pod \"node-exporter-mp9ts\" (UID: \"1b8cfc23-4e2b-46cc-9c6d-82e27573d34c\") " pod="openshift-monitoring/node-exporter-mp9ts"
Jul 22 02:20:56.878523 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.878503    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1b8cfc23-4e2b-46cc-9c6d-82e27573d34c-node-exporter-tls\") pod \"node-exporter-mp9ts\" (UID: \"1b8cfc23-4e2b-46cc-9c6d-82e27573d34c\") " pod="openshift-monitoring/node-exporter-mp9ts"
Jul 22 02:20:56.878720 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.878703    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1049839-4275-4715-bea3-230b58d93041-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-v78rs\" (UID: \"d1049839-4275-4715-bea3-230b58d93041\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-v78rs"
Jul 22 02:20:56.878881 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.878858    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1b8cfc23-4e2b-46cc-9c6d-82e27573d34c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mp9ts\" (UID: \"1b8cfc23-4e2b-46cc-9c6d-82e27573d34c\") " pod="openshift-monitoring/node-exporter-mp9ts"
Jul 22 02:20:56.878959 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.878946    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d1049839-4275-4715-bea3-230b58d93041-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-v78rs\" (UID: \"d1049839-4275-4715-bea3-230b58d93041\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-v78rs"
Jul 22 02:20:56.883782 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.883758    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg97m\" (UniqueName: \"kubernetes.io/projected/1b8cfc23-4e2b-46cc-9c6d-82e27573d34c-kube-api-access-mg97m\") pod \"node-exporter-mp9ts\" (UID: \"1b8cfc23-4e2b-46cc-9c6d-82e27573d34c\") " pod="openshift-monitoring/node-exporter-mp9ts"
Jul 22 02:20:56.884196 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.884176    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qf842\" (UniqueName: \"kubernetes.io/projected/d1049839-4275-4715-bea3-230b58d93041-kube-api-access-qf842\") pod \"openshift-state-metrics-79f6958cdf-v78rs\" (UID: \"d1049839-4275-4715-bea3-230b58d93041\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-v78rs"
Jul 22 02:20:56.977320 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.977282    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/49ea4211-194d-4d3d-992d-20f3462d03b3-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-f2pg8\" (UID: \"49ea4211-194d-4d3d-992d-20f3462d03b3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-f2pg8"
Jul 22 02:20:56.977320 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.977318    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/49ea4211-194d-4d3d-992d-20f3462d03b3-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-f2pg8\" (UID: \"49ea4211-194d-4d3d-992d-20f3462d03b3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-f2pg8"
Jul 22 02:20:56.977551 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.977343    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/49ea4211-194d-4d3d-992d-20f3462d03b3-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-f2pg8\" (UID: \"49ea4211-194d-4d3d-992d-20f3462d03b3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-f2pg8"
Jul 22 02:20:56.977551 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.977382    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/49ea4211-194d-4d3d-992d-20f3462d03b3-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-f2pg8\" (UID: \"49ea4211-194d-4d3d-992d-20f3462d03b3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-f2pg8"
Jul 22 02:20:56.977551 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.977411    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zp9rd\" (UniqueName: \"kubernetes.io/projected/49ea4211-194d-4d3d-992d-20f3462d03b3-kube-api-access-zp9rd\") pod \"kube-state-metrics-56c5594674-f2pg8\" (UID: \"49ea4211-194d-4d3d-992d-20f3462d03b3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-f2pg8"
Jul 22 02:20:56.977551 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.977476    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49ea4211-194d-4d3d-992d-20f3462d03b3-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-f2pg8\" (UID: \"49ea4211-194d-4d3d-992d-20f3462d03b3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-f2pg8"
Jul 22 02:20:56.977829 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.977801    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/49ea4211-194d-4d3d-992d-20f3462d03b3-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-f2pg8\" (UID: \"49ea4211-194d-4d3d-992d-20f3462d03b3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-f2pg8"
Jul 22 02:20:56.978174 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.978149    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/49ea4211-194d-4d3d-992d-20f3462d03b3-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-f2pg8\" (UID: \"49ea4211-194d-4d3d-992d-20f3462d03b3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-f2pg8"
Jul 22 02:20:56.978256 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.978159    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49ea4211-194d-4d3d-992d-20f3462d03b3-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-f2pg8\" (UID: \"49ea4211-194d-4d3d-992d-20f3462d03b3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-f2pg8"
Jul 22 02:20:56.979809 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.979784    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/49ea4211-194d-4d3d-992d-20f3462d03b3-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-f2pg8\" (UID: \"49ea4211-194d-4d3d-992d-20f3462d03b3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-f2pg8"
Jul 22 02:20:56.979883 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.979865    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/49ea4211-194d-4d3d-992d-20f3462d03b3-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-f2pg8\" (UID: \"49ea4211-194d-4d3d-992d-20f3462d03b3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-f2pg8"
Jul 22 02:20:56.980613 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.980601    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-v78rs"
Jul 22 02:20:56.986796 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.986773    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zp9rd\" (UniqueName: \"kubernetes.io/projected/49ea4211-194d-4d3d-992d-20f3462d03b3-kube-api-access-zp9rd\") pod \"kube-state-metrics-56c5594674-f2pg8\" (UID: \"49ea4211-194d-4d3d-992d-20f3462d03b3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-f2pg8"
Jul 22 02:20:56.991549 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:56.991532    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mp9ts"
Jul 22 02:20:57.002731 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:20:57.002707    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b8cfc23_4e2b_46cc_9c6d_82e27573d34c.slice/crio-1921209e752b57ff5f205e2924667dec0fbca40a8974a5b2ae6744f78e104d4b WatchSource:0}: Error finding container 1921209e752b57ff5f205e2924667dec0fbca40a8974a5b2ae6744f78e104d4b: Status 404 returned error can't find the container with id 1921209e752b57ff5f205e2924667dec0fbca40a8974a5b2ae6744f78e104d4b
Jul 22 02:20:57.069923 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.069896    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:57.095084 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.095057    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-f2pg8"
Jul 22 02:20:57.132399 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.132354    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-v78rs"]
Jul 22 02:20:57.136980 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:20:57.136935    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1049839_4275_4715_bea3_230b58d93041.slice/crio-da91187cd8950d2bf500b2673ad20af70aa41d48ad3aeb5fde65b2f45a7d6cd3 WatchSource:0}: Error finding container da91187cd8950d2bf500b2673ad20af70aa41d48ad3aeb5fde65b2f45a7d6cd3: Status 404 returned error can't find the container with id da91187cd8950d2bf500b2673ad20af70aa41d48ad3aeb5fde65b2f45a7d6cd3
Jul 22 02:20:57.184546 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.184503    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:57.252698 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.252561    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-f2pg8"]
Jul 22 02:20:57.254567 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.254542    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:57.259969 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:20:57.259923    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49ea4211_194d_4d3d_992d_20f3462d03b3.slice/crio-18b5691f2cb6792404dc595ac2ed9a0e66cdd55376993104c500b99f58453395 WatchSource:0}: Error finding container 18b5691f2cb6792404dc595ac2ed9a0e66cdd55376993104c500b99f58453395: Status 404 returned error can't find the container with id 18b5691f2cb6792404dc595ac2ed9a0e66cdd55376993104c500b99f58453395
Jul 22 02:20:57.333060 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.332607    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-v78rs" event={"ID":"d1049839-4275-4715-bea3-230b58d93041","Type":"ContainerStarted","Data":"3d06d5b9d7027fbce3cfca14e05cffa7b007f7b7506c2243749b918b450f1e09"}
Jul 22 02:20:57.333235 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.333092    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-v78rs" event={"ID":"d1049839-4275-4715-bea3-230b58d93041","Type":"ContainerStarted","Data":"8a9a023bbd8542bc98b2032874bef9864c1657b80989d319f1e711c4ee7fd42e"}
Jul 22 02:20:57.333235 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.333133    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-v78rs" event={"ID":"d1049839-4275-4715-bea3-230b58d93041","Type":"ContainerStarted","Data":"da91187cd8950d2bf500b2673ad20af70aa41d48ad3aeb5fde65b2f45a7d6cd3"}
Jul 22 02:20:57.334398 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.334371    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-f2pg8" event={"ID":"49ea4211-194d-4d3d-992d-20f3462d03b3","Type":"ContainerStarted","Data":"18b5691f2cb6792404dc595ac2ed9a0e66cdd55376993104c500b99f58453395"}
Jul 22 02:20:57.335757 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.335634    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mp9ts" event={"ID":"1b8cfc23-4e2b-46cc-9c6d-82e27573d34c","Type":"ContainerStarted","Data":"1921209e752b57ff5f205e2924667dec0fbca40a8974a5b2ae6744f78e104d4b"}
Jul 22 02:20:57.337028 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.337000    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-dkd8c" event={"ID":"121417f5-c17d-457c-a8f1-6ba982980cec","Type":"ContainerStarted","Data":"ce5f028e03759d3276f71c42b2b19912f2175962f820a7f4d77d3a214079e3d1"}
Jul 22 02:20:57.434313 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.434280    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 02:20:57.439058 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.439032    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.442265 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.441955    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 22 02:20:57.442265 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.441977    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 22 02:20:57.442265 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.442015    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 22 02:20:57.442265 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.442096    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 22 02:20:57.442265 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.441962    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 22 02:20:57.442592 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.442340    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-h8rvs\""
Jul 22 02:20:57.442592 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.442358    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 22 02:20:57.442592 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.442378    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 22 02:20:57.442592 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.442574    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 22 02:20:57.448427 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.448405    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 22 02:20:57.456648 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.456618    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 02:20:57.582934 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.582850    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.582934 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.582901    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8dd7cb2e-66c4-40af-8c25-bb81184d688a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.583175 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.582934    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8dd7cb2e-66c4-40af-8c25-bb81184d688a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.583175 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.582969    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-config-volume\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.583175 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.582994    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8dd7cb2e-66c4-40af-8c25-bb81184d688a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.583175 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.583025    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.583175 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.583095    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.583175 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.583170    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrdq9\" (UniqueName: \"kubernetes.io/projected/8dd7cb2e-66c4-40af-8c25-bb81184d688a-kube-api-access-rrdq9\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.583456 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.583197    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8dd7cb2e-66c4-40af-8c25-bb81184d688a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.583456 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.583248    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.583456 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.583281    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-web-config\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.583456 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.583313    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8dd7cb2e-66c4-40af-8c25-bb81184d688a-config-out\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.583456 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.583346    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.607701 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.607652    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:57.684146 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.684102    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.684342 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.684159    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.684342 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.684197    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrdq9\" (UniqueName: \"kubernetes.io/projected/8dd7cb2e-66c4-40af-8c25-bb81184d688a-kube-api-access-rrdq9\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.684342 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.684214    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8dd7cb2e-66c4-40af-8c25-bb81184d688a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.684342 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.684242    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.684342 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.684276    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-web-config\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.684342 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.684309    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8dd7cb2e-66c4-40af-8c25-bb81184d688a-config-out\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.684635 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.684344    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.684635 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.684371    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.684635 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.684394    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8dd7cb2e-66c4-40af-8c25-bb81184d688a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.684635 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.684424    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8dd7cb2e-66c4-40af-8c25-bb81184d688a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.684635 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.684456    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-config-volume\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.684635 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.684481    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8dd7cb2e-66c4-40af-8c25-bb81184d688a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.684949 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.684759    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8dd7cb2e-66c4-40af-8c25-bb81184d688a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.685778 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.685409    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8dd7cb2e-66c4-40af-8c25-bb81184d688a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.686308 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.686244    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8dd7cb2e-66c4-40af-8c25-bb81184d688a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.688368 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.688343    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.688484 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.688374    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8dd7cb2e-66c4-40af-8c25-bb81184d688a-config-out\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.688572 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.688550    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8dd7cb2e-66c4-40af-8c25-bb81184d688a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.689018 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.688956    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.689018 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.688967    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.689629 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.689603    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-config-volume\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.690048 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.690027    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.690209 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.690185    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-web-config\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.690688 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.690662    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.694311 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.694289    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrdq9\" (UniqueName: \"kubernetes.io/projected/8dd7cb2e-66c4-40af-8c25-bb81184d688a-kube-api-access-rrdq9\") pod \"alertmanager-main-0\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.708189 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.708162    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:57.751136 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.751092    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:20:57.808408 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.808340    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:57.909233 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.909159    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:57.949120 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:57.949011    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 02:20:58.063232 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:58.063199    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:58.231039 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:20:58.230962    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dd7cb2e_66c4_40af_8c25_bb81184d688a.slice/crio-b5e8cab89b5409036937090c55c189d64468b4f4f163caac2236bf0c1fe5018a WatchSource:0}: Error finding container b5e8cab89b5409036937090c55c189d64468b4f4f163caac2236bf0c1fe5018a: Status 404 returned error can't find the container with id b5e8cab89b5409036937090c55c189d64468b4f4f163caac2236bf0c1fe5018a
Jul 22 02:20:58.318069 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:58.318031    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:58.344811 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:58.344694    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8dd7cb2e-66c4-40af-8c25-bb81184d688a","Type":"ContainerStarted","Data":"b5e8cab89b5409036937090c55c189d64468b4f4f163caac2236bf0c1fe5018a"}
Jul 22 02:20:58.547383 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:58.547300    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:59.335179 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.335123    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:59.351462 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.351425    2581 generic.go:356] "Generic (PLEG): container finished" podID="1b8cfc23-4e2b-46cc-9c6d-82e27573d34c" containerID="b6e4f7c44348af39a902df61f8c9301e4352bcff0c3e44fb494fab40bd91232c" exitCode=0
Jul 22 02:20:59.351625 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.351559    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mp9ts" event={"ID":"1b8cfc23-4e2b-46cc-9c6d-82e27573d34c","Type":"ContainerDied","Data":"b6e4f7c44348af39a902df61f8c9301e4352bcff0c3e44fb494fab40bd91232c"}
Jul 22 02:20:59.354578 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.354548    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-v78rs" event={"ID":"d1049839-4275-4715-bea3-230b58d93041","Type":"ContainerStarted","Data":"b3c94557b8531e22a7ccd7237e94299cf65c2cce96f6c1342e36bbd363a2e3a6"}
Jul 22 02:20:59.392944 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.392878    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-v78rs" podStartSLOduration=2.152060014 podStartE2EDuration="3.392854259s" totalImagesPullingTime="1.240794245s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:20:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:59.392099503 +0000 UTC m=+80.932893132" watchObservedRunningTime="2026-07-22 02:20:59.392854259 +0000 UTC m=+80.933647887"
Jul 22 02:20:59.396227 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.396201    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:59.450856 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.450818    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-776964c7b7-z4wzv"]
Jul 22 02:20:59.455098 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.455069    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-776964c7b7-z4wzv"
Jul 22 02:20:59.458660 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.458435    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-nxgsl\""
Jul 22 02:20:59.458660 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.458475    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 22 02:20:59.458660 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.458511    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 22 02:20:59.458660 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.458447    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 22 02:20:59.458660 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.458647    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 22 02:20:59.459009 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.458760    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 22 02:20:59.459009 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.458907    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-db98d75le811b\""
Jul 22 02:20:59.468868 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.468825    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-776964c7b7-z4wzv"]
Jul 22 02:20:59.518777 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.518711    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:59.603088 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.603001    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/fea2175d-60d6-4fca-b26f-c935babe00c1-secret-thanos-querier-tls\") pod \"thanos-querier-776964c7b7-z4wzv\" (UID: \"fea2175d-60d6-4fca-b26f-c935babe00c1\") " pod="openshift-monitoring/thanos-querier-776964c7b7-z4wzv"
Jul 22 02:20:59.603302 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.603083    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fea2175d-60d6-4fca-b26f-c935babe00c1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-776964c7b7-z4wzv\" (UID: \"fea2175d-60d6-4fca-b26f-c935babe00c1\") " pod="openshift-monitoring/thanos-querier-776964c7b7-z4wzv"
Jul 22 02:20:59.603302 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.603160    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fea2175d-60d6-4fca-b26f-c935babe00c1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-776964c7b7-z4wzv\" (UID: \"fea2175d-60d6-4fca-b26f-c935babe00c1\") " pod="openshift-monitoring/thanos-querier-776964c7b7-z4wzv"
Jul 22 02:20:59.603302 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.603195    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fea2175d-60d6-4fca-b26f-c935babe00c1-metrics-client-ca\") pod \"thanos-querier-776964c7b7-z4wzv\" (UID: \"fea2175d-60d6-4fca-b26f-c935babe00c1\") " pod="openshift-monitoring/thanos-querier-776964c7b7-z4wzv"
Jul 22 02:20:59.603302 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.603224    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/fea2175d-60d6-4fca-b26f-c935babe00c1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-776964c7b7-z4wzv\" (UID: \"fea2175d-60d6-4fca-b26f-c935babe00c1\") " pod="openshift-monitoring/thanos-querier-776964c7b7-z4wzv"
Jul 22 02:20:59.603302 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.603258    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pxqc\" (UniqueName: \"kubernetes.io/projected/fea2175d-60d6-4fca-b26f-c935babe00c1-kube-api-access-5pxqc\") pod \"thanos-querier-776964c7b7-z4wzv\" (UID: \"fea2175d-60d6-4fca-b26f-c935babe00c1\") " pod="openshift-monitoring/thanos-querier-776964c7b7-z4wzv"
Jul 22 02:20:59.603302 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.603280    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/fea2175d-60d6-4fca-b26f-c935babe00c1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-776964c7b7-z4wzv\" (UID: \"fea2175d-60d6-4fca-b26f-c935babe00c1\") " pod="openshift-monitoring/thanos-querier-776964c7b7-z4wzv"
Jul 22 02:20:59.603609 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.603307    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fea2175d-60d6-4fca-b26f-c935babe00c1-secret-grpc-tls\") pod \"thanos-querier-776964c7b7-z4wzv\" (UID: \"fea2175d-60d6-4fca-b26f-c935babe00c1\") " pod="openshift-monitoring/thanos-querier-776964c7b7-z4wzv"
Jul 22 02:20:59.704446 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.704414    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/fea2175d-60d6-4fca-b26f-c935babe00c1-secret-thanos-querier-tls\") pod \"thanos-querier-776964c7b7-z4wzv\" (UID: \"fea2175d-60d6-4fca-b26f-c935babe00c1\") " pod="openshift-monitoring/thanos-querier-776964c7b7-z4wzv"
Jul 22 02:20:59.704446 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.704447    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fea2175d-60d6-4fca-b26f-c935babe00c1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-776964c7b7-z4wzv\" (UID: \"fea2175d-60d6-4fca-b26f-c935babe00c1\") " pod="openshift-monitoring/thanos-querier-776964c7b7-z4wzv"
Jul 22 02:20:59.704682 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.704481    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fea2175d-60d6-4fca-b26f-c935babe00c1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-776964c7b7-z4wzv\" (UID: \"fea2175d-60d6-4fca-b26f-c935babe00c1\") " pod="openshift-monitoring/thanos-querier-776964c7b7-z4wzv"
Jul 22 02:20:59.704682 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.704522    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fea2175d-60d6-4fca-b26f-c935babe00c1-metrics-client-ca\") pod \"thanos-querier-776964c7b7-z4wzv\" (UID: \"fea2175d-60d6-4fca-b26f-c935babe00c1\") " pod="openshift-monitoring/thanos-querier-776964c7b7-z4wzv"
Jul 22 02:20:59.704682 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.704557    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/fea2175d-60d6-4fca-b26f-c935babe00c1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-776964c7b7-z4wzv\" (UID: \"fea2175d-60d6-4fca-b26f-c935babe00c1\") " pod="openshift-monitoring/thanos-querier-776964c7b7-z4wzv"
Jul 22 02:20:59.704682 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.704576    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pxqc\" (UniqueName: \"kubernetes.io/projected/fea2175d-60d6-4fca-b26f-c935babe00c1-kube-api-access-5pxqc\") pod \"thanos-querier-776964c7b7-z4wzv\" (UID: \"fea2175d-60d6-4fca-b26f-c935babe00c1\") " pod="openshift-monitoring/thanos-querier-776964c7b7-z4wzv"
Jul 22 02:20:59.704682 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.704605    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/fea2175d-60d6-4fca-b26f-c935babe00c1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-776964c7b7-z4wzv\" (UID: \"fea2175d-60d6-4fca-b26f-c935babe00c1\") " pod="openshift-monitoring/thanos-querier-776964c7b7-z4wzv"
Jul 22 02:20:59.704682 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.704633    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fea2175d-60d6-4fca-b26f-c935babe00c1-secret-grpc-tls\") pod \"thanos-querier-776964c7b7-z4wzv\" (UID: \"fea2175d-60d6-4fca-b26f-c935babe00c1\") " pod="openshift-monitoring/thanos-querier-776964c7b7-z4wzv"
Jul 22 02:20:59.705705 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.705626    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fea2175d-60d6-4fca-b26f-c935babe00c1-metrics-client-ca\") pod \"thanos-querier-776964c7b7-z4wzv\" (UID: \"fea2175d-60d6-4fca-b26f-c935babe00c1\") " pod="openshift-monitoring/thanos-querier-776964c7b7-z4wzv"
Jul 22 02:20:59.707729 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.707679    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/fea2175d-60d6-4fca-b26f-c935babe00c1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-776964c7b7-z4wzv\" (UID: \"fea2175d-60d6-4fca-b26f-c935babe00c1\") " pod="openshift-monitoring/thanos-querier-776964c7b7-z4wzv"
Jul 22 02:20:59.707904 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.707881    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fea2175d-60d6-4fca-b26f-c935babe00c1-secret-grpc-tls\") pod \"thanos-querier-776964c7b7-z4wzv\" (UID: \"fea2175d-60d6-4fca-b26f-c935babe00c1\") " pod="openshift-monitoring/thanos-querier-776964c7b7-z4wzv"
Jul 22 02:20:59.708185 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.708139    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fea2175d-60d6-4fca-b26f-c935babe00c1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-776964c7b7-z4wzv\" (UID: \"fea2175d-60d6-4fca-b26f-c935babe00c1\") " pod="openshift-monitoring/thanos-querier-776964c7b7-z4wzv"
Jul 22 02:20:59.708185 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.708160    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fea2175d-60d6-4fca-b26f-c935babe00c1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-776964c7b7-z4wzv\" (UID: \"fea2175d-60d6-4fca-b26f-c935babe00c1\") " pod="openshift-monitoring/thanos-querier-776964c7b7-z4wzv"
Jul 22 02:20:59.708299 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.708278    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/fea2175d-60d6-4fca-b26f-c935babe00c1-secret-thanos-querier-tls\") pod \"thanos-querier-776964c7b7-z4wzv\" (UID: \"fea2175d-60d6-4fca-b26f-c935babe00c1\") " pod="openshift-monitoring/thanos-querier-776964c7b7-z4wzv"
Jul 22 02:20:59.708520 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.708498    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/fea2175d-60d6-4fca-b26f-c935babe00c1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-776964c7b7-z4wzv\" (UID: \"fea2175d-60d6-4fca-b26f-c935babe00c1\") " pod="openshift-monitoring/thanos-querier-776964c7b7-z4wzv"
Jul 22 02:20:59.709060 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.709030    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:20:59.714329 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.714306    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pxqc\" (UniqueName: \"kubernetes.io/projected/fea2175d-60d6-4fca-b26f-c935babe00c1-kube-api-access-5pxqc\") pod \"thanos-querier-776964c7b7-z4wzv\" (UID: \"fea2175d-60d6-4fca-b26f-c935babe00c1\") " pod="openshift-monitoring/thanos-querier-776964c7b7-z4wzv"
Jul 22 02:20:59.770705 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.770512    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-776964c7b7-z4wzv"
Jul 22 02:20:59.979515 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:20:59.979141    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-776964c7b7-z4wzv"]
Jul 22 02:20:59.985386 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:20:59.985326    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfea2175d_60d6_4fca_b26f_c935babe00c1.slice/crio-a295212874142efc345fdc345626a6f3c84910fe539d03fab00be2d50c688a09 WatchSource:0}: Error finding container a295212874142efc345fdc345626a6f3c84910fe539d03fab00be2d50c688a09: Status 404 returned error can't find the container with id a295212874142efc345fdc345626a6f3c84910fe539d03fab00be2d50c688a09
Jul 22 02:21:00.366838 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:00.366792    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-f2pg8" event={"ID":"49ea4211-194d-4d3d-992d-20f3462d03b3","Type":"ContainerStarted","Data":"62e6812a7028a9737bd878050ab6aab1e3726895a13e9bee4d7ed0a766bcd16e"}
Jul 22 02:21:00.367330 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:00.366907    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-f2pg8" event={"ID":"49ea4211-194d-4d3d-992d-20f3462d03b3","Type":"ContainerStarted","Data":"33432bb6508e5b5f4fc8ea68ebe96df3135760cfb3151c1698538e457963c4c6"}
Jul 22 02:21:00.367330 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:00.366938    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-f2pg8" event={"ID":"49ea4211-194d-4d3d-992d-20f3462d03b3","Type":"ContainerStarted","Data":"6524ae8e8ec07760b206b36beeaee74326936ede19bcc2d26942cdbe1ac6ac58"}
Jul 22 02:21:00.368935 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:00.368903    2581 generic.go:356] "Generic (PLEG): container finished" podID="8dd7cb2e-66c4-40af-8c25-bb81184d688a" containerID="896826a169b9c822552ac4b3020fe5f4ae1393bc31977605457d7634a7b1437b" exitCode=0
Jul 22 02:21:00.369043 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:00.368995    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8dd7cb2e-66c4-40af-8c25-bb81184d688a","Type":"ContainerDied","Data":"896826a169b9c822552ac4b3020fe5f4ae1393bc31977605457d7634a7b1437b"}
Jul 22 02:21:00.374323 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:00.374229    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mp9ts" event={"ID":"1b8cfc23-4e2b-46cc-9c6d-82e27573d34c","Type":"ContainerStarted","Data":"53bba9fa94e113a0f2aa83cb1d9163bf8f23a24628b5b3fbbf8114c97baa4e4b"}
Jul 22 02:21:00.374323 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:00.374281    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mp9ts" event={"ID":"1b8cfc23-4e2b-46cc-9c6d-82e27573d34c","Type":"ContainerStarted","Data":"60eaaef302a93f27c8ec7a84adccf68524c6d4c622a81ff45ea6bd6121366015"}
Jul 22 02:21:00.376553 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:00.376468    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-776964c7b7-z4wzv" event={"ID":"fea2175d-60d6-4fca-b26f-c935babe00c1","Type":"ContainerStarted","Data":"a295212874142efc345fdc345626a6f3c84910fe539d03fab00be2d50c688a09"}
Jul 22 02:21:00.386838 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:00.386787    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-f2pg8" podStartSLOduration=1.905349626 podStartE2EDuration="4.386771172s" totalImagesPullingTime="2.481421546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:20:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:21:00.385297449 +0000 UTC m=+81.926091077" watchObservedRunningTime="2026-07-22 02:21:00.386771172 +0000 UTC m=+81.927564798"
Jul 22 02:21:00.405879 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:00.405825    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mp9ts" podStartSLOduration=2.797697474 podStartE2EDuration="4.405805985s" totalImagesPullingTime="1.518686997s" totalInitContainerRuntime="89.421514ms" isStatefulPod=true podCreationTimestamp="2026-07-22 02:20:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:21:00.405313352 +0000 UTC m=+81.946106979" watchObservedRunningTime="2026-07-22 02:21:00.405805985 +0000 UTC m=+81.946599606"
Jul 22 02:21:00.686786 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:00.686678    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:21:00.782592 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:00.782515    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:21:01.145281 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.145146    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:21:01.161171 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.159866    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wvc9x"]
Jul 22 02:21:01.167389 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.167353    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wvc9x"
Jul 22 02:21:01.171028 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.171002    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-sk4xn\""
Jul 22 02:21:01.171156 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.171010    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 22 02:21:01.176099 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.176035    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wvc9x"]
Jul 22 02:21:01.231337 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.231264    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:21:01.235574 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.235539    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-78d6957c46-pqvxl"]
Jul 22 02:21:01.239167 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.239141    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-78d6957c46-pqvxl"
Jul 22 02:21:01.241794 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.241768    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 22 02:21:01.242970 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.242935    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 22 02:21:01.244011 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.243990    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 22 02:21:01.244124 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.244044    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-5kbijjvdqd3np\""
Jul 22 02:21:01.244287 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.244269    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 22 02:21:01.244363 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.244293    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-j97ht\""
Jul 22 02:21:01.249673 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.249610    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-78d6957c46-pqvxl"]
Jul 22 02:21:01.321640 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.321566    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/46360a1c-c876-4fd3-ac0c-09956174bebd-metrics-server-audit-profiles\") pod \"metrics-server-78d6957c46-pqvxl\" (UID: \"46360a1c-c876-4fd3-ac0c-09956174bebd\") " pod="openshift-monitoring/metrics-server-78d6957c46-pqvxl"
Jul 22 02:21:01.321909 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.321692    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/46360a1c-c876-4fd3-ac0c-09956174bebd-secret-metrics-server-client-certs\") pod \"metrics-server-78d6957c46-pqvxl\" (UID: \"46360a1c-c876-4fd3-ac0c-09956174bebd\") " pod="openshift-monitoring/metrics-server-78d6957c46-pqvxl"
Jul 22 02:21:01.321909 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.321729    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/91bd715b-d6dc-4447-85c2-74a058a4478e-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-wvc9x\" (UID: \"91bd715b-d6dc-4447-85c2-74a058a4478e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wvc9x"
Jul 22 02:21:01.321909 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.321783    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46360a1c-c876-4fd3-ac0c-09956174bebd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-78d6957c46-pqvxl\" (UID: \"46360a1c-c876-4fd3-ac0c-09956174bebd\") " pod="openshift-monitoring/metrics-server-78d6957c46-pqvxl"
Jul 22 02:21:01.321909 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.321824    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4p9bs\" (UniqueName: \"kubernetes.io/projected/91bd715b-d6dc-4447-85c2-74a058a4478e-kube-api-access-4p9bs\") pod \"monitoring-plugin-7b7b7bc8bf-wvc9x\" (UID: \"91bd715b-d6dc-4447-85c2-74a058a4478e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wvc9x"
Jul 22 02:21:01.321909 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.321852    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26s6f\" (UniqueName: \"kubernetes.io/projected/46360a1c-c876-4fd3-ac0c-09956174bebd-kube-api-access-26s6f\") pod \"metrics-server-78d6957c46-pqvxl\" (UID: \"46360a1c-c876-4fd3-ac0c-09956174bebd\") " pod="openshift-monitoring/metrics-server-78d6957c46-pqvxl"
Jul 22 02:21:01.321909 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.321891    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/46360a1c-c876-4fd3-ac0c-09956174bebd-client-ca-bundle\") pod \"metrics-server-78d6957c46-pqvxl\" (UID: \"46360a1c-c876-4fd3-ac0c-09956174bebd\") " pod="openshift-monitoring/metrics-server-78d6957c46-pqvxl"
Jul 22 02:21:01.322206 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.321921    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/46360a1c-c876-4fd3-ac0c-09956174bebd-secret-metrics-server-tls\") pod \"metrics-server-78d6957c46-pqvxl\" (UID: \"46360a1c-c876-4fd3-ac0c-09956174bebd\") " pod="openshift-monitoring/metrics-server-78d6957c46-pqvxl"
Jul 22 02:21:01.322206 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.321944    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/46360a1c-c876-4fd3-ac0c-09956174bebd-audit-log\") pod \"metrics-server-78d6957c46-pqvxl\" (UID: \"46360a1c-c876-4fd3-ac0c-09956174bebd\") " pod="openshift-monitoring/metrics-server-78d6957c46-pqvxl"
Jul 22 02:21:01.423249 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.423170    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/46360a1c-c876-4fd3-ac0c-09956174bebd-secret-metrics-server-client-certs\") pod \"metrics-server-78d6957c46-pqvxl\" (UID: \"46360a1c-c876-4fd3-ac0c-09956174bebd\") " pod="openshift-monitoring/metrics-server-78d6957c46-pqvxl"
Jul 22 02:21:01.423249 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.423219    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/91bd715b-d6dc-4447-85c2-74a058a4478e-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-wvc9x\" (UID: \"91bd715b-d6dc-4447-85c2-74a058a4478e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wvc9x"
Jul 22 02:21:01.423249 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.423251    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46360a1c-c876-4fd3-ac0c-09956174bebd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-78d6957c46-pqvxl\" (UID: \"46360a1c-c876-4fd3-ac0c-09956174bebd\") " pod="openshift-monitoring/metrics-server-78d6957c46-pqvxl"
Jul 22 02:21:01.423863 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.423288    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4p9bs\" (UniqueName: \"kubernetes.io/projected/91bd715b-d6dc-4447-85c2-74a058a4478e-kube-api-access-4p9bs\") pod \"monitoring-plugin-7b7b7bc8bf-wvc9x\" (UID: \"91bd715b-d6dc-4447-85c2-74a058a4478e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wvc9x"
Jul 22 02:21:01.423863 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.423315    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26s6f\" (UniqueName: \"kubernetes.io/projected/46360a1c-c876-4fd3-ac0c-09956174bebd-kube-api-access-26s6f\") pod \"metrics-server-78d6957c46-pqvxl\" (UID: \"46360a1c-c876-4fd3-ac0c-09956174bebd\") " pod="openshift-monitoring/metrics-server-78d6957c46-pqvxl"
Jul 22 02:21:01.423863 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.423358    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/46360a1c-c876-4fd3-ac0c-09956174bebd-client-ca-bundle\") pod \"metrics-server-78d6957c46-pqvxl\" (UID: \"46360a1c-c876-4fd3-ac0c-09956174bebd\") " pod="openshift-monitoring/metrics-server-78d6957c46-pqvxl"
Jul 22 02:21:01.423863 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.423388    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/46360a1c-c876-4fd3-ac0c-09956174bebd-secret-metrics-server-tls\") pod \"metrics-server-78d6957c46-pqvxl\" (UID: \"46360a1c-c876-4fd3-ac0c-09956174bebd\") " pod="openshift-monitoring/metrics-server-78d6957c46-pqvxl"
Jul 22 02:21:01.423863 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:21:01.423402    2581 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 22 02:21:01.423863 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.423413    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/46360a1c-c876-4fd3-ac0c-09956174bebd-audit-log\") pod \"metrics-server-78d6957c46-pqvxl\" (UID: \"46360a1c-c876-4fd3-ac0c-09956174bebd\") " pod="openshift-monitoring/metrics-server-78d6957c46-pqvxl"
Jul 22 02:21:01.423863 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.423439    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/46360a1c-c876-4fd3-ac0c-09956174bebd-metrics-server-audit-profiles\") pod \"metrics-server-78d6957c46-pqvxl\" (UID: \"46360a1c-c876-4fd3-ac0c-09956174bebd\") " pod="openshift-monitoring/metrics-server-78d6957c46-pqvxl"
Jul 22 02:21:01.423863 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:21:01.423478    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91bd715b-d6dc-4447-85c2-74a058a4478e-monitoring-plugin-cert podName:91bd715b-d6dc-4447-85c2-74a058a4478e nodeName:}" failed. No retries permitted until 2026-07-22 02:21:01.923456348 +0000 UTC m=+83.464249969 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/91bd715b-d6dc-4447-85c2-74a058a4478e-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-wvc9x" (UID: "91bd715b-d6dc-4447-85c2-74a058a4478e") : secret "monitoring-plugin-cert" not found
Jul 22 02:21:01.424241 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.424053    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46360a1c-c876-4fd3-ac0c-09956174bebd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-78d6957c46-pqvxl\" (UID: \"46360a1c-c876-4fd3-ac0c-09956174bebd\") " pod="openshift-monitoring/metrics-server-78d6957c46-pqvxl"
Jul 22 02:21:01.424574 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.424537    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/46360a1c-c876-4fd3-ac0c-09956174bebd-audit-log\") pod \"metrics-server-78d6957c46-pqvxl\" (UID: \"46360a1c-c876-4fd3-ac0c-09956174bebd\") " pod="openshift-monitoring/metrics-server-78d6957c46-pqvxl"
Jul 22 02:21:01.425075 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.425035    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/46360a1c-c876-4fd3-ac0c-09956174bebd-metrics-server-audit-profiles\") pod \"metrics-server-78d6957c46-pqvxl\" (UID: \"46360a1c-c876-4fd3-ac0c-09956174bebd\") " pod="openshift-monitoring/metrics-server-78d6957c46-pqvxl"
Jul 22 02:21:01.427040 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.427010    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/46360a1c-c876-4fd3-ac0c-09956174bebd-secret-metrics-server-tls\") pod \"metrics-server-78d6957c46-pqvxl\" (UID: \"46360a1c-c876-4fd3-ac0c-09956174bebd\") " pod="openshift-monitoring/metrics-server-78d6957c46-pqvxl"
Jul 22 02:21:01.427193 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.427156    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/46360a1c-c876-4fd3-ac0c-09956174bebd-secret-metrics-server-client-certs\") pod \"metrics-server-78d6957c46-pqvxl\" (UID: \"46360a1c-c876-4fd3-ac0c-09956174bebd\") " pod="openshift-monitoring/metrics-server-78d6957c46-pqvxl"
Jul 22 02:21:01.430078 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.430056    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/46360a1c-c876-4fd3-ac0c-09956174bebd-client-ca-bundle\") pod \"metrics-server-78d6957c46-pqvxl\" (UID: \"46360a1c-c876-4fd3-ac0c-09956174bebd\") " pod="openshift-monitoring/metrics-server-78d6957c46-pqvxl"
Jul 22 02:21:01.435878 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.435831    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26s6f\" (UniqueName: \"kubernetes.io/projected/46360a1c-c876-4fd3-ac0c-09956174bebd-kube-api-access-26s6f\") pod \"metrics-server-78d6957c46-pqvxl\" (UID: \"46360a1c-c876-4fd3-ac0c-09956174bebd\") " pod="openshift-monitoring/metrics-server-78d6957c46-pqvxl"
Jul 22 02:21:01.438592 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.438199    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4p9bs\" (UniqueName: \"kubernetes.io/projected/91bd715b-d6dc-4447-85c2-74a058a4478e-kube-api-access-4p9bs\") pod \"monitoring-plugin-7b7b7bc8bf-wvc9x\" (UID: \"91bd715b-d6dc-4447-85c2-74a058a4478e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wvc9x"
Jul 22 02:21:01.552265 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.552221    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-78d6957c46-pqvxl"
Jul 22 02:21:01.666626 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.666543    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:21:01.928826 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.928785    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/91bd715b-d6dc-4447-85c2-74a058a4478e-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-wvc9x\" (UID: \"91bd715b-d6dc-4447-85c2-74a058a4478e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wvc9x"
Jul 22 02:21:01.931494 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:01.931466    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/91bd715b-d6dc-4447-85c2-74a058a4478e-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-wvc9x\" (UID: \"91bd715b-d6dc-4447-85c2-74a058a4478e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wvc9x"
Jul 22 02:21:02.087349 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.087310    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wvc9x"
Jul 22 02:21:02.553311 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.553113    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wvc9x"]
Jul 22 02:21:02.561855 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:21:02.561810    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91bd715b_d6dc_4447_85c2_74a058a4478e.slice/crio-0e7c52be5a1ae25fdfedecf44294a1983cbdebb7f4fef4d74e672654a8ec7fc3 WatchSource:0}: Error finding container 0e7c52be5a1ae25fdfedecf44294a1983cbdebb7f4fef4d74e672654a8ec7fc3: Status 404 returned error can't find the container with id 0e7c52be5a1ae25fdfedecf44294a1983cbdebb7f4fef4d74e672654a8ec7fc3
Jul 22 02:21:02.589930 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.589858    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-78d6957c46-pqvxl"]
Jul 22 02:21:02.594387 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:21:02.594355    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46360a1c_c876_4fd3_ac0c_09956174bebd.slice/crio-50a815cd952059ce8a366bf5539704248f289e219fe70b891798040545711538 WatchSource:0}: Error finding container 50a815cd952059ce8a366bf5539704248f289e219fe70b891798040545711538: Status 404 returned error can't find the container with id 50a815cd952059ce8a366bf5539704248f289e219fe70b891798040545711538
Jul 22 02:21:02.712354 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.712297    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:21:02.717912 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.717866    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 02:21:02.722966 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.722941    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.726290 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.725789    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-drectbtkpsgj2\""
Jul 22 02:21:02.726290 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.726014    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 22 02:21:02.726764 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.726557    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-wzj4q\""
Jul 22 02:21:02.729866 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.727106    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 22 02:21:02.729866 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.727323    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 22 02:21:02.729866 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.727494    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 22 02:21:02.729866 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.727674    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 22 02:21:02.729866 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.728028    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 22 02:21:02.729866 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.729163    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 22 02:21:02.729866 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.729307    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 22 02:21:02.729866 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.729456    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 22 02:21:02.729866 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.729643    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 22 02:21:02.729866 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.729680    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 22 02:21:02.730439 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.730005    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 22 02:21:02.732867 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.732632    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 22 02:21:02.736796 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.736775    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 22 02:21:02.744489 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.744067    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 02:21:02.837973 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.837942    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.837973 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.837972    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.838161 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.837990    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdvgn\" (UniqueName: \"kubernetes.io/projected/128db35e-9df1-4698-ae00-0dad30ec97e7-kube-api-access-vdvgn\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.838161 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.838012    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.838161 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.838106    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-config\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.838161 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.838121    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-web-config\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.838161 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.838144    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/128db35e-9df1-4698-ae00-0dad30ec97e7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.838308 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.838171    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.838308 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.838222    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.838308 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.838292    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.838388 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.838341    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.838422 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.838378    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/128db35e-9df1-4698-ae00-0dad30ec97e7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.838451 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.838416    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.838486 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.838450    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.838486 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.838472    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.838545 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.838523    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.838574 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.838547    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/128db35e-9df1-4698-ae00-0dad30ec97e7-config-out\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.838574 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.838567    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.838633 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.838584    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.838633 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.838605    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.875109 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.875077    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:21:02.939998 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.939589    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.939998 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.939648    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.939998 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.939680    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.939998 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.939706    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/128db35e-9df1-4698-ae00-0dad30ec97e7-config-out\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.939998 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.939748    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.939998 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.939771    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.939998 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.939803    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.939998 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.939872    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.939998 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.939894    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.939998 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.939915    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdvgn\" (UniqueName: \"kubernetes.io/projected/128db35e-9df1-4698-ae00-0dad30ec97e7-kube-api-access-vdvgn\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.939998 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.939941    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.939998 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.939973    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-config\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.939998 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.939994    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-web-config\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.940972 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.940020    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/128db35e-9df1-4698-ae00-0dad30ec97e7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.940972 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.940045    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.940972 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.940070    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.940972 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.940096    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.940972 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.940120    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.940972 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.940146    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/128db35e-9df1-4698-ae00-0dad30ec97e7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.940972 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.940181    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.944569 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.944053    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.945783 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.944690    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.945783 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.945418    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.946787 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.946765    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.953151 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.947310    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.953151 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.947462    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.953151 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.947928    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.953151 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.949640    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.953151 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.950116    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/128db35e-9df1-4698-ae00-0dad30ec97e7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.953151 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.951032    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.953151 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.951958    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.954985 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.954702    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/128db35e-9df1-4698-ae00-0dad30ec97e7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.955418 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.955382    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/128db35e-9df1-4698-ae00-0dad30ec97e7-config-out\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.961494 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.961447    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.962301 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.962139    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-web-config\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.964136 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.964109    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.964901 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.964853    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.968937 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.967065    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-config\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.968937 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.968895    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.969706 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.969687    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdvgn\" (UniqueName: \"kubernetes.io/projected/128db35e-9df1-4698-ae00-0dad30ec97e7-kube-api-access-vdvgn\") pod \"prometheus-k8s-0\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:02.975286 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:02.975184    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:21:03.053517 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:03.053474    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:03.233529 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:03.233372    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 02:21:03.238158 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:21:03.238125    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod128db35e_9df1_4698_ae00_0dad30ec97e7.slice/crio-d674c235d91ceb116f4793dc18a2a2c67b8034710c609c8742a75e6c12fa50cf WatchSource:0}: Error finding container d674c235d91ceb116f4793dc18a2a2c67b8034710c609c8742a75e6c12fa50cf: Status 404 returned error can't find the container with id d674c235d91ceb116f4793dc18a2a2c67b8034710c609c8742a75e6c12fa50cf
Jul 22 02:21:03.406940 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:03.406652    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8dd7cb2e-66c4-40af-8c25-bb81184d688a","Type":"ContainerStarted","Data":"9cce0b2c0b502bff41b98ce190bf20417a682fd56f3a07dd4e95786ce96af074"}
Jul 22 02:21:03.406940 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:03.406755    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8dd7cb2e-66c4-40af-8c25-bb81184d688a","Type":"ContainerStarted","Data":"0de116cd82f84555313649e4eeb051956a3d6361dc1af5f4f0648831753ea7ae"}
Jul 22 02:21:03.406940 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:03.406804    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8dd7cb2e-66c4-40af-8c25-bb81184d688a","Type":"ContainerStarted","Data":"5567c4f045dccdf516a0322a42faa63336842a7bebc0a8b305f3c3e50e94a713"}
Jul 22 02:21:03.406940 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:03.406842    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8dd7cb2e-66c4-40af-8c25-bb81184d688a","Type":"ContainerStarted","Data":"b6e7c714bcf99417f16e16feabeec127f3f6eb8a5870755283a1011f84e2932f"}
Jul 22 02:21:03.406940 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:03.406881    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8dd7cb2e-66c4-40af-8c25-bb81184d688a","Type":"ContainerStarted","Data":"6f67bcf657232aa6d5fc4035595866cef046bd8fcc1f419763bd1fb27c11903d"}
Jul 22 02:21:03.410325 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:03.410125    2581 generic.go:356] "Generic (PLEG): container finished" podID="128db35e-9df1-4698-ae00-0dad30ec97e7" containerID="e585971196599582b2a7ca6030a7021c3dc0505ece69dac78715616189a7b272" exitCode=0
Jul 22 02:21:03.410325 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:03.410224    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"128db35e-9df1-4698-ae00-0dad30ec97e7","Type":"ContainerDied","Data":"e585971196599582b2a7ca6030a7021c3dc0505ece69dac78715616189a7b272"}
Jul 22 02:21:03.412322 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:03.411511    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"128db35e-9df1-4698-ae00-0dad30ec97e7","Type":"ContainerStarted","Data":"d674c235d91ceb116f4793dc18a2a2c67b8034710c609c8742a75e6c12fa50cf"}
Jul 22 02:21:03.413807 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:03.412732    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-78d6957c46-pqvxl" event={"ID":"46360a1c-c876-4fd3-ac0c-09956174bebd","Type":"ContainerStarted","Data":"50a815cd952059ce8a366bf5539704248f289e219fe70b891798040545711538"}
Jul 22 02:21:03.416211 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:03.416023    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-776964c7b7-z4wzv" event={"ID":"fea2175d-60d6-4fca-b26f-c935babe00c1","Type":"ContainerStarted","Data":"09bbcf19d87f1fe2fd6825b3a9552a6b4abfd41cd3833a83bf3f1f012c02068c"}
Jul 22 02:21:03.416211 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:03.416094    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-776964c7b7-z4wzv" event={"ID":"fea2175d-60d6-4fca-b26f-c935babe00c1","Type":"ContainerStarted","Data":"6c0013dfcd05b5f846a056ffa56064d3d61b6bd25f1cfc15257131e938e10a66"}
Jul 22 02:21:03.416211 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:03.416138    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-776964c7b7-z4wzv" event={"ID":"fea2175d-60d6-4fca-b26f-c935babe00c1","Type":"ContainerStarted","Data":"13b5fe96a6b8d4c57723810426a7baee8923cc8cb79e39e51a650cb4bcf4cf9a"}
Jul 22 02:21:03.418177 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:03.418005    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wvc9x" event={"ID":"91bd715b-d6dc-4447-85c2-74a058a4478e","Type":"ContainerStarted","Data":"0e7c52be5a1ae25fdfedecf44294a1983cbdebb7f4fef4d74e672654a8ec7fc3"}
Jul 22 02:21:03.442990 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:03.442942    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:21:04.428411 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:04.428316    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8dd7cb2e-66c4-40af-8c25-bb81184d688a","Type":"ContainerStarted","Data":"903004def3d5e48e784ce0c6bf89cdfca93c387cceb0c3ffb0a9676d136e382b"}
Jul 22 02:21:04.436537 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:04.436397    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-776964c7b7-z4wzv" event={"ID":"fea2175d-60d6-4fca-b26f-c935babe00c1","Type":"ContainerStarted","Data":"b82d1a67fd60767f7b5a56b4e5d8f97f5b27880f2e9ddd83af7d48572e2ab888"}
Jul 22 02:21:04.436537 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:04.436479    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-776964c7b7-z4wzv" event={"ID":"fea2175d-60d6-4fca-b26f-c935babe00c1","Type":"ContainerStarted","Data":"50e43717c0330077fbd9b3965be99b6c4391cb5246fbf5d2cc93a42744f5de30"}
Jul 22 02:21:04.459411 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:04.459194    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.068394457 podStartE2EDuration="7.459171917s" totalImagesPullingTime="4.356417363s" totalInitContainerRuntime="34.360097ms" isStatefulPod=false podCreationTimestamp="2026-07-22 02:20:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:21:04.456974085 +0000 UTC m=+85.997767714" watchObservedRunningTime="2026-07-22 02:21:04.459171917 +0000 UTC m=+85.999965525"
Jul 22 02:21:05.115785 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:05.115708    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:21:05.248682 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:05.248645    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:21:05.446269 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:05.446228    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-78d6957c46-pqvxl" event={"ID":"46360a1c-c876-4fd3-ac0c-09956174bebd","Type":"ContainerStarted","Data":"583bd84e41e91574f7b3e341b198774f3747e73522924db7d6e4c3fa57709aa5"}
Jul 22 02:21:05.451218 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:05.451187    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-776964c7b7-z4wzv" event={"ID":"fea2175d-60d6-4fca-b26f-c935babe00c1","Type":"ContainerStarted","Data":"90d63328d41b6dd123e98bc271248dc8f3a2e33392564b14da8300f093cf434d"}
Jul 22 02:21:05.451543 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:05.451518    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-776964c7b7-z4wzv"
Jul 22 02:21:05.453819 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:05.453791    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wvc9x" event={"ID":"91bd715b-d6dc-4447-85c2-74a058a4478e","Type":"ContainerStarted","Data":"43f513c5004c3e9fefea8c1ff9a1924aed8e0116ca4b47ac95edca77e42bd1c8"}
Jul 22 02:21:05.479378 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:05.479292    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-78d6957c46-pqvxl" podStartSLOduration=2.149333635 podStartE2EDuration="4.479268403s" totalImagesPullingTime="2.329934768s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:21:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:21:05.475850569 +0000 UTC m=+87.016644197" watchObservedRunningTime="2026-07-22 02:21:05.479268403 +0000 UTC m=+87.020062031"
Jul 22 02:21:05.503423 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:05.503329    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:21:05.503609 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:05.503560    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-776964c7b7-z4wzv" podStartSLOduration=3.097666663 podStartE2EDuration="6.503540418s" totalImagesPullingTime="3.405873755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:20:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:21:05.501525691 +0000 UTC m=+87.042319319" watchObservedRunningTime="2026-07-22 02:21:05.503540418 +0000 UTC m=+87.044334046"
Jul 22 02:21:05.525833 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:05.525773    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wvc9x" podStartSLOduration=2.164685566 podStartE2EDuration="4.525749612s" totalImagesPullingTime="2.361064046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:21:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:21:05.524379586 +0000 UTC m=+87.065173214" watchObservedRunningTime="2026-07-22 02:21:05.525749612 +0000 UTC m=+87.066543238"
Jul 22 02:21:05.620205 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:05.620090    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:21:05.718302 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:05.718262    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:21:06.135772 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:06.135716    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:21:06.212290 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:06.212251    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:21:06.459880 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:06.459782    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wvc9x"
Jul 22 02:21:06.466865 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:06.466814    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wvc9x"
Jul 22 02:21:07.473118 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:07.472932    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"128db35e-9df1-4698-ae00-0dad30ec97e7","Type":"ContainerStarted","Data":"a14de2feca99ab793d32ebe371982df560fa6f1dba2f08d39b15c6c48848eb54"}
Jul 22 02:21:07.473118 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:07.473004    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"128db35e-9df1-4698-ae00-0dad30ec97e7","Type":"ContainerStarted","Data":"ca394967362963f62bd1f19bd6ed7904385b377e940f7b87afb5f5f478aa793f"}
Jul 22 02:21:07.488389 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:07.488251    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:21:07.701421 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:07.701340    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:21:08.482984 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:08.482945    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"128db35e-9df1-4698-ae00-0dad30ec97e7","Type":"ContainerStarted","Data":"711cba6b59e5343647fff2d80c63adea86b403b2e316ead9b9a3b5092664ea84"}
Jul 22 02:21:08.483462 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:08.483066    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"128db35e-9df1-4698-ae00-0dad30ec97e7","Type":"ContainerStarted","Data":"7f6fc007612fa41b56c072a015370066789b82cba436063cf2dc718735b84746"}
Jul 22 02:21:08.483462 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:08.483116    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"128db35e-9df1-4698-ae00-0dad30ec97e7","Type":"ContainerStarted","Data":"24a299e0b140c13219a1b56d45d4411a382c45bceb5b982184b6938d13bea805"}
Jul 22 02:21:08.483462 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:08.483151    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"128db35e-9df1-4698-ae00-0dad30ec97e7","Type":"ContainerStarted","Data":"576c3208c45ef0d0ebb19e53f5816a2b6b2ae3681798c6536ba3cbbb017f8847"}
Jul 22 02:21:08.524233 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:08.524169    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.660908193 podStartE2EDuration="6.524145361s" totalImagesPullingTime="3.833894625s" totalInitContainerRuntime="29.342543ms" isStatefulPod=false podCreationTimestamp="2026-07-22 02:21:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:21:08.520988363 +0000 UTC m=+90.061782023" watchObservedRunningTime="2026-07-22 02:21:08.524145361 +0000 UTC m=+90.064938991"
Jul 22 02:21:11.466634 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:11.466587    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-776964c7b7-z4wzv"
Jul 22 02:21:12.166753 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:12.166716    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:21:12.225971 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:12.225929    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:21:12.576986 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:12.576929    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:21:13.054883 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:13.054845    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:21:16.292598 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:16.292569    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:21:16.394165 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:16.394137    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:21:16.524979 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:16.524945    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-dkd8c" event={"ID":"121417f5-c17d-457c-a8f1-6ba982980cec","Type":"ContainerStarted","Data":"4fb13cd959d80633b0d834283ab7e64ab1fc5d727de0d959bc2e7c2b08ce10b6"}
Jul 22 02:21:16.525207 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:16.525192    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-dkd8c"
Jul 22 02:21:16.526340 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:16.526318    2581 patch_prober.go:28] interesting pod/downloads-5bc67dcd46-dkd8c container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.134.0.11:8080/\": dial tcp 10.134.0.11:8080: connect: connection refused" start-of-body=
Jul 22 02:21:16.526477 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:16.526372    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-5bc67dcd46-dkd8c" podUID="121417f5-c17d-457c-a8f1-6ba982980cec" containerName="download-server" probeResult="failure" output="Get \"http://10.134.0.11:8080/\": dial tcp 10.134.0.11:8080: connect: connection refused"
Jul 22 02:21:16.539860 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:16.539831    2581 patch_prober.go:28] interesting pod/downloads-5bc67dcd46-dkd8c container/download-server namespace/openshift-console: Liveness probe status=failure output="Get \"http://10.134.0.11:8080/\": dial tcp 10.134.0.11:8080: connect: connection refused" start-of-body=
Jul 22 02:21:16.539950 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:16.539892    2581 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-console/downloads-5bc67dcd46-dkd8c" podUID="121417f5-c17d-457c-a8f1-6ba982980cec" containerName="download-server" probeResult="failure" output="Get \"http://10.134.0.11:8080/\": dial tcp 10.134.0.11:8080: connect: connection refused"
Jul 22 02:21:16.547587 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:16.547545    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-dkd8c" podStartSLOduration=1.048235477 podStartE2EDuration="20.547533961s" totalImagesPullingTime="19.499298484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:20:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:21:16.546122378 +0000 UTC m=+98.086916006" watchObservedRunningTime="2026-07-22 02:21:16.547533961 +0000 UTC m=+98.088327588"
Jul 22 02:21:17.547055 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:17.547018    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-dkd8c"
Jul 22 02:21:18.312073 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:18.312034    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:21:18.374615 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:18.374567    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:21:20.088466 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:20.088425    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:21:20.148895 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:20.148858    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:21:21.553108 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:21.553076    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-78d6957c46-pqvxl"
Jul 22 02:21:21.553537 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:21.553144    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-78d6957c46-pqvxl"
Jul 22 02:21:25.325211 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:25.325181    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-bsnsc"
Jul 22 02:21:25.790106 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:25.790067    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:21:27.167460 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:27.167429    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:21:27.267847 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:27.267819    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:21:28.887800 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:28.887767    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:21:29.484982 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:29.484948    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:21:29.541701 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:29.541672    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:21:41.558160 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:41.558129    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-78d6957c46-pqvxl"
Jul 22 02:21:41.561943 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:21:41.561922    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-78d6957c46-pqvxl"
Jul 22 02:22:03.054116 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:03.054080    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:03.114160 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:03.114132    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:03.745559 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:03.745535    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:15.663073 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:15.663036    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:22:15.760557 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:15.760522    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:22:16.678308 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:16.678275    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 02:22:16.678693 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:16.678579    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8dd7cb2e-66c4-40af-8c25-bb81184d688a" containerName="alertmanager" containerID="cri-o://6f67bcf657232aa6d5fc4035595866cef046bd8fcc1f419763bd1fb27c11903d" gracePeriod=120
Jul 22 02:22:16.678813 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:16.678676    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8dd7cb2e-66c4-40af-8c25-bb81184d688a" containerName="kube-rbac-proxy-metric" containerID="cri-o://9cce0b2c0b502bff41b98ce190bf20417a682fd56f3a07dd4e95786ce96af074" gracePeriod=120
Jul 22 02:22:16.678813 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:16.678676    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8dd7cb2e-66c4-40af-8c25-bb81184d688a" containerName="prom-label-proxy" containerID="cri-o://903004def3d5e48e784ce0c6bf89cdfca93c387cceb0c3ffb0a9676d136e382b" gracePeriod=120
Jul 22 02:22:16.678813 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:16.678708    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8dd7cb2e-66c4-40af-8c25-bb81184d688a" containerName="kube-rbac-proxy-web" containerID="cri-o://5567c4f045dccdf516a0322a42faa63336842a7bebc0a8b305f3c3e50e94a713" gracePeriod=120
Jul 22 02:22:16.678813 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:16.678766    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8dd7cb2e-66c4-40af-8c25-bb81184d688a" containerName="kube-rbac-proxy" containerID="cri-o://0de116cd82f84555313649e4eeb051956a3d6361dc1af5f4f0648831753ea7ae" gracePeriod=120
Jul 22 02:22:16.678813 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:16.678781    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8dd7cb2e-66c4-40af-8c25-bb81184d688a" containerName="config-reloader" containerID="cri-o://b6e7c714bcf99417f16e16feabeec127f3f6eb8a5870755283a1011f84e2932f" gracePeriod=120
Jul 22 02:22:16.787312 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:16.787254    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:22:17.793339 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:17.793312    2581 generic.go:356] "Generic (PLEG): container finished" podID="8dd7cb2e-66c4-40af-8c25-bb81184d688a" containerID="903004def3d5e48e784ce0c6bf89cdfca93c387cceb0c3ffb0a9676d136e382b" exitCode=0
Jul 22 02:22:17.793339 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:17.793336    2581 generic.go:356] "Generic (PLEG): container finished" podID="8dd7cb2e-66c4-40af-8c25-bb81184d688a" containerID="0de116cd82f84555313649e4eeb051956a3d6361dc1af5f4f0648831753ea7ae" exitCode=0
Jul 22 02:22:17.793339 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:17.793344    2581 generic.go:356] "Generic (PLEG): container finished" podID="8dd7cb2e-66c4-40af-8c25-bb81184d688a" containerID="b6e7c714bcf99417f16e16feabeec127f3f6eb8a5870755283a1011f84e2932f" exitCode=0
Jul 22 02:22:17.793715 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:17.793350    2581 generic.go:356] "Generic (PLEG): container finished" podID="8dd7cb2e-66c4-40af-8c25-bb81184d688a" containerID="6f67bcf657232aa6d5fc4035595866cef046bd8fcc1f419763bd1fb27c11903d" exitCode=0
Jul 22 02:22:17.793715 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:17.793381    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8dd7cb2e-66c4-40af-8c25-bb81184d688a","Type":"ContainerDied","Data":"903004def3d5e48e784ce0c6bf89cdfca93c387cceb0c3ffb0a9676d136e382b"}
Jul 22 02:22:17.793715 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:17.793444    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8dd7cb2e-66c4-40af-8c25-bb81184d688a","Type":"ContainerDied","Data":"0de116cd82f84555313649e4eeb051956a3d6361dc1af5f4f0648831753ea7ae"}
Jul 22 02:22:17.793715 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:17.793476    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8dd7cb2e-66c4-40af-8c25-bb81184d688a","Type":"ContainerDied","Data":"b6e7c714bcf99417f16e16feabeec127f3f6eb8a5870755283a1011f84e2932f"}
Jul 22 02:22:17.793715 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:17.793501    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8dd7cb2e-66c4-40af-8c25-bb81184d688a","Type":"ContainerDied","Data":"6f67bcf657232aa6d5fc4035595866cef046bd8fcc1f419763bd1fb27c11903d"}
Jul 22 02:22:17.927796 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:17.927771    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:18.027554 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.027525    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rrdq9\" (UniqueName: \"kubernetes.io/projected/8dd7cb2e-66c4-40af-8c25-bb81184d688a-kube-api-access-rrdq9\") pod \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") "
Jul 22 02:22:18.027727 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.027566    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-secret-alertmanager-kube-rbac-proxy\") pod \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") "
Jul 22 02:22:18.027727 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.027590    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8dd7cb2e-66c4-40af-8c25-bb81184d688a-tls-assets\") pod \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") "
Jul 22 02:22:18.027727 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.027610    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8dd7cb2e-66c4-40af-8c25-bb81184d688a-alertmanager-trusted-ca-bundle\") pod \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") "
Jul 22 02:22:18.027939 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.027762    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-config-volume\") pod \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") "
Jul 22 02:22:18.027939 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.027808    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-web-config\") pod \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") "
Jul 22 02:22:18.027939 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.027842    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8dd7cb2e-66c4-40af-8c25-bb81184d688a-alertmanager-main-db\") pod \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") "
Jul 22 02:22:18.027939 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.027872    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") "
Jul 22 02:22:18.027939 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.027913    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-secret-alertmanager-kube-rbac-proxy-web\") pod \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") "
Jul 22 02:22:18.028188 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.027941    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-cluster-tls-config\") pod \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") "
Jul 22 02:22:18.028188 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.027984    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-secret-alertmanager-main-tls\") pod \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") "
Jul 22 02:22:18.028188 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.028015    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8dd7cb2e-66c4-40af-8c25-bb81184d688a-metrics-client-ca\") pod \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") "
Jul 22 02:22:18.028188 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.028076    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8dd7cb2e-66c4-40af-8c25-bb81184d688a-config-out\") pod \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\" (UID: \"8dd7cb2e-66c4-40af-8c25-bb81184d688a\") "
Jul 22 02:22:18.028188 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.028137    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8dd7cb2e-66c4-40af-8c25-bb81184d688a-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "8dd7cb2e-66c4-40af-8c25-bb81184d688a" (UID: "8dd7cb2e-66c4-40af-8c25-bb81184d688a"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 02:22:18.028188 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.028141    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dd7cb2e-66c4-40af-8c25-bb81184d688a-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "8dd7cb2e-66c4-40af-8c25-bb81184d688a" (UID: "8dd7cb2e-66c4-40af-8c25-bb81184d688a"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:22:18.028490 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.028407    2581 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8dd7cb2e-66c4-40af-8c25-bb81184d688a-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:22:18.028490 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.028431    2581 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8dd7cb2e-66c4-40af-8c25-bb81184d688a-alertmanager-main-db\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:22:18.029206 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.028589    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8dd7cb2e-66c4-40af-8c25-bb81184d688a-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "8dd7cb2e-66c4-40af-8c25-bb81184d688a" (UID: "8dd7cb2e-66c4-40af-8c25-bb81184d688a"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 02:22:18.030679 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.030638    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dd7cb2e-66c4-40af-8c25-bb81184d688a-kube-api-access-rrdq9" (OuterVolumeSpecName: "kube-api-access-rrdq9") pod "8dd7cb2e-66c4-40af-8c25-bb81184d688a" (UID: "8dd7cb2e-66c4-40af-8c25-bb81184d688a"). InnerVolumeSpecName "kube-api-access-rrdq9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 02:22:18.030907 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.030881    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dd7cb2e-66c4-40af-8c25-bb81184d688a-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "8dd7cb2e-66c4-40af-8c25-bb81184d688a" (UID: "8dd7cb2e-66c4-40af-8c25-bb81184d688a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 02:22:18.031173 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.031146    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "8dd7cb2e-66c4-40af-8c25-bb81184d688a" (UID: "8dd7cb2e-66c4-40af-8c25-bb81184d688a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 02:22:18.031276 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.031213    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dd7cb2e-66c4-40af-8c25-bb81184d688a-config-out" (OuterVolumeSpecName: "config-out") pod "8dd7cb2e-66c4-40af-8c25-bb81184d688a" (UID: "8dd7cb2e-66c4-40af-8c25-bb81184d688a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:22:18.031417 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.031386    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-config-volume" (OuterVolumeSpecName: "config-volume") pod "8dd7cb2e-66c4-40af-8c25-bb81184d688a" (UID: "8dd7cb2e-66c4-40af-8c25-bb81184d688a"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 02:22:18.032107 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.032086    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "8dd7cb2e-66c4-40af-8c25-bb81184d688a" (UID: "8dd7cb2e-66c4-40af-8c25-bb81184d688a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 02:22:18.032539 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.032522    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "8dd7cb2e-66c4-40af-8c25-bb81184d688a" (UID: "8dd7cb2e-66c4-40af-8c25-bb81184d688a"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 02:22:18.033879 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.033856    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "8dd7cb2e-66c4-40af-8c25-bb81184d688a" (UID: "8dd7cb2e-66c4-40af-8c25-bb81184d688a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 02:22:18.035641 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.035587    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "8dd7cb2e-66c4-40af-8c25-bb81184d688a" (UID: "8dd7cb2e-66c4-40af-8c25-bb81184d688a"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 02:22:18.041385 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.041363    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-web-config" (OuterVolumeSpecName: "web-config") pod "8dd7cb2e-66c4-40af-8c25-bb81184d688a" (UID: "8dd7cb2e-66c4-40af-8c25-bb81184d688a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 02:22:18.129403 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.129362    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrdq9\" (UniqueName: \"kubernetes.io/projected/8dd7cb2e-66c4-40af-8c25-bb81184d688a-kube-api-access-rrdq9\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:22:18.129403 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.129398    2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:22:18.129403 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.129408    2581 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8dd7cb2e-66c4-40af-8c25-bb81184d688a-tls-assets\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:22:18.129633 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.129418    2581 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-config-volume\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:22:18.129633 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.129427    2581 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-web-config\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:22:18.129633 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.129435    2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:22:18.129633 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.129445    2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:22:18.129633 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.129454    2581 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-cluster-tls-config\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:22:18.129633 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.129462    2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8dd7cb2e-66c4-40af-8c25-bb81184d688a-secret-alertmanager-main-tls\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:22:18.129633 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.129470    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8dd7cb2e-66c4-40af-8c25-bb81184d688a-metrics-client-ca\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:22:18.129633 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.129478    2581 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8dd7cb2e-66c4-40af-8c25-bb81184d688a-config-out\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:22:18.800285 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.800249    2581 generic.go:356] "Generic (PLEG): container finished" podID="8dd7cb2e-66c4-40af-8c25-bb81184d688a" containerID="9cce0b2c0b502bff41b98ce190bf20417a682fd56f3a07dd4e95786ce96af074" exitCode=0
Jul 22 02:22:18.800285 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.800282    2581 generic.go:356] "Generic (PLEG): container finished" podID="8dd7cb2e-66c4-40af-8c25-bb81184d688a" containerID="5567c4f045dccdf516a0322a42faa63336842a7bebc0a8b305f3c3e50e94a713" exitCode=0
Jul 22 02:22:18.800688 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.800328    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8dd7cb2e-66c4-40af-8c25-bb81184d688a","Type":"ContainerDied","Data":"9cce0b2c0b502bff41b98ce190bf20417a682fd56f3a07dd4e95786ce96af074"}
Jul 22 02:22:18.800688 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.800401    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8dd7cb2e-66c4-40af-8c25-bb81184d688a","Type":"ContainerDied","Data":"5567c4f045dccdf516a0322a42faa63336842a7bebc0a8b305f3c3e50e94a713"}
Jul 22 02:22:18.800688 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.800341    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:18.800688 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.800429    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8dd7cb2e-66c4-40af-8c25-bb81184d688a","Type":"ContainerDied","Data":"b5e8cab89b5409036937090c55c189d64468b4f4f163caac2236bf0c1fe5018a"}
Jul 22 02:22:18.800688 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.800461    2581 scope.go:117] "RemoveContainer" containerID="903004def3d5e48e784ce0c6bf89cdfca93c387cceb0c3ffb0a9676d136e382b"
Jul 22 02:22:18.811427 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.811408    2581 scope.go:117] "RemoveContainer" containerID="9cce0b2c0b502bff41b98ce190bf20417a682fd56f3a07dd4e95786ce96af074"
Jul 22 02:22:18.818909 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.818892    2581 scope.go:117] "RemoveContainer" containerID="0de116cd82f84555313649e4eeb051956a3d6361dc1af5f4f0648831753ea7ae"
Jul 22 02:22:18.825992 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.825969    2581 scope.go:117] "RemoveContainer" containerID="5567c4f045dccdf516a0322a42faa63336842a7bebc0a8b305f3c3e50e94a713"
Jul 22 02:22:18.828135 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.828113    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 02:22:18.831892 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.831872    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 02:22:18.834060 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.834046    2581 scope.go:117] "RemoveContainer" containerID="b6e7c714bcf99417f16e16feabeec127f3f6eb8a5870755283a1011f84e2932f"
Jul 22 02:22:18.841001 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.840986    2581 scope.go:117] "RemoveContainer" containerID="6f67bcf657232aa6d5fc4035595866cef046bd8fcc1f419763bd1fb27c11903d"
Jul 22 02:22:18.847790 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.847770    2581 scope.go:117] "RemoveContainer" containerID="896826a169b9c822552ac4b3020fe5f4ae1393bc31977605457d7634a7b1437b"
Jul 22 02:22:18.854619 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.854604    2581 scope.go:117] "RemoveContainer" containerID="903004def3d5e48e784ce0c6bf89cdfca93c387cceb0c3ffb0a9676d136e382b"
Jul 22 02:22:18.856409 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:22:18.856384    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"903004def3d5e48e784ce0c6bf89cdfca93c387cceb0c3ffb0a9676d136e382b\": container with ID starting with 903004def3d5e48e784ce0c6bf89cdfca93c387cceb0c3ffb0a9676d136e382b not found: ID does not exist" containerID="903004def3d5e48e784ce0c6bf89cdfca93c387cceb0c3ffb0a9676d136e382b"
Jul 22 02:22:18.856480 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.856420    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"903004def3d5e48e784ce0c6bf89cdfca93c387cceb0c3ffb0a9676d136e382b"} err="failed to get container status \"903004def3d5e48e784ce0c6bf89cdfca93c387cceb0c3ffb0a9676d136e382b\": rpc error: code = NotFound desc = could not find container \"903004def3d5e48e784ce0c6bf89cdfca93c387cceb0c3ffb0a9676d136e382b\": container with ID starting with 903004def3d5e48e784ce0c6bf89cdfca93c387cceb0c3ffb0a9676d136e382b not found: ID does not exist"
Jul 22 02:22:18.856480 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.856464    2581 scope.go:117] "RemoveContainer" containerID="9cce0b2c0b502bff41b98ce190bf20417a682fd56f3a07dd4e95786ce96af074"
Jul 22 02:22:18.856835 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:22:18.856814    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cce0b2c0b502bff41b98ce190bf20417a682fd56f3a07dd4e95786ce96af074\": container with ID starting with 9cce0b2c0b502bff41b98ce190bf20417a682fd56f3a07dd4e95786ce96af074 not found: ID does not exist" containerID="9cce0b2c0b502bff41b98ce190bf20417a682fd56f3a07dd4e95786ce96af074"
Jul 22 02:22:18.856882 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.856841    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cce0b2c0b502bff41b98ce190bf20417a682fd56f3a07dd4e95786ce96af074"} err="failed to get container status \"9cce0b2c0b502bff41b98ce190bf20417a682fd56f3a07dd4e95786ce96af074\": rpc error: code = NotFound desc = could not find container \"9cce0b2c0b502bff41b98ce190bf20417a682fd56f3a07dd4e95786ce96af074\": container with ID starting with 9cce0b2c0b502bff41b98ce190bf20417a682fd56f3a07dd4e95786ce96af074 not found: ID does not exist"
Jul 22 02:22:18.856882 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.856864    2581 scope.go:117] "RemoveContainer" containerID="0de116cd82f84555313649e4eeb051956a3d6361dc1af5f4f0648831753ea7ae"
Jul 22 02:22:18.857113 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:22:18.857096    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0de116cd82f84555313649e4eeb051956a3d6361dc1af5f4f0648831753ea7ae\": container with ID starting with 0de116cd82f84555313649e4eeb051956a3d6361dc1af5f4f0648831753ea7ae not found: ID does not exist" containerID="0de116cd82f84555313649e4eeb051956a3d6361dc1af5f4f0648831753ea7ae"
Jul 22 02:22:18.857182 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.857117    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0de116cd82f84555313649e4eeb051956a3d6361dc1af5f4f0648831753ea7ae"} err="failed to get container status \"0de116cd82f84555313649e4eeb051956a3d6361dc1af5f4f0648831753ea7ae\": rpc error: code = NotFound desc = could not find container \"0de116cd82f84555313649e4eeb051956a3d6361dc1af5f4f0648831753ea7ae\": container with ID starting with 0de116cd82f84555313649e4eeb051956a3d6361dc1af5f4f0648831753ea7ae not found: ID does not exist"
Jul 22 02:22:18.857182 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.857130    2581 scope.go:117] "RemoveContainer" containerID="5567c4f045dccdf516a0322a42faa63336842a7bebc0a8b305f3c3e50e94a713"
Jul 22 02:22:18.857334 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:22:18.857316    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5567c4f045dccdf516a0322a42faa63336842a7bebc0a8b305f3c3e50e94a713\": container with ID starting with 5567c4f045dccdf516a0322a42faa63336842a7bebc0a8b305f3c3e50e94a713 not found: ID does not exist" containerID="5567c4f045dccdf516a0322a42faa63336842a7bebc0a8b305f3c3e50e94a713"
Jul 22 02:22:18.857393 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.857343    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5567c4f045dccdf516a0322a42faa63336842a7bebc0a8b305f3c3e50e94a713"} err="failed to get container status \"5567c4f045dccdf516a0322a42faa63336842a7bebc0a8b305f3c3e50e94a713\": rpc error: code = NotFound desc = could not find container \"5567c4f045dccdf516a0322a42faa63336842a7bebc0a8b305f3c3e50e94a713\": container with ID starting with 5567c4f045dccdf516a0322a42faa63336842a7bebc0a8b305f3c3e50e94a713 not found: ID does not exist"
Jul 22 02:22:18.857393 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.857367    2581 scope.go:117] "RemoveContainer" containerID="b6e7c714bcf99417f16e16feabeec127f3f6eb8a5870755283a1011f84e2932f"
Jul 22 02:22:18.857631 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:22:18.857616    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6e7c714bcf99417f16e16feabeec127f3f6eb8a5870755283a1011f84e2932f\": container with ID starting with b6e7c714bcf99417f16e16feabeec127f3f6eb8a5870755283a1011f84e2932f not found: ID does not exist" containerID="b6e7c714bcf99417f16e16feabeec127f3f6eb8a5870755283a1011f84e2932f"
Jul 22 02:22:18.857694 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.857637    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6e7c714bcf99417f16e16feabeec127f3f6eb8a5870755283a1011f84e2932f"} err="failed to get container status \"b6e7c714bcf99417f16e16feabeec127f3f6eb8a5870755283a1011f84e2932f\": rpc error: code = NotFound desc = could not find container \"b6e7c714bcf99417f16e16feabeec127f3f6eb8a5870755283a1011f84e2932f\": container with ID starting with b6e7c714bcf99417f16e16feabeec127f3f6eb8a5870755283a1011f84e2932f not found: ID does not exist"
Jul 22 02:22:18.857694 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.857655    2581 scope.go:117] "RemoveContainer" containerID="6f67bcf657232aa6d5fc4035595866cef046bd8fcc1f419763bd1fb27c11903d"
Jul 22 02:22:18.857935 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:22:18.857906    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f67bcf657232aa6d5fc4035595866cef046bd8fcc1f419763bd1fb27c11903d\": container with ID starting with 6f67bcf657232aa6d5fc4035595866cef046bd8fcc1f419763bd1fb27c11903d not found: ID does not exist" containerID="6f67bcf657232aa6d5fc4035595866cef046bd8fcc1f419763bd1fb27c11903d"
Jul 22 02:22:18.857935 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.857931    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f67bcf657232aa6d5fc4035595866cef046bd8fcc1f419763bd1fb27c11903d"} err="failed to get container status \"6f67bcf657232aa6d5fc4035595866cef046bd8fcc1f419763bd1fb27c11903d\": rpc error: code = NotFound desc = could not find container \"6f67bcf657232aa6d5fc4035595866cef046bd8fcc1f419763bd1fb27c11903d\": container with ID starting with 6f67bcf657232aa6d5fc4035595866cef046bd8fcc1f419763bd1fb27c11903d not found: ID does not exist"
Jul 22 02:22:18.858063 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.857947    2581 scope.go:117] "RemoveContainer" containerID="896826a169b9c822552ac4b3020fe5f4ae1393bc31977605457d7634a7b1437b"
Jul 22 02:22:18.858186 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:22:18.858167    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"896826a169b9c822552ac4b3020fe5f4ae1393bc31977605457d7634a7b1437b\": container with ID starting with 896826a169b9c822552ac4b3020fe5f4ae1393bc31977605457d7634a7b1437b not found: ID does not exist" containerID="896826a169b9c822552ac4b3020fe5f4ae1393bc31977605457d7634a7b1437b"
Jul 22 02:22:18.858238 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.858190    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"896826a169b9c822552ac4b3020fe5f4ae1393bc31977605457d7634a7b1437b"} err="failed to get container status \"896826a169b9c822552ac4b3020fe5f4ae1393bc31977605457d7634a7b1437b\": rpc error: code = NotFound desc = could not find container \"896826a169b9c822552ac4b3020fe5f4ae1393bc31977605457d7634a7b1437b\": container with ID starting with 896826a169b9c822552ac4b3020fe5f4ae1393bc31977605457d7634a7b1437b not found: ID does not exist"
Jul 22 02:22:18.858238 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.858204    2581 scope.go:117] "RemoveContainer" containerID="903004def3d5e48e784ce0c6bf89cdfca93c387cceb0c3ffb0a9676d136e382b"
Jul 22 02:22:18.858756 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.858585    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"903004def3d5e48e784ce0c6bf89cdfca93c387cceb0c3ffb0a9676d136e382b"} err="failed to get container status \"903004def3d5e48e784ce0c6bf89cdfca93c387cceb0c3ffb0a9676d136e382b\": rpc error: code = NotFound desc = could not find container \"903004def3d5e48e784ce0c6bf89cdfca93c387cceb0c3ffb0a9676d136e382b\": container with ID starting with 903004def3d5e48e784ce0c6bf89cdfca93c387cceb0c3ffb0a9676d136e382b not found: ID does not exist"
Jul 22 02:22:18.858756 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.858610    2581 scope.go:117] "RemoveContainer" containerID="9cce0b2c0b502bff41b98ce190bf20417a682fd56f3a07dd4e95786ce96af074"
Jul 22 02:22:18.858920 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.858871    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cce0b2c0b502bff41b98ce190bf20417a682fd56f3a07dd4e95786ce96af074"} err="failed to get container status \"9cce0b2c0b502bff41b98ce190bf20417a682fd56f3a07dd4e95786ce96af074\": rpc error: code = NotFound desc = could not find container \"9cce0b2c0b502bff41b98ce190bf20417a682fd56f3a07dd4e95786ce96af074\": container with ID starting with 9cce0b2c0b502bff41b98ce190bf20417a682fd56f3a07dd4e95786ce96af074 not found: ID does not exist"
Jul 22 02:22:18.858920 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.858893    2581 scope.go:117] "RemoveContainer" containerID="0de116cd82f84555313649e4eeb051956a3d6361dc1af5f4f0648831753ea7ae"
Jul 22 02:22:18.859902 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.859366    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0de116cd82f84555313649e4eeb051956a3d6361dc1af5f4f0648831753ea7ae"} err="failed to get container status \"0de116cd82f84555313649e4eeb051956a3d6361dc1af5f4f0648831753ea7ae\": rpc error: code = NotFound desc = could not find container \"0de116cd82f84555313649e4eeb051956a3d6361dc1af5f4f0648831753ea7ae\": container with ID starting with 0de116cd82f84555313649e4eeb051956a3d6361dc1af5f4f0648831753ea7ae not found: ID does not exist"
Jul 22 02:22:18.859902 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.859392    2581 scope.go:117] "RemoveContainer" containerID="5567c4f045dccdf516a0322a42faa63336842a7bebc0a8b305f3c3e50e94a713"
Jul 22 02:22:18.860095 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.860013    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5567c4f045dccdf516a0322a42faa63336842a7bebc0a8b305f3c3e50e94a713"} err="failed to get container status \"5567c4f045dccdf516a0322a42faa63336842a7bebc0a8b305f3c3e50e94a713\": rpc error: code = NotFound desc = could not find container \"5567c4f045dccdf516a0322a42faa63336842a7bebc0a8b305f3c3e50e94a713\": container with ID starting with 5567c4f045dccdf516a0322a42faa63336842a7bebc0a8b305f3c3e50e94a713 not found: ID does not exist"
Jul 22 02:22:18.860095 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.860057    2581 scope.go:117] "RemoveContainer" containerID="b6e7c714bcf99417f16e16feabeec127f3f6eb8a5870755283a1011f84e2932f"
Jul 22 02:22:18.860426 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.860399    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6e7c714bcf99417f16e16feabeec127f3f6eb8a5870755283a1011f84e2932f"} err="failed to get container status \"b6e7c714bcf99417f16e16feabeec127f3f6eb8a5870755283a1011f84e2932f\": rpc error: code = NotFound desc = could not find container \"b6e7c714bcf99417f16e16feabeec127f3f6eb8a5870755283a1011f84e2932f\": container with ID starting with b6e7c714bcf99417f16e16feabeec127f3f6eb8a5870755283a1011f84e2932f not found: ID does not exist"
Jul 22 02:22:18.860426 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.860427    2581 scope.go:117] "RemoveContainer" containerID="6f67bcf657232aa6d5fc4035595866cef046bd8fcc1f419763bd1fb27c11903d"
Jul 22 02:22:18.860963 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.860894    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f67bcf657232aa6d5fc4035595866cef046bd8fcc1f419763bd1fb27c11903d"} err="failed to get container status \"6f67bcf657232aa6d5fc4035595866cef046bd8fcc1f419763bd1fb27c11903d\": rpc error: code = NotFound desc = could not find container \"6f67bcf657232aa6d5fc4035595866cef046bd8fcc1f419763bd1fb27c11903d\": container with ID starting with 6f67bcf657232aa6d5fc4035595866cef046bd8fcc1f419763bd1fb27c11903d not found: ID does not exist"
Jul 22 02:22:18.860963 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.860918    2581 scope.go:117] "RemoveContainer" containerID="896826a169b9c822552ac4b3020fe5f4ae1393bc31977605457d7634a7b1437b"
Jul 22 02:22:18.861167 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.861142    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"896826a169b9c822552ac4b3020fe5f4ae1393bc31977605457d7634a7b1437b"} err="failed to get container status \"896826a169b9c822552ac4b3020fe5f4ae1393bc31977605457d7634a7b1437b\": rpc error: code = NotFound desc = could not find container \"896826a169b9c822552ac4b3020fe5f4ae1393bc31977605457d7634a7b1437b\": container with ID starting with 896826a169b9c822552ac4b3020fe5f4ae1393bc31977605457d7634a7b1437b not found: ID does not exist"
Jul 22 02:22:18.861479 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.861459    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 02:22:18.861910 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.861895    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dd7cb2e-66c4-40af-8c25-bb81184d688a" containerName="config-reloader"
Jul 22 02:22:18.862000 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.861913    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dd7cb2e-66c4-40af-8c25-bb81184d688a" containerName="config-reloader"
Jul 22 02:22:18.862000 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.861933    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dd7cb2e-66c4-40af-8c25-bb81184d688a" containerName="init-config-reloader"
Jul 22 02:22:18.862000 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.861942    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dd7cb2e-66c4-40af-8c25-bb81184d688a" containerName="init-config-reloader"
Jul 22 02:22:18.862000 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.861949    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dd7cb2e-66c4-40af-8c25-bb81184d688a" containerName="prom-label-proxy"
Jul 22 02:22:18.862000 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.861954    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dd7cb2e-66c4-40af-8c25-bb81184d688a" containerName="prom-label-proxy"
Jul 22 02:22:18.862000 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.861964    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dd7cb2e-66c4-40af-8c25-bb81184d688a" containerName="kube-rbac-proxy-web"
Jul 22 02:22:18.862000 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.861969    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dd7cb2e-66c4-40af-8c25-bb81184d688a" containerName="kube-rbac-proxy-web"
Jul 22 02:22:18.862000 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.861976    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dd7cb2e-66c4-40af-8c25-bb81184d688a" containerName="kube-rbac-proxy"
Jul 22 02:22:18.862000 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.861982    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dd7cb2e-66c4-40af-8c25-bb81184d688a" containerName="kube-rbac-proxy"
Jul 22 02:22:18.862000 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.861988    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dd7cb2e-66c4-40af-8c25-bb81184d688a" containerName="kube-rbac-proxy-metric"
Jul 22 02:22:18.862000 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.861993    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dd7cb2e-66c4-40af-8c25-bb81184d688a" containerName="kube-rbac-proxy-metric"
Jul 22 02:22:18.862000 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.862004    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dd7cb2e-66c4-40af-8c25-bb81184d688a" containerName="alertmanager"
Jul 22 02:22:18.862474 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.862012    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dd7cb2e-66c4-40af-8c25-bb81184d688a" containerName="alertmanager"
Jul 22 02:22:18.862474 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.862094    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8dd7cb2e-66c4-40af-8c25-bb81184d688a" containerName="kube-rbac-proxy"
Jul 22 02:22:18.862474 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.862108    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8dd7cb2e-66c4-40af-8c25-bb81184d688a" containerName="config-reloader"
Jul 22 02:22:18.862474 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.862115    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8dd7cb2e-66c4-40af-8c25-bb81184d688a" containerName="prom-label-proxy"
Jul 22 02:22:18.862474 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.862122    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8dd7cb2e-66c4-40af-8c25-bb81184d688a" containerName="alertmanager"
Jul 22 02:22:18.862474 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.862128    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8dd7cb2e-66c4-40af-8c25-bb81184d688a" containerName="kube-rbac-proxy-metric"
Jul 22 02:22:18.862474 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.862134    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8dd7cb2e-66c4-40af-8c25-bb81184d688a" containerName="kube-rbac-proxy-web"
Jul 22 02:22:18.865635 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.865617    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:18.868647 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.868623    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 22 02:22:18.869080 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.868690    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 22 02:22:18.869080 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.868934    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 22 02:22:18.869080 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.868934    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 22 02:22:18.869080 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.868958    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 22 02:22:18.869560 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.869503    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 22 02:22:18.869560 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.869533    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 22 02:22:18.869560 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.869558    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 22 02:22:18.869757 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.869566    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-h8rvs\""
Jul 22 02:22:18.878825 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.878597    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 22 02:22:18.884850 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.884829    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 02:22:18.901793 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.901732    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8dd7cb2e-66c4-40af-8c25-bb81184d688a" path="/var/lib/kubelet/pods/8dd7cb2e-66c4-40af-8c25-bb81184d688a/volumes"
Jul 22 02:22:18.938562 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.938527    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c5203a3b-a172-418e-8953-233fc2d4ae89-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:18.938778 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.938569    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c5203a3b-a172-418e-8953-233fc2d4ae89-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:18.938778 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.938601    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c5203a3b-a172-418e-8953-233fc2d4ae89-web-config\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:18.938778 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.938669    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c5203a3b-a172-418e-8953-233fc2d4ae89-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:18.938778 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.938701    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c5203a3b-a172-418e-8953-233fc2d4ae89-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:18.938778 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.938725    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lswk9\" (UniqueName: \"kubernetes.io/projected/c5203a3b-a172-418e-8953-233fc2d4ae89-kube-api-access-lswk9\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:18.938941 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.938824    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c5203a3b-a172-418e-8953-233fc2d4ae89-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:18.938941 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.938861    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c5203a3b-a172-418e-8953-233fc2d4ae89-config-out\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:18.938941 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.938877    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c5203a3b-a172-418e-8953-233fc2d4ae89-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:18.938941 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.938916    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c5203a3b-a172-418e-8953-233fc2d4ae89-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:18.939055 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.938952    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c5203a3b-a172-418e-8953-233fc2d4ae89-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:18.939055 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.938977    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c5203a3b-a172-418e-8953-233fc2d4ae89-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:18.939055 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:18.938996    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c5203a3b-a172-418e-8953-233fc2d4ae89-config-volume\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:19.039683 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:19.039645    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c5203a3b-a172-418e-8953-233fc2d4ae89-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:19.039863 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:19.039692    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lswk9\" (UniqueName: \"kubernetes.io/projected/c5203a3b-a172-418e-8953-233fc2d4ae89-kube-api-access-lswk9\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:19.039863 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:19.039771    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c5203a3b-a172-418e-8953-233fc2d4ae89-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:19.039863 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:19.039802    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c5203a3b-a172-418e-8953-233fc2d4ae89-config-out\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:19.039863 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:19.039828    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c5203a3b-a172-418e-8953-233fc2d4ae89-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:19.039863 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:19.039853    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c5203a3b-a172-418e-8953-233fc2d4ae89-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:19.040117 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:19.039883    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c5203a3b-a172-418e-8953-233fc2d4ae89-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:19.040117 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:19.039923    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c5203a3b-a172-418e-8953-233fc2d4ae89-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:19.040117 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:19.039954    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c5203a3b-a172-418e-8953-233fc2d4ae89-config-volume\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:19.040117 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:19.040011    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c5203a3b-a172-418e-8953-233fc2d4ae89-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:19.040117 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:19.040036    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c5203a3b-a172-418e-8953-233fc2d4ae89-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:19.040117 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:19.040062    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c5203a3b-a172-418e-8953-233fc2d4ae89-web-config\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:19.040778 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:19.040727    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c5203a3b-a172-418e-8953-233fc2d4ae89-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:19.040905 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:19.040878    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c5203a3b-a172-418e-8953-233fc2d4ae89-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:19.041137 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:19.041069    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c5203a3b-a172-418e-8953-233fc2d4ae89-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:19.041357 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:19.041334    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c5203a3b-a172-418e-8953-233fc2d4ae89-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:19.042812 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:19.042788    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c5203a3b-a172-418e-8953-233fc2d4ae89-config-out\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:19.043254 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:19.043230    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c5203a3b-a172-418e-8953-233fc2d4ae89-config-volume\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:19.043352 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:19.043331    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c5203a3b-a172-418e-8953-233fc2d4ae89-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:19.043417 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:19.043353    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c5203a3b-a172-418e-8953-233fc2d4ae89-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:19.043471 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:19.043423    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c5203a3b-a172-418e-8953-233fc2d4ae89-web-config\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:19.043512 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:19.043478    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c5203a3b-a172-418e-8953-233fc2d4ae89-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:19.043695 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:19.043677    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c5203a3b-a172-418e-8953-233fc2d4ae89-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:19.043872 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:19.043853    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c5203a3b-a172-418e-8953-233fc2d4ae89-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:19.044852 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:19.044830    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c5203a3b-a172-418e-8953-233fc2d4ae89-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:19.048534 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:19.048517    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lswk9\" (UniqueName: \"kubernetes.io/projected/c5203a3b-a172-418e-8953-233fc2d4ae89-kube-api-access-lswk9\") pod \"alertmanager-main-0\" (UID: \"c5203a3b-a172-418e-8953-233fc2d4ae89\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:19.181104 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:19.181068    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 22 02:22:19.327049 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:19.326989    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 02:22:19.329211 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:22:19.329182    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5203a3b_a172_418e_8953_233fc2d4ae89.slice/crio-08e1bb47024243bf56cdbf7ddf6e0cd39aade053548bce7e97be509b0ab0b996 WatchSource:0}: Error finding container 08e1bb47024243bf56cdbf7ddf6e0cd39aade053548bce7e97be509b0ab0b996: Status 404 returned error can't find the container with id 08e1bb47024243bf56cdbf7ddf6e0cd39aade053548bce7e97be509b0ab0b996
Jul 22 02:22:19.806545 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:19.806509    2581 generic.go:356] "Generic (PLEG): container finished" podID="c5203a3b-a172-418e-8953-233fc2d4ae89" containerID="0a89beaefc5bbbe37df633244d27273c86e81d739d941f0e4b6eac7cca3c7644" exitCode=0
Jul 22 02:22:19.806996 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:19.806593    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c5203a3b-a172-418e-8953-233fc2d4ae89","Type":"ContainerDied","Data":"0a89beaefc5bbbe37df633244d27273c86e81d739d941f0e4b6eac7cca3c7644"}
Jul 22 02:22:19.806996 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:19.806679    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c5203a3b-a172-418e-8953-233fc2d4ae89","Type":"ContainerStarted","Data":"08e1bb47024243bf56cdbf7ddf6e0cd39aade053548bce7e97be509b0ab0b996"}
Jul 22 02:22:20.756549 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.756510    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5ddb9449c-h5zj7"]
Jul 22 02:22:20.760156 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.760124    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5ddb9449c-h5zj7"
Jul 22 02:22:20.763077 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.763053    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 22 02:22:20.763321 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.763304    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 22 02:22:20.763561 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.763538    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 22 02:22:20.763710 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.763577    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 22 02:22:20.763893 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.763859    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 22 02:22:20.763945 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.763919    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-4drnr\""
Jul 22 02:22:20.771233 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.771202    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 22 02:22:20.779981 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.779947    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5ddb9449c-h5zj7"]
Jul 22 02:22:20.816787 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.816713    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c5203a3b-a172-418e-8953-233fc2d4ae89","Type":"ContainerStarted","Data":"f076f6a948bbc888a9491c0def1f2242ea7997268fbb9c4812f7486daaa44330"}
Jul 22 02:22:20.817169 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.816828    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c5203a3b-a172-418e-8953-233fc2d4ae89","Type":"ContainerStarted","Data":"606b0c6832f898f94e6b36ab40aac70fce12e9d6b2bfc3a3a367428426affd12"}
Jul 22 02:22:20.817169 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.816869    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c5203a3b-a172-418e-8953-233fc2d4ae89","Type":"ContainerStarted","Data":"2beddb89fe3546fed2e7850efe4a04fb2921f995a8b881321bd0b230ab53257e"}
Jul 22 02:22:20.817169 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.816906    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c5203a3b-a172-418e-8953-233fc2d4ae89","Type":"ContainerStarted","Data":"4a14eaad7c4eaf22ad2b4edbbf85dabd4f1fe5e2d6f386c7435ebf81abf8b933"}
Jul 22 02:22:20.817169 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.816943    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c5203a3b-a172-418e-8953-233fc2d4ae89","Type":"ContainerStarted","Data":"f176e0400cb749c702c95614f0862c62995d7fb290dc24439feadf0012c63a00"}
Jul 22 02:22:20.817169 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.816977    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c5203a3b-a172-418e-8953-233fc2d4ae89","Type":"ContainerStarted","Data":"9586e2d43272921dc4d62a9e3dd966fbf056b89c1f14c19e7de0e70d52a87f0f"}
Jul 22 02:22:20.846170 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.846115    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.831399092 podStartE2EDuration="2.846096995s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.697903ms" isStatefulPod=false podCreationTimestamp="2026-07-22 02:22:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:22:20.845585276 +0000 UTC m=+162.386378904" watchObservedRunningTime="2026-07-22 02:22:20.846096995 +0000 UTC m=+162.386890622"
Jul 22 02:22:20.858117 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.858086    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/10f71205-4482-4c08-8fc4-e8a17b554b5b-secret-telemeter-client\") pod \"telemeter-client-5ddb9449c-h5zj7\" (UID: \"10f71205-4482-4c08-8fc4-e8a17b554b5b\") " pod="openshift-monitoring/telemeter-client-5ddb9449c-h5zj7"
Jul 22 02:22:20.858287 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.858135    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/10f71205-4482-4c08-8fc4-e8a17b554b5b-metrics-client-ca\") pod \"telemeter-client-5ddb9449c-h5zj7\" (UID: \"10f71205-4482-4c08-8fc4-e8a17b554b5b\") " pod="openshift-monitoring/telemeter-client-5ddb9449c-h5zj7"
Jul 22 02:22:20.858287 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.858217    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/10f71205-4482-4c08-8fc4-e8a17b554b5b-telemeter-client-tls\") pod \"telemeter-client-5ddb9449c-h5zj7\" (UID: \"10f71205-4482-4c08-8fc4-e8a17b554b5b\") " pod="openshift-monitoring/telemeter-client-5ddb9449c-h5zj7"
Jul 22 02:22:20.858287 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.858255    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10f71205-4482-4c08-8fc4-e8a17b554b5b-serving-certs-ca-bundle\") pod \"telemeter-client-5ddb9449c-h5zj7\" (UID: \"10f71205-4482-4c08-8fc4-e8a17b554b5b\") " pod="openshift-monitoring/telemeter-client-5ddb9449c-h5zj7"
Jul 22 02:22:20.858448 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.858288    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/10f71205-4482-4c08-8fc4-e8a17b554b5b-federate-client-tls\") pod \"telemeter-client-5ddb9449c-h5zj7\" (UID: \"10f71205-4482-4c08-8fc4-e8a17b554b5b\") " pod="openshift-monitoring/telemeter-client-5ddb9449c-h5zj7"
Jul 22 02:22:20.858448 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.858335    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/10f71205-4482-4c08-8fc4-e8a17b554b5b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5ddb9449c-h5zj7\" (UID: \"10f71205-4482-4c08-8fc4-e8a17b554b5b\") " pod="openshift-monitoring/telemeter-client-5ddb9449c-h5zj7"
Jul 22 02:22:20.858448 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.858379    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10f71205-4482-4c08-8fc4-e8a17b554b5b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5ddb9449c-h5zj7\" (UID: \"10f71205-4482-4c08-8fc4-e8a17b554b5b\") " pod="openshift-monitoring/telemeter-client-5ddb9449c-h5zj7"
Jul 22 02:22:20.858448 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.858399    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lt5hx\" (UniqueName: \"kubernetes.io/projected/10f71205-4482-4c08-8fc4-e8a17b554b5b-kube-api-access-lt5hx\") pod \"telemeter-client-5ddb9449c-h5zj7\" (UID: \"10f71205-4482-4c08-8fc4-e8a17b554b5b\") " pod="openshift-monitoring/telemeter-client-5ddb9449c-h5zj7"
Jul 22 02:22:20.959116 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.959063    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/10f71205-4482-4c08-8fc4-e8a17b554b5b-federate-client-tls\") pod \"telemeter-client-5ddb9449c-h5zj7\" (UID: \"10f71205-4482-4c08-8fc4-e8a17b554b5b\") " pod="openshift-monitoring/telemeter-client-5ddb9449c-h5zj7"
Jul 22 02:22:20.959328 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.959147    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/10f71205-4482-4c08-8fc4-e8a17b554b5b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5ddb9449c-h5zj7\" (UID: \"10f71205-4482-4c08-8fc4-e8a17b554b5b\") " pod="openshift-monitoring/telemeter-client-5ddb9449c-h5zj7"
Jul 22 02:22:20.959328 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.959256    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10f71205-4482-4c08-8fc4-e8a17b554b5b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5ddb9449c-h5zj7\" (UID: \"10f71205-4482-4c08-8fc4-e8a17b554b5b\") " pod="openshift-monitoring/telemeter-client-5ddb9449c-h5zj7"
Jul 22 02:22:20.959328 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.959299    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lt5hx\" (UniqueName: \"kubernetes.io/projected/10f71205-4482-4c08-8fc4-e8a17b554b5b-kube-api-access-lt5hx\") pod \"telemeter-client-5ddb9449c-h5zj7\" (UID: \"10f71205-4482-4c08-8fc4-e8a17b554b5b\") " pod="openshift-monitoring/telemeter-client-5ddb9449c-h5zj7"
Jul 22 02:22:20.959506 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.959377    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/10f71205-4482-4c08-8fc4-e8a17b554b5b-secret-telemeter-client\") pod \"telemeter-client-5ddb9449c-h5zj7\" (UID: \"10f71205-4482-4c08-8fc4-e8a17b554b5b\") " pod="openshift-monitoring/telemeter-client-5ddb9449c-h5zj7"
Jul 22 02:22:20.959649 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.959623    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/10f71205-4482-4c08-8fc4-e8a17b554b5b-metrics-client-ca\") pod \"telemeter-client-5ddb9449c-h5zj7\" (UID: \"10f71205-4482-4c08-8fc4-e8a17b554b5b\") " pod="openshift-monitoring/telemeter-client-5ddb9449c-h5zj7"
Jul 22 02:22:20.959712 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.959686    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/10f71205-4482-4c08-8fc4-e8a17b554b5b-telemeter-client-tls\") pod \"telemeter-client-5ddb9449c-h5zj7\" (UID: \"10f71205-4482-4c08-8fc4-e8a17b554b5b\") " pod="openshift-monitoring/telemeter-client-5ddb9449c-h5zj7"
Jul 22 02:22:20.959812 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.959797    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10f71205-4482-4c08-8fc4-e8a17b554b5b-serving-certs-ca-bundle\") pod \"telemeter-client-5ddb9449c-h5zj7\" (UID: \"10f71205-4482-4c08-8fc4-e8a17b554b5b\") " pod="openshift-monitoring/telemeter-client-5ddb9449c-h5zj7"
Jul 22 02:22:20.960941 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.960506    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10f71205-4482-4c08-8fc4-e8a17b554b5b-serving-certs-ca-bundle\") pod \"telemeter-client-5ddb9449c-h5zj7\" (UID: \"10f71205-4482-4c08-8fc4-e8a17b554b5b\") " pod="openshift-monitoring/telemeter-client-5ddb9449c-h5zj7"
Jul 22 02:22:20.960941 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.960510    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10f71205-4482-4c08-8fc4-e8a17b554b5b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5ddb9449c-h5zj7\" (UID: \"10f71205-4482-4c08-8fc4-e8a17b554b5b\") " pod="openshift-monitoring/telemeter-client-5ddb9449c-h5zj7"
Jul 22 02:22:20.960941 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.960877    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/10f71205-4482-4c08-8fc4-e8a17b554b5b-metrics-client-ca\") pod \"telemeter-client-5ddb9449c-h5zj7\" (UID: \"10f71205-4482-4c08-8fc4-e8a17b554b5b\") " pod="openshift-monitoring/telemeter-client-5ddb9449c-h5zj7"
Jul 22 02:22:20.962194 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.962114    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/10f71205-4482-4c08-8fc4-e8a17b554b5b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5ddb9449c-h5zj7\" (UID: \"10f71205-4482-4c08-8fc4-e8a17b554b5b\") " pod="openshift-monitoring/telemeter-client-5ddb9449c-h5zj7"
Jul 22 02:22:20.963248 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.962778    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/10f71205-4482-4c08-8fc4-e8a17b554b5b-secret-telemeter-client\") pod \"telemeter-client-5ddb9449c-h5zj7\" (UID: \"10f71205-4482-4c08-8fc4-e8a17b554b5b\") " pod="openshift-monitoring/telemeter-client-5ddb9449c-h5zj7"
Jul 22 02:22:20.963248 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.963208    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/10f71205-4482-4c08-8fc4-e8a17b554b5b-telemeter-client-tls\") pod \"telemeter-client-5ddb9449c-h5zj7\" (UID: \"10f71205-4482-4c08-8fc4-e8a17b554b5b\") " pod="openshift-monitoring/telemeter-client-5ddb9449c-h5zj7"
Jul 22 02:22:20.964041 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.964015    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/10f71205-4482-4c08-8fc4-e8a17b554b5b-federate-client-tls\") pod \"telemeter-client-5ddb9449c-h5zj7\" (UID: \"10f71205-4482-4c08-8fc4-e8a17b554b5b\") " pod="openshift-monitoring/telemeter-client-5ddb9449c-h5zj7"
Jul 22 02:22:20.970985 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:20.970961    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lt5hx\" (UniqueName: \"kubernetes.io/projected/10f71205-4482-4c08-8fc4-e8a17b554b5b-kube-api-access-lt5hx\") pod \"telemeter-client-5ddb9449c-h5zj7\" (UID: \"10f71205-4482-4c08-8fc4-e8a17b554b5b\") " pod="openshift-monitoring/telemeter-client-5ddb9449c-h5zj7"
Jul 22 02:22:21.076216 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.076134    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 02:22:21.076504 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.076447    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="128db35e-9df1-4698-ae00-0dad30ec97e7" containerName="prometheus" containerID="cri-o://ca394967362963f62bd1f19bd6ed7904385b377e940f7b87afb5f5f478aa793f" gracePeriod=600
Jul 22 02:22:21.076656 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.076509    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="128db35e-9df1-4698-ae00-0dad30ec97e7" containerName="config-reloader" containerID="cri-o://a14de2feca99ab793d32ebe371982df560fa6f1dba2f08d39b15c6c48848eb54" gracePeriod=600
Jul 22 02:22:21.076656 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.076519    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="128db35e-9df1-4698-ae00-0dad30ec97e7" containerName="kube-rbac-proxy-thanos" containerID="cri-o://711cba6b59e5343647fff2d80c63adea86b403b2e316ead9b9a3b5092664ea84" gracePeriod=600
Jul 22 02:22:21.076656 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.076479    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="128db35e-9df1-4698-ae00-0dad30ec97e7" containerName="kube-rbac-proxy" containerID="cri-o://7f6fc007612fa41b56c072a015370066789b82cba436063cf2dc718735b84746" gracePeriod=600
Jul 22 02:22:21.076656 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.076521    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="128db35e-9df1-4698-ae00-0dad30ec97e7" containerName="kube-rbac-proxy-web" containerID="cri-o://24a299e0b140c13219a1b56d45d4411a382c45bceb5b982184b6938d13bea805" gracePeriod=600
Jul 22 02:22:21.076896 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.076476    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="128db35e-9df1-4698-ae00-0dad30ec97e7" containerName="thanos-sidecar" containerID="cri-o://576c3208c45ef0d0ebb19e53f5816a2b6b2ae3681798c6536ba3cbbb017f8847" gracePeriod=600
Jul 22 02:22:21.076896 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.076715    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5ddb9449c-h5zj7"
Jul 22 02:22:21.253358 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.253327    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5ddb9449c-h5zj7"]
Jul 22 02:22:21.256371 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:22:21.256336    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10f71205_4482_4c08_8fc4_e8a17b554b5b.slice/crio-30b3a39c63789ee983b98ff0e0117cada4715df23cc0b8367b15d33aa97d2cc7 WatchSource:0}: Error finding container 30b3a39c63789ee983b98ff0e0117cada4715df23cc0b8367b15d33aa97d2cc7: Status 404 returned error can't find the container with id 30b3a39c63789ee983b98ff0e0117cada4715df23cc0b8367b15d33aa97d2cc7
Jul 22 02:22:21.350141 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.350118    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:21.464487 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.464449    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-thanos-prometheus-http-client-file\") pod \"128db35e-9df1-4698-ae00-0dad30ec97e7\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") "
Jul 22 02:22:21.464487 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.464486    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/128db35e-9df1-4698-ae00-0dad30ec97e7-config-out\") pod \"128db35e-9df1-4698-ae00-0dad30ec97e7\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") "
Jul 22 02:22:21.464711 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.464523    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"128db35e-9df1-4698-ae00-0dad30ec97e7\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") "
Jul 22 02:22:21.464711 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.464551    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-configmap-kubelet-serving-ca-bundle\") pod \"128db35e-9df1-4698-ae00-0dad30ec97e7\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") "
Jul 22 02:22:21.464711 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.464580    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vdvgn\" (UniqueName: \"kubernetes.io/projected/128db35e-9df1-4698-ae00-0dad30ec97e7-kube-api-access-vdvgn\") pod \"128db35e-9df1-4698-ae00-0dad30ec97e7\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") "
Jul 22 02:22:21.464711 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.464607    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-web-config\") pod \"128db35e-9df1-4698-ae00-0dad30ec97e7\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") "
Jul 22 02:22:21.464940 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.464725    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-secret-prometheus-k8s-tls\") pod \"128db35e-9df1-4698-ae00-0dad30ec97e7\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") "
Jul 22 02:22:21.464940 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.464789    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-prometheus-k8s-rulefiles-2\") pod \"128db35e-9df1-4698-ae00-0dad30ec97e7\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") "
Jul 22 02:22:21.464940 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.464837    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-configmap-metrics-client-ca\") pod \"128db35e-9df1-4698-ae00-0dad30ec97e7\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") "
Jul 22 02:22:21.464940 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.464875    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-secret-grpc-tls\") pod \"128db35e-9df1-4698-ae00-0dad30ec97e7\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") "
Jul 22 02:22:21.464940 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.464902    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-prometheus-k8s-rulefiles-1\") pod \"128db35e-9df1-4698-ae00-0dad30ec97e7\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") "
Jul 22 02:22:21.464940 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.464930    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"128db35e-9df1-4698-ae00-0dad30ec97e7\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") "
Jul 22 02:22:21.465243 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.464952    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "128db35e-9df1-4698-ae00-0dad30ec97e7" (UID: "128db35e-9df1-4698-ae00-0dad30ec97e7"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 02:22:21.465243 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.464973    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-secret-kube-rbac-proxy\") pod \"128db35e-9df1-4698-ae00-0dad30ec97e7\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") "
Jul 22 02:22:21.465243 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.465015    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-prometheus-trusted-ca-bundle\") pod \"128db35e-9df1-4698-ae00-0dad30ec97e7\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") "
Jul 22 02:22:21.465243 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.465045    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/128db35e-9df1-4698-ae00-0dad30ec97e7-tls-assets\") pod \"128db35e-9df1-4698-ae00-0dad30ec97e7\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") "
Jul 22 02:22:21.465243 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.465075    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-secret-metrics-client-certs\") pod \"128db35e-9df1-4698-ae00-0dad30ec97e7\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") "
Jul 22 02:22:21.465243 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.465106    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-config\") pod \"128db35e-9df1-4698-ae00-0dad30ec97e7\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") "
Jul 22 02:22:21.465243 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.465193    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-configmap-serving-certs-ca-bundle\") pod \"128db35e-9df1-4698-ae00-0dad30ec97e7\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") "
Jul 22 02:22:21.465243 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.465241    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/128db35e-9df1-4698-ae00-0dad30ec97e7-prometheus-k8s-db\") pod \"128db35e-9df1-4698-ae00-0dad30ec97e7\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") "
Jul 22 02:22:21.465720 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.465281    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-prometheus-k8s-rulefiles-0\") pod \"128db35e-9df1-4698-ae00-0dad30ec97e7\" (UID: \"128db35e-9df1-4698-ae00-0dad30ec97e7\") "
Jul 22 02:22:21.465720 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.465575    2581 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:22:21.467445 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.466495    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "128db35e-9df1-4698-ae00-0dad30ec97e7" (UID: "128db35e-9df1-4698-ae00-0dad30ec97e7"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 02:22:21.467445 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.466716    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "128db35e-9df1-4698-ae00-0dad30ec97e7" (UID: "128db35e-9df1-4698-ae00-0dad30ec97e7"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 02:22:21.467445 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.466911    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "128db35e-9df1-4698-ae00-0dad30ec97e7" (UID: "128db35e-9df1-4698-ae00-0dad30ec97e7"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 02:22:21.467445 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.466941    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "128db35e-9df1-4698-ae00-0dad30ec97e7" (UID: "128db35e-9df1-4698-ae00-0dad30ec97e7"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 02:22:21.467445 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.467339    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "128db35e-9df1-4698-ae00-0dad30ec97e7" (UID: "128db35e-9df1-4698-ae00-0dad30ec97e7"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 02:22:21.467445 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.467356    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "128db35e-9df1-4698-ae00-0dad30ec97e7" (UID: "128db35e-9df1-4698-ae00-0dad30ec97e7"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 02:22:21.467445 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.467415    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "128db35e-9df1-4698-ae00-0dad30ec97e7" (UID: "128db35e-9df1-4698-ae00-0dad30ec97e7"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 02:22:21.467445 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.467437    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/128db35e-9df1-4698-ae00-0dad30ec97e7-config-out" (OuterVolumeSpecName: "config-out") pod "128db35e-9df1-4698-ae00-0dad30ec97e7" (UID: "128db35e-9df1-4698-ae00-0dad30ec97e7"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:22:21.468087 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.467832    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "128db35e-9df1-4698-ae00-0dad30ec97e7" (UID: "128db35e-9df1-4698-ae00-0dad30ec97e7"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 02:22:21.468589 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.468553    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "128db35e-9df1-4698-ae00-0dad30ec97e7" (UID: "128db35e-9df1-4698-ae00-0dad30ec97e7"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 02:22:21.468809 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.468782    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "128db35e-9df1-4698-ae00-0dad30ec97e7" (UID: "128db35e-9df1-4698-ae00-0dad30ec97e7"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 02:22:21.468990 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.468961    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/128db35e-9df1-4698-ae00-0dad30ec97e7-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "128db35e-9df1-4698-ae00-0dad30ec97e7" (UID: "128db35e-9df1-4698-ae00-0dad30ec97e7"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:22:21.469237 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.469217    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "128db35e-9df1-4698-ae00-0dad30ec97e7" (UID: "128db35e-9df1-4698-ae00-0dad30ec97e7"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 02:22:21.469304 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.469254    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/128db35e-9df1-4698-ae00-0dad30ec97e7-kube-api-access-vdvgn" (OuterVolumeSpecName: "kube-api-access-vdvgn") pod "128db35e-9df1-4698-ae00-0dad30ec97e7" (UID: "128db35e-9df1-4698-ae00-0dad30ec97e7"). InnerVolumeSpecName "kube-api-access-vdvgn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 02:22:21.469868 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.469848    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "128db35e-9df1-4698-ae00-0dad30ec97e7" (UID: "128db35e-9df1-4698-ae00-0dad30ec97e7"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 02:22:21.470334 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.470314    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-config" (OuterVolumeSpecName: "config") pod "128db35e-9df1-4698-ae00-0dad30ec97e7" (UID: "128db35e-9df1-4698-ae00-0dad30ec97e7"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 02:22:21.470405 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.470328    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "128db35e-9df1-4698-ae00-0dad30ec97e7" (UID: "128db35e-9df1-4698-ae00-0dad30ec97e7"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 02:22:21.470443 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.470405    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/128db35e-9df1-4698-ae00-0dad30ec97e7-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "128db35e-9df1-4698-ae00-0dad30ec97e7" (UID: "128db35e-9df1-4698-ae00-0dad30ec97e7"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 02:22:21.478527 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.478503    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-web-config" (OuterVolumeSpecName: "web-config") pod "128db35e-9df1-4698-ae00-0dad30ec97e7" (UID: "128db35e-9df1-4698-ae00-0dad30ec97e7"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 02:22:21.566723 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.566691    2581 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-secret-grpc-tls\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:22:21.566723 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.566721    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:22:21.566939 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.566732    2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:22:21.566939 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.566768    2581 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-secret-kube-rbac-proxy\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:22:21.566939 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.566777    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-prometheus-trusted-ca-bundle\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:22:21.566939 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.566786    2581 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/128db35e-9df1-4698-ae00-0dad30ec97e7-tls-assets\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:22:21.566939 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.566795    2581 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-secret-metrics-client-certs\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:22:21.566939 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.566803    2581 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-config\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:22:21.566939 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.566811    2581 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:22:21.566939 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.566820    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/128db35e-9df1-4698-ae00-0dad30ec97e7-prometheus-k8s-db\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:22:21.566939 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.566828    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:22:21.566939 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.566837    2581 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-thanos-prometheus-http-client-file\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:22:21.566939 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.566845    2581 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/128db35e-9df1-4698-ae00-0dad30ec97e7-config-out\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:22:21.566939 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.566853    2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:22:21.566939 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.566862    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vdvgn\" (UniqueName: \"kubernetes.io/projected/128db35e-9df1-4698-ae00-0dad30ec97e7-kube-api-access-vdvgn\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:22:21.566939 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.566871    2581 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-web-config\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:22:21.566939 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.566879    2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/128db35e-9df1-4698-ae00-0dad30ec97e7-secret-prometheus-k8s-tls\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:22:21.566939 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.566886    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:22:21.566939 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.566895    2581 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/128db35e-9df1-4698-ae00-0dad30ec97e7-configmap-metrics-client-ca\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:22:21.822577 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.822535    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5ddb9449c-h5zj7" event={"ID":"10f71205-4482-4c08-8fc4-e8a17b554b5b","Type":"ContainerStarted","Data":"30b3a39c63789ee983b98ff0e0117cada4715df23cc0b8367b15d33aa97d2cc7"}
Jul 22 02:22:21.827729 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.827694    2581 generic.go:356] "Generic (PLEG): container finished" podID="128db35e-9df1-4698-ae00-0dad30ec97e7" containerID="711cba6b59e5343647fff2d80c63adea86b403b2e316ead9b9a3b5092664ea84" exitCode=0
Jul 22 02:22:21.827729 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.827723    2581 generic.go:356] "Generic (PLEG): container finished" podID="128db35e-9df1-4698-ae00-0dad30ec97e7" containerID="7f6fc007612fa41b56c072a015370066789b82cba436063cf2dc718735b84746" exitCode=0
Jul 22 02:22:21.827729 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.827732    2581 generic.go:356] "Generic (PLEG): container finished" podID="128db35e-9df1-4698-ae00-0dad30ec97e7" containerID="24a299e0b140c13219a1b56d45d4411a382c45bceb5b982184b6938d13bea805" exitCode=0
Jul 22 02:22:21.827988 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.827765    2581 generic.go:356] "Generic (PLEG): container finished" podID="128db35e-9df1-4698-ae00-0dad30ec97e7" containerID="576c3208c45ef0d0ebb19e53f5816a2b6b2ae3681798c6536ba3cbbb017f8847" exitCode=0
Jul 22 02:22:21.827988 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.827774    2581 generic.go:356] "Generic (PLEG): container finished" podID="128db35e-9df1-4698-ae00-0dad30ec97e7" containerID="a14de2feca99ab793d32ebe371982df560fa6f1dba2f08d39b15c6c48848eb54" exitCode=0
Jul 22 02:22:21.827988 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.827783    2581 generic.go:356] "Generic (PLEG): container finished" podID="128db35e-9df1-4698-ae00-0dad30ec97e7" containerID="ca394967362963f62bd1f19bd6ed7904385b377e940f7b87afb5f5f478aa793f" exitCode=0
Jul 22 02:22:21.827988 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.827822    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:21.827988 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.827775    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"128db35e-9df1-4698-ae00-0dad30ec97e7","Type":"ContainerDied","Data":"711cba6b59e5343647fff2d80c63adea86b403b2e316ead9b9a3b5092664ea84"}
Jul 22 02:22:21.827988 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.827933    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"128db35e-9df1-4698-ae00-0dad30ec97e7","Type":"ContainerDied","Data":"7f6fc007612fa41b56c072a015370066789b82cba436063cf2dc718735b84746"}
Jul 22 02:22:21.827988 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.827958    2581 scope.go:117] "RemoveContainer" containerID="711cba6b59e5343647fff2d80c63adea86b403b2e316ead9b9a3b5092664ea84"
Jul 22 02:22:21.827988 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.827977    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"128db35e-9df1-4698-ae00-0dad30ec97e7","Type":"ContainerDied","Data":"24a299e0b140c13219a1b56d45d4411a382c45bceb5b982184b6938d13bea805"}
Jul 22 02:22:21.828335 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.828015    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"128db35e-9df1-4698-ae00-0dad30ec97e7","Type":"ContainerDied","Data":"576c3208c45ef0d0ebb19e53f5816a2b6b2ae3681798c6536ba3cbbb017f8847"}
Jul 22 02:22:21.828335 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.828054    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"128db35e-9df1-4698-ae00-0dad30ec97e7","Type":"ContainerDied","Data":"a14de2feca99ab793d32ebe371982df560fa6f1dba2f08d39b15c6c48848eb54"}
Jul 22 02:22:21.828335 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.828095    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"128db35e-9df1-4698-ae00-0dad30ec97e7","Type":"ContainerDied","Data":"ca394967362963f62bd1f19bd6ed7904385b377e940f7b87afb5f5f478aa793f"}
Jul 22 02:22:21.828335 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.828134    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"128db35e-9df1-4698-ae00-0dad30ec97e7","Type":"ContainerDied","Data":"d674c235d91ceb116f4793dc18a2a2c67b8034710c609c8742a75e6c12fa50cf"}
Jul 22 02:22:21.837956 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.837844    2581 scope.go:117] "RemoveContainer" containerID="7f6fc007612fa41b56c072a015370066789b82cba436063cf2dc718735b84746"
Jul 22 02:22:21.846967 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.846945    2581 scope.go:117] "RemoveContainer" containerID="24a299e0b140c13219a1b56d45d4411a382c45bceb5b982184b6938d13bea805"
Jul 22 02:22:21.855274 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.855254    2581 scope.go:117] "RemoveContainer" containerID="576c3208c45ef0d0ebb19e53f5816a2b6b2ae3681798c6536ba3cbbb017f8847"
Jul 22 02:22:21.862870 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.862844    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 02:22:21.865363 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.865343    2581 scope.go:117] "RemoveContainer" containerID="a14de2feca99ab793d32ebe371982df560fa6f1dba2f08d39b15c6c48848eb54"
Jul 22 02:22:21.867485 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.867460    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 02:22:21.874479 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.874462    2581 scope.go:117] "RemoveContainer" containerID="ca394967362963f62bd1f19bd6ed7904385b377e940f7b87afb5f5f478aa793f"
Jul 22 02:22:21.883201 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.883172    2581 scope.go:117] "RemoveContainer" containerID="e585971196599582b2a7ca6030a7021c3dc0505ece69dac78715616189a7b272"
Jul 22 02:22:21.891761 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.891716    2581 scope.go:117] "RemoveContainer" containerID="711cba6b59e5343647fff2d80c63adea86b403b2e316ead9b9a3b5092664ea84"
Jul 22 02:22:21.892068 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:22:21.892049    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"711cba6b59e5343647fff2d80c63adea86b403b2e316ead9b9a3b5092664ea84\": container with ID starting with 711cba6b59e5343647fff2d80c63adea86b403b2e316ead9b9a3b5092664ea84 not found: ID does not exist" containerID="711cba6b59e5343647fff2d80c63adea86b403b2e316ead9b9a3b5092664ea84"
Jul 22 02:22:21.892142 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.892077    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"711cba6b59e5343647fff2d80c63adea86b403b2e316ead9b9a3b5092664ea84"} err="failed to get container status \"711cba6b59e5343647fff2d80c63adea86b403b2e316ead9b9a3b5092664ea84\": rpc error: code = NotFound desc = could not find container \"711cba6b59e5343647fff2d80c63adea86b403b2e316ead9b9a3b5092664ea84\": container with ID starting with 711cba6b59e5343647fff2d80c63adea86b403b2e316ead9b9a3b5092664ea84 not found: ID does not exist"
Jul 22 02:22:21.892142 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.892096    2581 scope.go:117] "RemoveContainer" containerID="7f6fc007612fa41b56c072a015370066789b82cba436063cf2dc718735b84746"
Jul 22 02:22:21.892362 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:22:21.892341    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f6fc007612fa41b56c072a015370066789b82cba436063cf2dc718735b84746\": container with ID starting with 7f6fc007612fa41b56c072a015370066789b82cba436063cf2dc718735b84746 not found: ID does not exist" containerID="7f6fc007612fa41b56c072a015370066789b82cba436063cf2dc718735b84746"
Jul 22 02:22:21.892405 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.892368    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f6fc007612fa41b56c072a015370066789b82cba436063cf2dc718735b84746"} err="failed to get container status \"7f6fc007612fa41b56c072a015370066789b82cba436063cf2dc718735b84746\": rpc error: code = NotFound desc = could not find container \"7f6fc007612fa41b56c072a015370066789b82cba436063cf2dc718735b84746\": container with ID starting with 7f6fc007612fa41b56c072a015370066789b82cba436063cf2dc718735b84746 not found: ID does not exist"
Jul 22 02:22:21.892405 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.892387    2581 scope.go:117] "RemoveContainer" containerID="24a299e0b140c13219a1b56d45d4411a382c45bceb5b982184b6938d13bea805"
Jul 22 02:22:21.892590 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:22:21.892570    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24a299e0b140c13219a1b56d45d4411a382c45bceb5b982184b6938d13bea805\": container with ID starting with 24a299e0b140c13219a1b56d45d4411a382c45bceb5b982184b6938d13bea805 not found: ID does not exist" containerID="24a299e0b140c13219a1b56d45d4411a382c45bceb5b982184b6938d13bea805"
Jul 22 02:22:21.892645 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.892600    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24a299e0b140c13219a1b56d45d4411a382c45bceb5b982184b6938d13bea805"} err="failed to get container status \"24a299e0b140c13219a1b56d45d4411a382c45bceb5b982184b6938d13bea805\": rpc error: code = NotFound desc = could not find container \"24a299e0b140c13219a1b56d45d4411a382c45bceb5b982184b6938d13bea805\": container with ID starting with 24a299e0b140c13219a1b56d45d4411a382c45bceb5b982184b6938d13bea805 not found: ID does not exist"
Jul 22 02:22:21.892645 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.892621    2581 scope.go:117] "RemoveContainer" containerID="576c3208c45ef0d0ebb19e53f5816a2b6b2ae3681798c6536ba3cbbb017f8847"
Jul 22 02:22:21.892902 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:22:21.892872    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"576c3208c45ef0d0ebb19e53f5816a2b6b2ae3681798c6536ba3cbbb017f8847\": container with ID starting with 576c3208c45ef0d0ebb19e53f5816a2b6b2ae3681798c6536ba3cbbb017f8847 not found: ID does not exist" containerID="576c3208c45ef0d0ebb19e53f5816a2b6b2ae3681798c6536ba3cbbb017f8847"
Jul 22 02:22:21.893036 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.892900    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"576c3208c45ef0d0ebb19e53f5816a2b6b2ae3681798c6536ba3cbbb017f8847"} err="failed to get container status \"576c3208c45ef0d0ebb19e53f5816a2b6b2ae3681798c6536ba3cbbb017f8847\": rpc error: code = NotFound desc = could not find container \"576c3208c45ef0d0ebb19e53f5816a2b6b2ae3681798c6536ba3cbbb017f8847\": container with ID starting with 576c3208c45ef0d0ebb19e53f5816a2b6b2ae3681798c6536ba3cbbb017f8847 not found: ID does not exist"
Jul 22 02:22:21.893036 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.892921    2581 scope.go:117] "RemoveContainer" containerID="a14de2feca99ab793d32ebe371982df560fa6f1dba2f08d39b15c6c48848eb54"
Jul 22 02:22:21.893173 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:22:21.893153    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a14de2feca99ab793d32ebe371982df560fa6f1dba2f08d39b15c6c48848eb54\": container with ID starting with a14de2feca99ab793d32ebe371982df560fa6f1dba2f08d39b15c6c48848eb54 not found: ID does not exist" containerID="a14de2feca99ab793d32ebe371982df560fa6f1dba2f08d39b15c6c48848eb54"
Jul 22 02:22:21.893229 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.893177    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a14de2feca99ab793d32ebe371982df560fa6f1dba2f08d39b15c6c48848eb54"} err="failed to get container status \"a14de2feca99ab793d32ebe371982df560fa6f1dba2f08d39b15c6c48848eb54\": rpc error: code = NotFound desc = could not find container \"a14de2feca99ab793d32ebe371982df560fa6f1dba2f08d39b15c6c48848eb54\": container with ID starting with a14de2feca99ab793d32ebe371982df560fa6f1dba2f08d39b15c6c48848eb54 not found: ID does not exist"
Jul 22 02:22:21.893229 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.893192    2581 scope.go:117] "RemoveContainer" containerID="ca394967362963f62bd1f19bd6ed7904385b377e940f7b87afb5f5f478aa793f"
Jul 22 02:22:21.893414 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:22:21.893394    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca394967362963f62bd1f19bd6ed7904385b377e940f7b87afb5f5f478aa793f\": container with ID starting with ca394967362963f62bd1f19bd6ed7904385b377e940f7b87afb5f5f478aa793f not found: ID does not exist" containerID="ca394967362963f62bd1f19bd6ed7904385b377e940f7b87afb5f5f478aa793f"
Jul 22 02:22:21.893458 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.893419    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca394967362963f62bd1f19bd6ed7904385b377e940f7b87afb5f5f478aa793f"} err="failed to get container status \"ca394967362963f62bd1f19bd6ed7904385b377e940f7b87afb5f5f478aa793f\": rpc error: code = NotFound desc = could not find container \"ca394967362963f62bd1f19bd6ed7904385b377e940f7b87afb5f5f478aa793f\": container with ID starting with ca394967362963f62bd1f19bd6ed7904385b377e940f7b87afb5f5f478aa793f not found: ID does not exist"
Jul 22 02:22:21.893458 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.893434    2581 scope.go:117] "RemoveContainer" containerID="e585971196599582b2a7ca6030a7021c3dc0505ece69dac78715616189a7b272"
Jul 22 02:22:21.893761 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:22:21.893713    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e585971196599582b2a7ca6030a7021c3dc0505ece69dac78715616189a7b272\": container with ID starting with e585971196599582b2a7ca6030a7021c3dc0505ece69dac78715616189a7b272 not found: ID does not exist" containerID="e585971196599582b2a7ca6030a7021c3dc0505ece69dac78715616189a7b272"
Jul 22 02:22:21.893842 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.893766    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e585971196599582b2a7ca6030a7021c3dc0505ece69dac78715616189a7b272"} err="failed to get container status \"e585971196599582b2a7ca6030a7021c3dc0505ece69dac78715616189a7b272\": rpc error: code = NotFound desc = could not find container \"e585971196599582b2a7ca6030a7021c3dc0505ece69dac78715616189a7b272\": container with ID starting with e585971196599582b2a7ca6030a7021c3dc0505ece69dac78715616189a7b272 not found: ID does not exist"
Jul 22 02:22:21.893842 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.893788    2581 scope.go:117] "RemoveContainer" containerID="711cba6b59e5343647fff2d80c63adea86b403b2e316ead9b9a3b5092664ea84"
Jul 22 02:22:21.894046 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.894027    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"711cba6b59e5343647fff2d80c63adea86b403b2e316ead9b9a3b5092664ea84"} err="failed to get container status \"711cba6b59e5343647fff2d80c63adea86b403b2e316ead9b9a3b5092664ea84\": rpc error: code = NotFound desc = could not find container \"711cba6b59e5343647fff2d80c63adea86b403b2e316ead9b9a3b5092664ea84\": container with ID starting with 711cba6b59e5343647fff2d80c63adea86b403b2e316ead9b9a3b5092664ea84 not found: ID does not exist"
Jul 22 02:22:21.894098 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.894047    2581 scope.go:117] "RemoveContainer" containerID="7f6fc007612fa41b56c072a015370066789b82cba436063cf2dc718735b84746"
Jul 22 02:22:21.894331 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.894302    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f6fc007612fa41b56c072a015370066789b82cba436063cf2dc718735b84746"} err="failed to get container status \"7f6fc007612fa41b56c072a015370066789b82cba436063cf2dc718735b84746\": rpc error: code = NotFound desc = could not find container \"7f6fc007612fa41b56c072a015370066789b82cba436063cf2dc718735b84746\": container with ID starting with 7f6fc007612fa41b56c072a015370066789b82cba436063cf2dc718735b84746 not found: ID does not exist"
Jul 22 02:22:21.894331 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.894330    2581 scope.go:117] "RemoveContainer" containerID="24a299e0b140c13219a1b56d45d4411a382c45bceb5b982184b6938d13bea805"
Jul 22 02:22:21.894649 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.894623    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24a299e0b140c13219a1b56d45d4411a382c45bceb5b982184b6938d13bea805"} err="failed to get container status \"24a299e0b140c13219a1b56d45d4411a382c45bceb5b982184b6938d13bea805\": rpc error: code = NotFound desc = could not find container \"24a299e0b140c13219a1b56d45d4411a382c45bceb5b982184b6938d13bea805\": container with ID starting with 24a299e0b140c13219a1b56d45d4411a382c45bceb5b982184b6938d13bea805 not found: ID does not exist"
Jul 22 02:22:21.894805 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.894651    2581 scope.go:117] "RemoveContainer" containerID="576c3208c45ef0d0ebb19e53f5816a2b6b2ae3681798c6536ba3cbbb017f8847"
Jul 22 02:22:21.894966 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.894941    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"576c3208c45ef0d0ebb19e53f5816a2b6b2ae3681798c6536ba3cbbb017f8847"} err="failed to get container status \"576c3208c45ef0d0ebb19e53f5816a2b6b2ae3681798c6536ba3cbbb017f8847\": rpc error: code = NotFound desc = could not find container \"576c3208c45ef0d0ebb19e53f5816a2b6b2ae3681798c6536ba3cbbb017f8847\": container with ID starting with 576c3208c45ef0d0ebb19e53f5816a2b6b2ae3681798c6536ba3cbbb017f8847 not found: ID does not exist"
Jul 22 02:22:21.894966 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.894963    2581 scope.go:117] "RemoveContainer" containerID="a14de2feca99ab793d32ebe371982df560fa6f1dba2f08d39b15c6c48848eb54"
Jul 22 02:22:21.895371 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.895340    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a14de2feca99ab793d32ebe371982df560fa6f1dba2f08d39b15c6c48848eb54"} err="failed to get container status \"a14de2feca99ab793d32ebe371982df560fa6f1dba2f08d39b15c6c48848eb54\": rpc error: code = NotFound desc = could not find container \"a14de2feca99ab793d32ebe371982df560fa6f1dba2f08d39b15c6c48848eb54\": container with ID starting with a14de2feca99ab793d32ebe371982df560fa6f1dba2f08d39b15c6c48848eb54 not found: ID does not exist"
Jul 22 02:22:21.895371 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.895361    2581 scope.go:117] "RemoveContainer" containerID="ca394967362963f62bd1f19bd6ed7904385b377e940f7b87afb5f5f478aa793f"
Jul 22 02:22:21.895593 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.895576    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca394967362963f62bd1f19bd6ed7904385b377e940f7b87afb5f5f478aa793f"} err="failed to get container status \"ca394967362963f62bd1f19bd6ed7904385b377e940f7b87afb5f5f478aa793f\": rpc error: code = NotFound desc = could not find container \"ca394967362963f62bd1f19bd6ed7904385b377e940f7b87afb5f5f478aa793f\": container with ID starting with ca394967362963f62bd1f19bd6ed7904385b377e940f7b87afb5f5f478aa793f not found: ID does not exist"
Jul 22 02:22:21.895643 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.895594    2581 scope.go:117] "RemoveContainer" containerID="e585971196599582b2a7ca6030a7021c3dc0505ece69dac78715616189a7b272"
Jul 22 02:22:21.895866 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.895839    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e585971196599582b2a7ca6030a7021c3dc0505ece69dac78715616189a7b272"} err="failed to get container status \"e585971196599582b2a7ca6030a7021c3dc0505ece69dac78715616189a7b272\": rpc error: code = NotFound desc = could not find container \"e585971196599582b2a7ca6030a7021c3dc0505ece69dac78715616189a7b272\": container with ID starting with e585971196599582b2a7ca6030a7021c3dc0505ece69dac78715616189a7b272 not found: ID does not exist"
Jul 22 02:22:21.896043 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.895866    2581 scope.go:117] "RemoveContainer" containerID="711cba6b59e5343647fff2d80c63adea86b403b2e316ead9b9a3b5092664ea84"
Jul 22 02:22:21.896115 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.896096    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"711cba6b59e5343647fff2d80c63adea86b403b2e316ead9b9a3b5092664ea84"} err="failed to get container status \"711cba6b59e5343647fff2d80c63adea86b403b2e316ead9b9a3b5092664ea84\": rpc error: code = NotFound desc = could not find container \"711cba6b59e5343647fff2d80c63adea86b403b2e316ead9b9a3b5092664ea84\": container with ID starting with 711cba6b59e5343647fff2d80c63adea86b403b2e316ead9b9a3b5092664ea84 not found: ID does not exist"
Jul 22 02:22:21.896153 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.896116    2581 scope.go:117] "RemoveContainer" containerID="7f6fc007612fa41b56c072a015370066789b82cba436063cf2dc718735b84746"
Jul 22 02:22:21.896326 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.896308    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f6fc007612fa41b56c072a015370066789b82cba436063cf2dc718735b84746"} err="failed to get container status \"7f6fc007612fa41b56c072a015370066789b82cba436063cf2dc718735b84746\": rpc error: code = NotFound desc = could not find container \"7f6fc007612fa41b56c072a015370066789b82cba436063cf2dc718735b84746\": container with ID starting with 7f6fc007612fa41b56c072a015370066789b82cba436063cf2dc718735b84746 not found: ID does not exist"
Jul 22 02:22:21.896367 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.896327    2581 scope.go:117] "RemoveContainer" containerID="24a299e0b140c13219a1b56d45d4411a382c45bceb5b982184b6938d13bea805"
Jul 22 02:22:21.896519 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.896502    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24a299e0b140c13219a1b56d45d4411a382c45bceb5b982184b6938d13bea805"} err="failed to get container status \"24a299e0b140c13219a1b56d45d4411a382c45bceb5b982184b6938d13bea805\": rpc error: code = NotFound desc = could not find container \"24a299e0b140c13219a1b56d45d4411a382c45bceb5b982184b6938d13bea805\": container with ID starting with 24a299e0b140c13219a1b56d45d4411a382c45bceb5b982184b6938d13bea805 not found: ID does not exist"
Jul 22 02:22:21.896558 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.896519    2581 scope.go:117] "RemoveContainer" containerID="576c3208c45ef0d0ebb19e53f5816a2b6b2ae3681798c6536ba3cbbb017f8847"
Jul 22 02:22:21.896732 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.896711    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"576c3208c45ef0d0ebb19e53f5816a2b6b2ae3681798c6536ba3cbbb017f8847"} err="failed to get container status \"576c3208c45ef0d0ebb19e53f5816a2b6b2ae3681798c6536ba3cbbb017f8847\": rpc error: code = NotFound desc = could not find container \"576c3208c45ef0d0ebb19e53f5816a2b6b2ae3681798c6536ba3cbbb017f8847\": container with ID starting with 576c3208c45ef0d0ebb19e53f5816a2b6b2ae3681798c6536ba3cbbb017f8847 not found: ID does not exist"
Jul 22 02:22:21.896732 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.896747    2581 scope.go:117] "RemoveContainer" containerID="a14de2feca99ab793d32ebe371982df560fa6f1dba2f08d39b15c6c48848eb54"
Jul 22 02:22:21.896938 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.896923    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a14de2feca99ab793d32ebe371982df560fa6f1dba2f08d39b15c6c48848eb54"} err="failed to get container status \"a14de2feca99ab793d32ebe371982df560fa6f1dba2f08d39b15c6c48848eb54\": rpc error: code = NotFound desc = could not find container \"a14de2feca99ab793d32ebe371982df560fa6f1dba2f08d39b15c6c48848eb54\": container with ID starting with a14de2feca99ab793d32ebe371982df560fa6f1dba2f08d39b15c6c48848eb54 not found: ID does not exist"
Jul 22 02:22:21.896973 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.896939    2581 scope.go:117] "RemoveContainer" containerID="ca394967362963f62bd1f19bd6ed7904385b377e940f7b87afb5f5f478aa793f"
Jul 22 02:22:21.897121 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.897107    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca394967362963f62bd1f19bd6ed7904385b377e940f7b87afb5f5f478aa793f"} err="failed to get container status \"ca394967362963f62bd1f19bd6ed7904385b377e940f7b87afb5f5f478aa793f\": rpc error: code = NotFound desc = could not find container \"ca394967362963f62bd1f19bd6ed7904385b377e940f7b87afb5f5f478aa793f\": container with ID starting with ca394967362963f62bd1f19bd6ed7904385b377e940f7b87afb5f5f478aa793f not found: ID does not exist"
Jul 22 02:22:21.897169 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.897122    2581 scope.go:117] "RemoveContainer" containerID="e585971196599582b2a7ca6030a7021c3dc0505ece69dac78715616189a7b272"
Jul 22 02:22:21.897319 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.897304    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e585971196599582b2a7ca6030a7021c3dc0505ece69dac78715616189a7b272"} err="failed to get container status \"e585971196599582b2a7ca6030a7021c3dc0505ece69dac78715616189a7b272\": rpc error: code = NotFound desc = could not find container \"e585971196599582b2a7ca6030a7021c3dc0505ece69dac78715616189a7b272\": container with ID starting with e585971196599582b2a7ca6030a7021c3dc0505ece69dac78715616189a7b272 not found: ID does not exist"
Jul 22 02:22:21.897356 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.897320    2581 scope.go:117] "RemoveContainer" containerID="711cba6b59e5343647fff2d80c63adea86b403b2e316ead9b9a3b5092664ea84"
Jul 22 02:22:21.897500 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.897485    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"711cba6b59e5343647fff2d80c63adea86b403b2e316ead9b9a3b5092664ea84"} err="failed to get container status \"711cba6b59e5343647fff2d80c63adea86b403b2e316ead9b9a3b5092664ea84\": rpc error: code = NotFound desc = could not find container \"711cba6b59e5343647fff2d80c63adea86b403b2e316ead9b9a3b5092664ea84\": container with ID starting with 711cba6b59e5343647fff2d80c63adea86b403b2e316ead9b9a3b5092664ea84 not found: ID does not exist"
Jul 22 02:22:21.897538 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.897501    2581 scope.go:117] "RemoveContainer" containerID="7f6fc007612fa41b56c072a015370066789b82cba436063cf2dc718735b84746"
Jul 22 02:22:21.897726 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.897708    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f6fc007612fa41b56c072a015370066789b82cba436063cf2dc718735b84746"} err="failed to get container status \"7f6fc007612fa41b56c072a015370066789b82cba436063cf2dc718735b84746\": rpc error: code = NotFound desc = could not find container \"7f6fc007612fa41b56c072a015370066789b82cba436063cf2dc718735b84746\": container with ID starting with 7f6fc007612fa41b56c072a015370066789b82cba436063cf2dc718735b84746 not found: ID does not exist"
Jul 22 02:22:21.897784 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.897749    2581 scope.go:117] "RemoveContainer" containerID="24a299e0b140c13219a1b56d45d4411a382c45bceb5b982184b6938d13bea805"
Jul 22 02:22:21.897952 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.897933    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24a299e0b140c13219a1b56d45d4411a382c45bceb5b982184b6938d13bea805"} err="failed to get container status \"24a299e0b140c13219a1b56d45d4411a382c45bceb5b982184b6938d13bea805\": rpc error: code = NotFound desc = could not find container \"24a299e0b140c13219a1b56d45d4411a382c45bceb5b982184b6938d13bea805\": container with ID starting with 24a299e0b140c13219a1b56d45d4411a382c45bceb5b982184b6938d13bea805 not found: ID does not exist"
Jul 22 02:22:21.898064 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.897954    2581 scope.go:117] "RemoveContainer" containerID="576c3208c45ef0d0ebb19e53f5816a2b6b2ae3681798c6536ba3cbbb017f8847"
Jul 22 02:22:21.898173 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.898157    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"576c3208c45ef0d0ebb19e53f5816a2b6b2ae3681798c6536ba3cbbb017f8847"} err="failed to get container status \"576c3208c45ef0d0ebb19e53f5816a2b6b2ae3681798c6536ba3cbbb017f8847\": rpc error: code = NotFound desc = could not find container \"576c3208c45ef0d0ebb19e53f5816a2b6b2ae3681798c6536ba3cbbb017f8847\": container with ID starting with 576c3208c45ef0d0ebb19e53f5816a2b6b2ae3681798c6536ba3cbbb017f8847 not found: ID does not exist"
Jul 22 02:22:21.898218 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.898173    2581 scope.go:117] "RemoveContainer" containerID="a14de2feca99ab793d32ebe371982df560fa6f1dba2f08d39b15c6c48848eb54"
Jul 22 02:22:21.898375 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.898353    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a14de2feca99ab793d32ebe371982df560fa6f1dba2f08d39b15c6c48848eb54"} err="failed to get container status \"a14de2feca99ab793d32ebe371982df560fa6f1dba2f08d39b15c6c48848eb54\": rpc error: code = NotFound desc = could not find container \"a14de2feca99ab793d32ebe371982df560fa6f1dba2f08d39b15c6c48848eb54\": container with ID starting with a14de2feca99ab793d32ebe371982df560fa6f1dba2f08d39b15c6c48848eb54 not found: ID does not exist"
Jul 22 02:22:21.898413 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.898377    2581 scope.go:117] "RemoveContainer" containerID="ca394967362963f62bd1f19bd6ed7904385b377e940f7b87afb5f5f478aa793f"
Jul 22 02:22:21.898606 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.898588    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca394967362963f62bd1f19bd6ed7904385b377e940f7b87afb5f5f478aa793f"} err="failed to get container status \"ca394967362963f62bd1f19bd6ed7904385b377e940f7b87afb5f5f478aa793f\": rpc error: code = NotFound desc = could not find container \"ca394967362963f62bd1f19bd6ed7904385b377e940f7b87afb5f5f478aa793f\": container with ID starting with ca394967362963f62bd1f19bd6ed7904385b377e940f7b87afb5f5f478aa793f not found: ID does not exist"
Jul 22 02:22:21.898660 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.898606    2581 scope.go:117] "RemoveContainer" containerID="e585971196599582b2a7ca6030a7021c3dc0505ece69dac78715616189a7b272"
Jul 22 02:22:21.898817 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.898801    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e585971196599582b2a7ca6030a7021c3dc0505ece69dac78715616189a7b272"} err="failed to get container status \"e585971196599582b2a7ca6030a7021c3dc0505ece69dac78715616189a7b272\": rpc error: code = NotFound desc = could not find container \"e585971196599582b2a7ca6030a7021c3dc0505ece69dac78715616189a7b272\": container with ID starting with e585971196599582b2a7ca6030a7021c3dc0505ece69dac78715616189a7b272 not found: ID does not exist"
Jul 22 02:22:21.898865 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.898817    2581 scope.go:117] "RemoveContainer" containerID="711cba6b59e5343647fff2d80c63adea86b403b2e316ead9b9a3b5092664ea84"
Jul 22 02:22:21.899018 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.898999    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"711cba6b59e5343647fff2d80c63adea86b403b2e316ead9b9a3b5092664ea84"} err="failed to get container status \"711cba6b59e5343647fff2d80c63adea86b403b2e316ead9b9a3b5092664ea84\": rpc error: code = NotFound desc = could not find container \"711cba6b59e5343647fff2d80c63adea86b403b2e316ead9b9a3b5092664ea84\": container with ID starting with 711cba6b59e5343647fff2d80c63adea86b403b2e316ead9b9a3b5092664ea84 not found: ID does not exist"
Jul 22 02:22:21.899062 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.899018    2581 scope.go:117] "RemoveContainer" containerID="7f6fc007612fa41b56c072a015370066789b82cba436063cf2dc718735b84746"
Jul 22 02:22:21.899230 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.899211    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f6fc007612fa41b56c072a015370066789b82cba436063cf2dc718735b84746"} err="failed to get container status \"7f6fc007612fa41b56c072a015370066789b82cba436063cf2dc718735b84746\": rpc error: code = NotFound desc = could not find container \"7f6fc007612fa41b56c072a015370066789b82cba436063cf2dc718735b84746\": container with ID starting with 7f6fc007612fa41b56c072a015370066789b82cba436063cf2dc718735b84746 not found: ID does not exist"
Jul 22 02:22:21.899302 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.899232    2581 scope.go:117] "RemoveContainer" containerID="24a299e0b140c13219a1b56d45d4411a382c45bceb5b982184b6938d13bea805"
Jul 22 02:22:21.899431 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.899416    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24a299e0b140c13219a1b56d45d4411a382c45bceb5b982184b6938d13bea805"} err="failed to get container status \"24a299e0b140c13219a1b56d45d4411a382c45bceb5b982184b6938d13bea805\": rpc error: code = NotFound desc = could not find container \"24a299e0b140c13219a1b56d45d4411a382c45bceb5b982184b6938d13bea805\": container with ID starting with 24a299e0b140c13219a1b56d45d4411a382c45bceb5b982184b6938d13bea805 not found: ID does not exist"
Jul 22 02:22:21.899476 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.899431    2581 scope.go:117] "RemoveContainer" containerID="576c3208c45ef0d0ebb19e53f5816a2b6b2ae3681798c6536ba3cbbb017f8847"
Jul 22 02:22:21.899617 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.899599    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"576c3208c45ef0d0ebb19e53f5816a2b6b2ae3681798c6536ba3cbbb017f8847"} err="failed to get container status \"576c3208c45ef0d0ebb19e53f5816a2b6b2ae3681798c6536ba3cbbb017f8847\": rpc error: code = NotFound desc = could not find container \"576c3208c45ef0d0ebb19e53f5816a2b6b2ae3681798c6536ba3cbbb017f8847\": container with ID starting with 576c3208c45ef0d0ebb19e53f5816a2b6b2ae3681798c6536ba3cbbb017f8847 not found: ID does not exist"
Jul 22 02:22:21.899676 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.899618    2581 scope.go:117] "RemoveContainer" containerID="a14de2feca99ab793d32ebe371982df560fa6f1dba2f08d39b15c6c48848eb54"
Jul 22 02:22:21.899836 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.899818    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a14de2feca99ab793d32ebe371982df560fa6f1dba2f08d39b15c6c48848eb54"} err="failed to get container status \"a14de2feca99ab793d32ebe371982df560fa6f1dba2f08d39b15c6c48848eb54\": rpc error: code = NotFound desc = could not find container \"a14de2feca99ab793d32ebe371982df560fa6f1dba2f08d39b15c6c48848eb54\": container with ID starting with a14de2feca99ab793d32ebe371982df560fa6f1dba2f08d39b15c6c48848eb54 not found: ID does not exist"
Jul 22 02:22:21.899879 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.899838    2581 scope.go:117] "RemoveContainer" containerID="ca394967362963f62bd1f19bd6ed7904385b377e940f7b87afb5f5f478aa793f"
Jul 22 02:22:21.900039 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.900022    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca394967362963f62bd1f19bd6ed7904385b377e940f7b87afb5f5f478aa793f"} err="failed to get container status \"ca394967362963f62bd1f19bd6ed7904385b377e940f7b87afb5f5f478aa793f\": rpc error: code = NotFound desc = could not find container \"ca394967362963f62bd1f19bd6ed7904385b377e940f7b87afb5f5f478aa793f\": container with ID starting with ca394967362963f62bd1f19bd6ed7904385b377e940f7b87afb5f5f478aa793f not found: ID does not exist"
Jul 22 02:22:21.900098 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.900041    2581 scope.go:117] "RemoveContainer" containerID="e585971196599582b2a7ca6030a7021c3dc0505ece69dac78715616189a7b272"
Jul 22 02:22:21.900300 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.900283    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e585971196599582b2a7ca6030a7021c3dc0505ece69dac78715616189a7b272"} err="failed to get container status \"e585971196599582b2a7ca6030a7021c3dc0505ece69dac78715616189a7b272\": rpc error: code = NotFound desc = could not find container \"e585971196599582b2a7ca6030a7021c3dc0505ece69dac78715616189a7b272\": container with ID starting with e585971196599582b2a7ca6030a7021c3dc0505ece69dac78715616189a7b272 not found: ID does not exist"
Jul 22 02:22:21.900350 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.900300    2581 scope.go:117] "RemoveContainer" containerID="711cba6b59e5343647fff2d80c63adea86b403b2e316ead9b9a3b5092664ea84"
Jul 22 02:22:21.900529 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.900511    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"711cba6b59e5343647fff2d80c63adea86b403b2e316ead9b9a3b5092664ea84"} err="failed to get container status \"711cba6b59e5343647fff2d80c63adea86b403b2e316ead9b9a3b5092664ea84\": rpc error: code = NotFound desc = could not find container \"711cba6b59e5343647fff2d80c63adea86b403b2e316ead9b9a3b5092664ea84\": container with ID starting with 711cba6b59e5343647fff2d80c63adea86b403b2e316ead9b9a3b5092664ea84 not found: ID does not exist"
Jul 22 02:22:21.900529 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.900528    2581 scope.go:117] "RemoveContainer" containerID="7f6fc007612fa41b56c072a015370066789b82cba436063cf2dc718735b84746"
Jul 22 02:22:21.900719 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.900700    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f6fc007612fa41b56c072a015370066789b82cba436063cf2dc718735b84746"} err="failed to get container status \"7f6fc007612fa41b56c072a015370066789b82cba436063cf2dc718735b84746\": rpc error: code = NotFound desc = could not find container \"7f6fc007612fa41b56c072a015370066789b82cba436063cf2dc718735b84746\": container with ID starting with 7f6fc007612fa41b56c072a015370066789b82cba436063cf2dc718735b84746 not found: ID does not exist"
Jul 22 02:22:21.900787 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.900720    2581 scope.go:117] "RemoveContainer" containerID="24a299e0b140c13219a1b56d45d4411a382c45bceb5b982184b6938d13bea805"
Jul 22 02:22:21.900941 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.900926    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24a299e0b140c13219a1b56d45d4411a382c45bceb5b982184b6938d13bea805"} err="failed to get container status \"24a299e0b140c13219a1b56d45d4411a382c45bceb5b982184b6938d13bea805\": rpc error: code = NotFound desc = could not find container \"24a299e0b140c13219a1b56d45d4411a382c45bceb5b982184b6938d13bea805\": container with ID starting with 24a299e0b140c13219a1b56d45d4411a382c45bceb5b982184b6938d13bea805 not found: ID does not exist"
Jul 22 02:22:21.900984 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.900940    2581 scope.go:117] "RemoveContainer" containerID="576c3208c45ef0d0ebb19e53f5816a2b6b2ae3681798c6536ba3cbbb017f8847"
Jul 22 02:22:21.901175 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.901157    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"576c3208c45ef0d0ebb19e53f5816a2b6b2ae3681798c6536ba3cbbb017f8847"} err="failed to get container status \"576c3208c45ef0d0ebb19e53f5816a2b6b2ae3681798c6536ba3cbbb017f8847\": rpc error: code = NotFound desc = could not find container \"576c3208c45ef0d0ebb19e53f5816a2b6b2ae3681798c6536ba3cbbb017f8847\": container with ID starting with 576c3208c45ef0d0ebb19e53f5816a2b6b2ae3681798c6536ba3cbbb017f8847 not found: ID does not exist"
Jul 22 02:22:21.901211 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.901176    2581 scope.go:117] "RemoveContainer" containerID="a14de2feca99ab793d32ebe371982df560fa6f1dba2f08d39b15c6c48848eb54"
Jul 22 02:22:21.901383 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.901368    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a14de2feca99ab793d32ebe371982df560fa6f1dba2f08d39b15c6c48848eb54"} err="failed to get container status \"a14de2feca99ab793d32ebe371982df560fa6f1dba2f08d39b15c6c48848eb54\": rpc error: code = NotFound desc = could not find container \"a14de2feca99ab793d32ebe371982df560fa6f1dba2f08d39b15c6c48848eb54\": container with ID starting with a14de2feca99ab793d32ebe371982df560fa6f1dba2f08d39b15c6c48848eb54 not found: ID does not exist"
Jul 22 02:22:21.901383 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.901383    2581 scope.go:117] "RemoveContainer" containerID="ca394967362963f62bd1f19bd6ed7904385b377e940f7b87afb5f5f478aa793f"
Jul 22 02:22:21.901585 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.901569    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca394967362963f62bd1f19bd6ed7904385b377e940f7b87afb5f5f478aa793f"} err="failed to get container status \"ca394967362963f62bd1f19bd6ed7904385b377e940f7b87afb5f5f478aa793f\": rpc error: code = NotFound desc = could not find container \"ca394967362963f62bd1f19bd6ed7904385b377e940f7b87afb5f5f478aa793f\": container with ID starting with ca394967362963f62bd1f19bd6ed7904385b377e940f7b87afb5f5f478aa793f not found: ID does not exist"
Jul 22 02:22:21.901585 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.901584    2581 scope.go:117] "RemoveContainer" containerID="e585971196599582b2a7ca6030a7021c3dc0505ece69dac78715616189a7b272"
Jul 22 02:22:21.901832 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.901802    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e585971196599582b2a7ca6030a7021c3dc0505ece69dac78715616189a7b272"} err="failed to get container status \"e585971196599582b2a7ca6030a7021c3dc0505ece69dac78715616189a7b272\": rpc error: code = NotFound desc = could not find container \"e585971196599582b2a7ca6030a7021c3dc0505ece69dac78715616189a7b272\": container with ID starting with e585971196599582b2a7ca6030a7021c3dc0505ece69dac78715616189a7b272 not found: ID does not exist"
Jul 22 02:22:21.908458 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.908426    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 02:22:21.909167 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.909148    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="128db35e-9df1-4698-ae00-0dad30ec97e7" containerName="prometheus"
Jul 22 02:22:21.909210 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.909171    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="128db35e-9df1-4698-ae00-0dad30ec97e7" containerName="prometheus"
Jul 22 02:22:21.909210 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.909189    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="128db35e-9df1-4698-ae00-0dad30ec97e7" containerName="kube-rbac-proxy"
Jul 22 02:22:21.909210 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.909199    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="128db35e-9df1-4698-ae00-0dad30ec97e7" containerName="kube-rbac-proxy"
Jul 22 02:22:21.909311 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.909213    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="128db35e-9df1-4698-ae00-0dad30ec97e7" containerName="kube-rbac-proxy-thanos"
Jul 22 02:22:21.909311 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.909223    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="128db35e-9df1-4698-ae00-0dad30ec97e7" containerName="kube-rbac-proxy-thanos"
Jul 22 02:22:21.909311 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.909244    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="128db35e-9df1-4698-ae00-0dad30ec97e7" containerName="config-reloader"
Jul 22 02:22:21.909311 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.909253    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="128db35e-9df1-4698-ae00-0dad30ec97e7" containerName="config-reloader"
Jul 22 02:22:21.909311 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.909301    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="128db35e-9df1-4698-ae00-0dad30ec97e7" containerName="init-config-reloader"
Jul 22 02:22:21.909311 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.909309    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="128db35e-9df1-4698-ae00-0dad30ec97e7" containerName="init-config-reloader"
Jul 22 02:22:21.909485 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.909326    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="128db35e-9df1-4698-ae00-0dad30ec97e7" containerName="thanos-sidecar"
Jul 22 02:22:21.909485 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.909334    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="128db35e-9df1-4698-ae00-0dad30ec97e7" containerName="thanos-sidecar"
Jul 22 02:22:21.909485 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.909345    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="128db35e-9df1-4698-ae00-0dad30ec97e7" containerName="kube-rbac-proxy-web"
Jul 22 02:22:21.909485 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.909354    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="128db35e-9df1-4698-ae00-0dad30ec97e7" containerName="kube-rbac-proxy-web"
Jul 22 02:22:21.909597 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.909500    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="128db35e-9df1-4698-ae00-0dad30ec97e7" containerName="kube-rbac-proxy-thanos"
Jul 22 02:22:21.909597 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.909514    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="128db35e-9df1-4698-ae00-0dad30ec97e7" containerName="prometheus"
Jul 22 02:22:21.909597 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.909530    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="128db35e-9df1-4698-ae00-0dad30ec97e7" containerName="kube-rbac-proxy"
Jul 22 02:22:21.909597 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.909540    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="128db35e-9df1-4698-ae00-0dad30ec97e7" containerName="thanos-sidecar"
Jul 22 02:22:21.909597 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.909549    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="128db35e-9df1-4698-ae00-0dad30ec97e7" containerName="config-reloader"
Jul 22 02:22:21.909597 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.909564    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="128db35e-9df1-4698-ae00-0dad30ec97e7" containerName="kube-rbac-proxy-web"
Jul 22 02:22:21.916784 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.916752    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:21.919794 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.919769    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 22 02:22:21.919960 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.919943    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 22 02:22:21.920018 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.919997    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 22 02:22:21.920275 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.920260    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 22 02:22:21.920621 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.920602    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-wzj4q\""
Jul 22 02:22:21.920773 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.920725    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 22 02:22:21.920937 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.920922    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-drectbtkpsgj2\""
Jul 22 02:22:21.921214 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.921198    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 22 02:22:21.921280 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.921218    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 22 02:22:21.921331 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.921203    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 22 02:22:21.921899 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.921875    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 22 02:22:21.921987 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.921895    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 22 02:22:21.922116 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.922100    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 22 02:22:21.922784 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.922767    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 22 02:22:21.923386 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.923370    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 22 02:22:21.925600 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.925580    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 22 02:22:21.930261 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:21.930239    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 02:22:22.072058 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.071971    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dde833f2-68cb-4230-bb48-3a6605c6457f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.072058 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.072014    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dde833f2-68cb-4230-bb48-3a6605c6457f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.072058 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.072048    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dde833f2-68cb-4230-bb48-3a6605c6457f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.072364 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.072072    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dde833f2-68cb-4230-bb48-3a6605c6457f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.072364 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.072092    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/dde833f2-68cb-4230-bb48-3a6605c6457f-config\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.072364 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.072108    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dde833f2-68cb-4230-bb48-3a6605c6457f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.072364 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.072135    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dde833f2-68cb-4230-bb48-3a6605c6457f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.072364 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.072230    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dde833f2-68cb-4230-bb48-3a6605c6457f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.072364 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.072269    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dde833f2-68cb-4230-bb48-3a6605c6457f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.072364 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.072319    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dde833f2-68cb-4230-bb48-3a6605c6457f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.072364 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.072365    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcltc\" (UniqueName: \"kubernetes.io/projected/dde833f2-68cb-4230-bb48-3a6605c6457f-kube-api-access-kcltc\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.072728 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.072419    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dde833f2-68cb-4230-bb48-3a6605c6457f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.072728 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.072466    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dde833f2-68cb-4230-bb48-3a6605c6457f-web-config\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.072728 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.072497    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dde833f2-68cb-4230-bb48-3a6605c6457f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.072728 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.072522    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dde833f2-68cb-4230-bb48-3a6605c6457f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.072728 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.072550    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dde833f2-68cb-4230-bb48-3a6605c6457f-config-out\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.072728 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.072575    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dde833f2-68cb-4230-bb48-3a6605c6457f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.072728 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.072603    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dde833f2-68cb-4230-bb48-3a6605c6457f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.072728 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.072627    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dde833f2-68cb-4230-bb48-3a6605c6457f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.072728 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.072657    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dde833f2-68cb-4230-bb48-3a6605c6457f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.173899 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.173855    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/dde833f2-68cb-4230-bb48-3a6605c6457f-config\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.174070 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.173906    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dde833f2-68cb-4230-bb48-3a6605c6457f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.174070 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.173939    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dde833f2-68cb-4230-bb48-3a6605c6457f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.174070 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.173981    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dde833f2-68cb-4230-bb48-3a6605c6457f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.174070 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.174007    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dde833f2-68cb-4230-bb48-3a6605c6457f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.174070 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.174038    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dde833f2-68cb-4230-bb48-3a6605c6457f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.174070 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.174068    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcltc\" (UniqueName: \"kubernetes.io/projected/dde833f2-68cb-4230-bb48-3a6605c6457f-kube-api-access-kcltc\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.174351 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.174121    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dde833f2-68cb-4230-bb48-3a6605c6457f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.174351 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.174169    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dde833f2-68cb-4230-bb48-3a6605c6457f-web-config\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.174351 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.174198    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dde833f2-68cb-4230-bb48-3a6605c6457f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.174351 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.174222    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dde833f2-68cb-4230-bb48-3a6605c6457f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.174351 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.174250    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dde833f2-68cb-4230-bb48-3a6605c6457f-config-out\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.174351 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.174276    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dde833f2-68cb-4230-bb48-3a6605c6457f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.174351 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.174304    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dde833f2-68cb-4230-bb48-3a6605c6457f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.174351 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.174333    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dde833f2-68cb-4230-bb48-3a6605c6457f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.174733 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.174356    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dde833f2-68cb-4230-bb48-3a6605c6457f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.174733 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.174384    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dde833f2-68cb-4230-bb48-3a6605c6457f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.174733 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.174417    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dde833f2-68cb-4230-bb48-3a6605c6457f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.174733 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.174446    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dde833f2-68cb-4230-bb48-3a6605c6457f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.174733 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.174475    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dde833f2-68cb-4230-bb48-3a6605c6457f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.174733 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.174836    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dde833f2-68cb-4230-bb48-3a6605c6457f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.174733 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.174997    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dde833f2-68cb-4230-bb48-3a6605c6457f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.174733 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.175022    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dde833f2-68cb-4230-bb48-3a6605c6457f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.174733 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.175276    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dde833f2-68cb-4230-bb48-3a6605c6457f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.176140 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.175872    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dde833f2-68cb-4230-bb48-3a6605c6457f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.176195 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.176139    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dde833f2-68cb-4230-bb48-3a6605c6457f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.177573 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.177548    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dde833f2-68cb-4230-bb48-3a6605c6457f-config-out\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.177907 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.177763    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dde833f2-68cb-4230-bb48-3a6605c6457f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.177907 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.177857    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dde833f2-68cb-4230-bb48-3a6605c6457f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.178092 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.178069    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dde833f2-68cb-4230-bb48-3a6605c6457f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.178154 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.178132    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dde833f2-68cb-4230-bb48-3a6605c6457f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.178211 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.178179    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dde833f2-68cb-4230-bb48-3a6605c6457f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.178865 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.178820    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dde833f2-68cb-4230-bb48-3a6605c6457f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.180256 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.180135    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/dde833f2-68cb-4230-bb48-3a6605c6457f-config\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.180256 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.180213    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dde833f2-68cb-4230-bb48-3a6605c6457f-web-config\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.180525 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.180499    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dde833f2-68cb-4230-bb48-3a6605c6457f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.180585 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.180524    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dde833f2-68cb-4230-bb48-3a6605c6457f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.180585 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.180510    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dde833f2-68cb-4230-bb48-3a6605c6457f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.181358 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.181336    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dde833f2-68cb-4230-bb48-3a6605c6457f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.186896 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.186876    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcltc\" (UniqueName: \"kubernetes.io/projected/dde833f2-68cb-4230-bb48-3a6605c6457f-kube-api-access-kcltc\") pod \"prometheus-k8s-0\" (UID: \"dde833f2-68cb-4230-bb48-3a6605c6457f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.226215 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.226190    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:22.418512 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.418432    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 02:22:22.422809 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:22:22.422780    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddde833f2_68cb_4230_bb48_3a6605c6457f.slice/crio-e351909f3959a42571451064393e0f66673ac24f082934e15da81bf4bcf7de7d WatchSource:0}: Error finding container e351909f3959a42571451064393e0f66673ac24f082934e15da81bf4bcf7de7d: Status 404 returned error can't find the container with id e351909f3959a42571451064393e0f66673ac24f082934e15da81bf4bcf7de7d
Jul 22 02:22:22.835830 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.835790    2581 generic.go:356] "Generic (PLEG): container finished" podID="dde833f2-68cb-4230-bb48-3a6605c6457f" containerID="636d44f590d3183ff53fed18e8d7c05b5f6ba260fc256c49aa135bd7de6ce363" exitCode=0
Jul 22 02:22:22.836295 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.835875    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dde833f2-68cb-4230-bb48-3a6605c6457f","Type":"ContainerDied","Data":"636d44f590d3183ff53fed18e8d7c05b5f6ba260fc256c49aa135bd7de6ce363"}
Jul 22 02:22:22.836295 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.835956    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dde833f2-68cb-4230-bb48-3a6605c6457f","Type":"ContainerStarted","Data":"e351909f3959a42571451064393e0f66673ac24f082934e15da81bf4bcf7de7d"}
Jul 22 02:22:22.904385 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:22.904343    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="128db35e-9df1-4698-ae00-0dad30ec97e7" path="/var/lib/kubelet/pods/128db35e-9df1-4698-ae00-0dad30ec97e7/volumes"
Jul 22 02:22:23.844592 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:23.844551    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dde833f2-68cb-4230-bb48-3a6605c6457f","Type":"ContainerStarted","Data":"2e8032eef0bc31a5097b99fef492ef302dc3f84bec2f31bd335c118b2ac89672"}
Jul 22 02:22:23.845041 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:23.844673    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dde833f2-68cb-4230-bb48-3a6605c6457f","Type":"ContainerStarted","Data":"00534eefbf881c08114578122a75f9f80d3355a2f3a251a47cea879f041640c5"}
Jul 22 02:22:23.845041 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:23.844708    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dde833f2-68cb-4230-bb48-3a6605c6457f","Type":"ContainerStarted","Data":"55a9623dec8dddaf702ee2fed3d539e045f5327538ae15c281c0e0941a34c256"}
Jul 22 02:22:23.845041 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:23.844748    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dde833f2-68cb-4230-bb48-3a6605c6457f","Type":"ContainerStarted","Data":"76bde76b99020e4911eb463424976cb23144fea88390b94accacd49616db83a1"}
Jul 22 02:22:23.845041 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:23.844781    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dde833f2-68cb-4230-bb48-3a6605c6457f","Type":"ContainerStarted","Data":"4ee08cdd24d8712963c4ff10117a209c2adbe02ac1a7c687ba0725e9e709163b"}
Jul 22 02:22:23.845041 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:23.844809    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dde833f2-68cb-4230-bb48-3a6605c6457f","Type":"ContainerStarted","Data":"bece185f3c82b8e2bcecb90d99cb30e7537904e7d8bbe32a41e562e67eaa1484"}
Jul 22 02:22:23.846575 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:23.846551    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5ddb9449c-h5zj7" event={"ID":"10f71205-4482-4c08-8fc4-e8a17b554b5b","Type":"ContainerStarted","Data":"5c28d52eeaf19a497eab87bf62d893b091562b3384b9fcb81dc053854b4027a9"}
Jul 22 02:22:23.846659 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:23.846597    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5ddb9449c-h5zj7" event={"ID":"10f71205-4482-4c08-8fc4-e8a17b554b5b","Type":"ContainerStarted","Data":"53995750f5a55a3668b3fd420f14a8b40ca50f0468b3c1c8d807112dbc60630c"}
Jul 22 02:22:23.846659 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:23.846616    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5ddb9449c-h5zj7" event={"ID":"10f71205-4482-4c08-8fc4-e8a17b554b5b","Type":"ContainerStarted","Data":"8596359d8ac27bd1addc31167a2b14c6701c96936b16070486befa683cda8c23"}
Jul 22 02:22:23.880142 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:23.880095    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.856897744 podStartE2EDuration="2.880077439s" totalImagesPullingTime="0s" totalInitContainerRuntime="23.179695ms" isStatefulPod=false podCreationTimestamp="2026-07-22 02:22:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:22:23.879967965 +0000 UTC m=+165.420761592" watchObservedRunningTime="2026-07-22 02:22:23.880077439 +0000 UTC m=+165.420871065"
Jul 22 02:22:23.905465 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:23.905412    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5ddb9449c-h5zj7" podStartSLOduration=2.145858116 podStartE2EDuration="3.905398217s" totalImagesPullingTime="1.759540101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:22:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:22:23.905059906 +0000 UTC m=+165.445853535" watchObservedRunningTime="2026-07-22 02:22:23.905398217 +0000 UTC m=+165.446191844"
Jul 22 02:22:24.692711 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:24.692674    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:22:24.796725 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:24.796663    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:22:27.226554 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:27.226499    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:22:27.711944 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:27.711909    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:22:27.783009 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:27.782974    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:22:35.855473 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:35.855391    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:22:35.975016 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:35.974983    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:22:53.441779 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:53.441725    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:22:53.509432 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:22:53.509392    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:23:14.475368 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:23:14.475333    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:23:22.226550 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:23:22.226509    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:23:22.242187 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:23:22.242161    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:23:23.110216 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:23:23.110185    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 02:23:30.868614 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:23:30.868578    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:23:30.945891 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:23:30.945856    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:23:42.800663 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:23:42.800619    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:23:42.854400 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:23:42.854364    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:23:54.418365 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:23:54.418327    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:23:54.495054 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:23:54.495018    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:23:57.174827 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:23:57.174796    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:23:57.273383 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:23:57.273354    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:24:05.193830 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:24:05.193717    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:24:05.293884 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:24:05.293852    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:24:05.347772 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:24:05.347718    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:24:38.794682 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:24:38.794651    2581 kubelet.go:1648] "Image garbage collection succeeded"
Jul 22 02:24:40.160307 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:24:40.160274    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:24:41.366276 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:24:41.366243    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:25:02.160017 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:02.159981    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:25:03.579750 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:03.579700    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:25:05.772853 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:05.772818    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:25:05.874166 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:05.874132    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:25:09.835136 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:09.835102    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:25:09.950707 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:09.950671    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:25:10.966498 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:10.966461    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:25:11.064595 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:11.064561    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:25:12.215808 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:12.215771    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:25:12.614866 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:12.614831    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:25:16.983439 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:16.983398    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-h6tsm"]
Jul 22 02:25:16.987090 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:16.987067    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-h6tsm"
Jul 22 02:25:16.989912 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:16.989885    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 22 02:25:16.990063 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:16.989908    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 22 02:25:16.991187 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:16.991158    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-9ft22\""
Jul 22 02:25:16.996760 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:16.996724    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-h6tsm"]
Jul 22 02:25:17.090305 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:17.090258    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/38f49c56-64d2-43a8-bf6d-5b77aad4daad-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-h6tsm\" (UID: \"38f49c56-64d2-43a8-bf6d-5b77aad4daad\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-h6tsm"
Jul 22 02:25:17.090497 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:17.090388    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wc5r\" (UniqueName: \"kubernetes.io/projected/38f49c56-64d2-43a8-bf6d-5b77aad4daad-kube-api-access-2wc5r\") pod \"cert-manager-cainjector-559cdd8c87-h6tsm\" (UID: \"38f49c56-64d2-43a8-bf6d-5b77aad4daad\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-h6tsm"
Jul 22 02:25:17.191387 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:17.191337    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/38f49c56-64d2-43a8-bf6d-5b77aad4daad-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-h6tsm\" (UID: \"38f49c56-64d2-43a8-bf6d-5b77aad4daad\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-h6tsm"
Jul 22 02:25:17.191573 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:17.191415    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wc5r\" (UniqueName: \"kubernetes.io/projected/38f49c56-64d2-43a8-bf6d-5b77aad4daad-kube-api-access-2wc5r\") pod \"cert-manager-cainjector-559cdd8c87-h6tsm\" (UID: \"38f49c56-64d2-43a8-bf6d-5b77aad4daad\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-h6tsm"
Jul 22 02:25:17.204733 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:17.204703    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/38f49c56-64d2-43a8-bf6d-5b77aad4daad-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-h6tsm\" (UID: \"38f49c56-64d2-43a8-bf6d-5b77aad4daad\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-h6tsm"
Jul 22 02:25:17.206876 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:17.206852    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wc5r\" (UniqueName: \"kubernetes.io/projected/38f49c56-64d2-43a8-bf6d-5b77aad4daad-kube-api-access-2wc5r\") pod \"cert-manager-cainjector-559cdd8c87-h6tsm\" (UID: \"38f49c56-64d2-43a8-bf6d-5b77aad4daad\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-h6tsm"
Jul 22 02:25:17.308078 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:17.308041    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-h6tsm"
Jul 22 02:25:17.444431 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:17.444396    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-h6tsm"]
Jul 22 02:25:17.447950 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:25:17.447919    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod38f49c56_64d2_43a8_bf6d_5b77aad4daad.slice/crio-2350306d87d70ec2e391a83294fb56d330ba41a08fcd472d1716b682f6f28235 WatchSource:0}: Error finding container 2350306d87d70ec2e391a83294fb56d330ba41a08fcd472d1716b682f6f28235: Status 404 returned error can't find the container with id 2350306d87d70ec2e391a83294fb56d330ba41a08fcd472d1716b682f6f28235
Jul 22 02:25:17.449918 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:17.449899    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 02:25:17.564088 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:17.564004    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-h6tsm" event={"ID":"38f49c56-64d2-43a8-bf6d-5b77aad4daad","Type":"ContainerStarted","Data":"2350306d87d70ec2e391a83294fb56d330ba41a08fcd472d1716b682f6f28235"}
Jul 22 02:25:19.812629 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:19.812596    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:25:20.381678 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:20.381643    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-ndszq"]
Jul 22 02:25:20.385340 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:20.385318    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-ndszq"
Jul 22 02:25:20.387999 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:20.387972    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-5m4xv\""
Jul 22 02:25:20.394152 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:20.394127    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-ndszq"]
Jul 22 02:25:20.420417 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:20.420381    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7c5e44f9-dda0-4606-b58c-7cc503623850-bound-sa-token\") pod \"cert-manager-55b4d954d5-ndszq\" (UID: \"7c5e44f9-dda0-4606-b58c-7cc503623850\") " pod="cert-manager/cert-manager-55b4d954d5-ndszq"
Jul 22 02:25:20.420568 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:20.420543    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7nlc\" (UniqueName: \"kubernetes.io/projected/7c5e44f9-dda0-4606-b58c-7cc503623850-kube-api-access-f7nlc\") pod \"cert-manager-55b4d954d5-ndszq\" (UID: \"7c5e44f9-dda0-4606-b58c-7cc503623850\") " pod="cert-manager/cert-manager-55b4d954d5-ndszq"
Jul 22 02:25:20.521876 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:20.521841    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7nlc\" (UniqueName: \"kubernetes.io/projected/7c5e44f9-dda0-4606-b58c-7cc503623850-kube-api-access-f7nlc\") pod \"cert-manager-55b4d954d5-ndszq\" (UID: \"7c5e44f9-dda0-4606-b58c-7cc503623850\") " pod="cert-manager/cert-manager-55b4d954d5-ndszq"
Jul 22 02:25:20.522059 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:20.521910    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7c5e44f9-dda0-4606-b58c-7cc503623850-bound-sa-token\") pod \"cert-manager-55b4d954d5-ndszq\" (UID: \"7c5e44f9-dda0-4606-b58c-7cc503623850\") " pod="cert-manager/cert-manager-55b4d954d5-ndszq"
Jul 22 02:25:20.531288 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:20.531263    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7c5e44f9-dda0-4606-b58c-7cc503623850-bound-sa-token\") pod \"cert-manager-55b4d954d5-ndszq\" (UID: \"7c5e44f9-dda0-4606-b58c-7cc503623850\") " pod="cert-manager/cert-manager-55b4d954d5-ndszq"
Jul 22 02:25:20.531443 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:20.531411    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7nlc\" (UniqueName: \"kubernetes.io/projected/7c5e44f9-dda0-4606-b58c-7cc503623850-kube-api-access-f7nlc\") pod \"cert-manager-55b4d954d5-ndszq\" (UID: \"7c5e44f9-dda0-4606-b58c-7cc503623850\") " pod="cert-manager/cert-manager-55b4d954d5-ndszq"
Jul 22 02:25:20.697053 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:20.696969    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-ndszq"
Jul 22 02:25:21.322715 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:21.322650    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-ndszq"]
Jul 22 02:25:21.324714 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:25:21.324691    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7c5e44f9_dda0_4606_b58c_7cc503623850.slice/crio-acfc96a0385a658abf9cdb5b2af51cb36b824e5f416fab38aa620cf1f1ff8f39 WatchSource:0}: Error finding container acfc96a0385a658abf9cdb5b2af51cb36b824e5f416fab38aa620cf1f1ff8f39: Status 404 returned error can't find the container with id acfc96a0385a658abf9cdb5b2af51cb36b824e5f416fab38aa620cf1f1ff8f39
Jul 22 02:25:21.585204 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:21.585168    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-ndszq" event={"ID":"7c5e44f9-dda0-4606-b58c-7cc503623850","Type":"ContainerStarted","Data":"63ff2c52c136f4e83c05490ae7658b7d3d0790a7181bf854276e4fc693168bee"}
Jul 22 02:25:21.585204 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:21.585215    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-ndszq" event={"ID":"7c5e44f9-dda0-4606-b58c-7cc503623850","Type":"ContainerStarted","Data":"acfc96a0385a658abf9cdb5b2af51cb36b824e5f416fab38aa620cf1f1ff8f39"}
Jul 22 02:25:21.586701 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:21.586672    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-h6tsm" event={"ID":"38f49c56-64d2-43a8-bf6d-5b77aad4daad","Type":"ContainerStarted","Data":"9872317d960c0ae25961d73b540aae5a9470248a161b671ea83e29f2ef3cdf1b"}
Jul 22 02:25:21.603355 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:21.603313    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-ndszq" podStartSLOduration=1.603296971 podStartE2EDuration="1.603296971s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:25:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:25:21.602960227 +0000 UTC m=+343.143753853" watchObservedRunningTime="2026-07-22 02:25:21.603296971 +0000 UTC m=+343.144090597"
Jul 22 02:25:21.622926 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:21.622869    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-h6tsm" podStartSLOduration=1.806621534 podStartE2EDuration="5.622852139s" totalImagesPullingTime="3.816230605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:25:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:25:21.622300143 +0000 UTC m=+343.163093772" watchObservedRunningTime="2026-07-22 02:25:21.622852139 +0000 UTC m=+343.163645767"
Jul 22 02:25:22.046187 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:22.046147    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:25:22.102858 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:22.102831    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:25:24.315928 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:24.315879    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:25:24.415903 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:24.415860    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:25:30.845813 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:30.845731    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:25:30.945764 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:30.945712    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:25:32.746275 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:32.746240    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:25:41.017085 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:41.017054    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:25:41.067028 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:41.066994    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-645b87676-nt9sw"]
Jul 22 02:25:41.070172 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:41.070156    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-645b87676-nt9sw"
Jul 22 02:25:41.074132 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:41.074110    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 22 02:25:41.074132 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:41.074109    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 22 02:25:41.075215 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:41.075197    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 22 02:25:41.075301 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:41.075278    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 22 02:25:41.076056 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:41.076038    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-f42xq\""
Jul 22 02:25:41.076648 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:41.076626    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 22 02:25:41.091971 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:41.091935    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-645b87676-nt9sw"]
Jul 22 02:25:41.119354 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:41.119321    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:25:41.202795 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:41.202752    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqkk4\" (UniqueName: \"kubernetes.io/projected/ebe3ece5-117c-4ae9-adf8-bb7f16e7f328-kube-api-access-gqkk4\") pod \"lws-controller-manager-645b87676-nt9sw\" (UID: \"ebe3ece5-117c-4ae9-adf8-bb7f16e7f328\") " pod="openshift-lws-operator/lws-controller-manager-645b87676-nt9sw"
Jul 22 02:25:41.203004 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:41.202812    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ebe3ece5-117c-4ae9-adf8-bb7f16e7f328-manager-config\") pod \"lws-controller-manager-645b87676-nt9sw\" (UID: \"ebe3ece5-117c-4ae9-adf8-bb7f16e7f328\") " pod="openshift-lws-operator/lws-controller-manager-645b87676-nt9sw"
Jul 22 02:25:41.203004 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:41.202838    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ebe3ece5-117c-4ae9-adf8-bb7f16e7f328-cert\") pod \"lws-controller-manager-645b87676-nt9sw\" (UID: \"ebe3ece5-117c-4ae9-adf8-bb7f16e7f328\") " pod="openshift-lws-operator/lws-controller-manager-645b87676-nt9sw"
Jul 22 02:25:41.203004 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:41.202866    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ebe3ece5-117c-4ae9-adf8-bb7f16e7f328-metrics-cert\") pod \"lws-controller-manager-645b87676-nt9sw\" (UID: \"ebe3ece5-117c-4ae9-adf8-bb7f16e7f328\") " pod="openshift-lws-operator/lws-controller-manager-645b87676-nt9sw"
Jul 22 02:25:41.303854 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:41.303770    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqkk4\" (UniqueName: \"kubernetes.io/projected/ebe3ece5-117c-4ae9-adf8-bb7f16e7f328-kube-api-access-gqkk4\") pod \"lws-controller-manager-645b87676-nt9sw\" (UID: \"ebe3ece5-117c-4ae9-adf8-bb7f16e7f328\") " pod="openshift-lws-operator/lws-controller-manager-645b87676-nt9sw"
Jul 22 02:25:41.303854 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:41.303814    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ebe3ece5-117c-4ae9-adf8-bb7f16e7f328-manager-config\") pod \"lws-controller-manager-645b87676-nt9sw\" (UID: \"ebe3ece5-117c-4ae9-adf8-bb7f16e7f328\") " pod="openshift-lws-operator/lws-controller-manager-645b87676-nt9sw"
Jul 22 02:25:41.303854 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:41.303838    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ebe3ece5-117c-4ae9-adf8-bb7f16e7f328-cert\") pod \"lws-controller-manager-645b87676-nt9sw\" (UID: \"ebe3ece5-117c-4ae9-adf8-bb7f16e7f328\") " pod="openshift-lws-operator/lws-controller-manager-645b87676-nt9sw"
Jul 22 02:25:41.304073 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:41.303865    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ebe3ece5-117c-4ae9-adf8-bb7f16e7f328-metrics-cert\") pod \"lws-controller-manager-645b87676-nt9sw\" (UID: \"ebe3ece5-117c-4ae9-adf8-bb7f16e7f328\") " pod="openshift-lws-operator/lws-controller-manager-645b87676-nt9sw"
Jul 22 02:25:41.304534 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:41.304504    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ebe3ece5-117c-4ae9-adf8-bb7f16e7f328-manager-config\") pod \"lws-controller-manager-645b87676-nt9sw\" (UID: \"ebe3ece5-117c-4ae9-adf8-bb7f16e7f328\") " pod="openshift-lws-operator/lws-controller-manager-645b87676-nt9sw"
Jul 22 02:25:41.306358 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:41.306336    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ebe3ece5-117c-4ae9-adf8-bb7f16e7f328-cert\") pod \"lws-controller-manager-645b87676-nt9sw\" (UID: \"ebe3ece5-117c-4ae9-adf8-bb7f16e7f328\") " pod="openshift-lws-operator/lws-controller-manager-645b87676-nt9sw"
Jul 22 02:25:41.306450 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:41.306374    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ebe3ece5-117c-4ae9-adf8-bb7f16e7f328-metrics-cert\") pod \"lws-controller-manager-645b87676-nt9sw\" (UID: \"ebe3ece5-117c-4ae9-adf8-bb7f16e7f328\") " pod="openshift-lws-operator/lws-controller-manager-645b87676-nt9sw"
Jul 22 02:25:41.318404 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:41.318383    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqkk4\" (UniqueName: \"kubernetes.io/projected/ebe3ece5-117c-4ae9-adf8-bb7f16e7f328-kube-api-access-gqkk4\") pod \"lws-controller-manager-645b87676-nt9sw\" (UID: \"ebe3ece5-117c-4ae9-adf8-bb7f16e7f328\") " pod="openshift-lws-operator/lws-controller-manager-645b87676-nt9sw"
Jul 22 02:25:41.379408 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:41.379373    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-645b87676-nt9sw"
Jul 22 02:25:41.543368 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:41.543332    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-645b87676-nt9sw"]
Jul 22 02:25:41.546515 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:25:41.546477    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebe3ece5_117c_4ae9_adf8_bb7f16e7f328.slice/crio-affb57b129a705558bfacb1ffcfa4237b26be939ef40a7889babc1ad3d242cce WatchSource:0}: Error finding container affb57b129a705558bfacb1ffcfa4237b26be939ef40a7889babc1ad3d242cce: Status 404 returned error can't find the container with id affb57b129a705558bfacb1ffcfa4237b26be939ef40a7889babc1ad3d242cce
Jul 22 02:25:41.670803 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:41.670703    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-645b87676-nt9sw" event={"ID":"ebe3ece5-117c-4ae9-adf8-bb7f16e7f328","Type":"ContainerStarted","Data":"affb57b129a705558bfacb1ffcfa4237b26be939ef40a7889babc1ad3d242cce"}
Jul 22 02:25:44.665887 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:44.665849    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:25:44.785850 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:44.785808    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:25:45.234074 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:45.234013    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:25:45.491834 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:45.491801    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:25:45.598502 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:45.598471    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:25:45.692147 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:45.692108    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-645b87676-nt9sw" event={"ID":"ebe3ece5-117c-4ae9-adf8-bb7f16e7f328","Type":"ContainerStarted","Data":"50fcc588dad12d2d5f6822448b9e5fa8e3494e88c3b8657b5775242044265f86"}
Jul 22 02:25:45.692614 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:45.692298    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-645b87676-nt9sw"
Jul 22 02:25:45.696788 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:45.696749    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-fnrth"]
Jul 22 02:25:45.700234 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:45.700218    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-fnrth"
Jul 22 02:25:45.705466 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:45.705438    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 22 02:25:45.705466 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:45.705438    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 22 02:25:45.705624 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:45.705492    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 22 02:25:45.705624 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:45.705521    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-gf5lf\""
Jul 22 02:25:45.706669 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:45.706651    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 22 02:25:45.716861 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:45.716837    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-fnrth"]
Jul 22 02:25:45.721154 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:45.721114    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-645b87676-nt9sw" podStartSLOduration=0.979551243 podStartE2EDuration="4.721098321s" totalImagesPullingTime="3.741547078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:25:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:25:45.718590726 +0000 UTC m=+367.259384354" watchObservedRunningTime="2026-07-22 02:25:45.721098321 +0000 UTC m=+367.261891947"
Jul 22 02:25:45.818675 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:45.818586    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:25:45.845860 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:45.845825    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fca07606-2755-4cd9-af37-bbd12ba712b7-tmp\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-fnrth\" (UID: \"fca07606-2755-4cd9-af37-bbd12ba712b7\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-fnrth"
Jul 22 02:25:45.846010 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:45.845901    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fca07606-2755-4cd9-af37-bbd12ba712b7-manifests\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-fnrth\" (UID: \"fca07606-2755-4cd9-af37-bbd12ba712b7\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-fnrth"
Jul 22 02:25:45.846069 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:45.846040    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fca07606-2755-4cd9-af37-bbd12ba712b7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-fnrth\" (UID: \"fca07606-2755-4cd9-af37-bbd12ba712b7\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-fnrth"
Jul 22 02:25:45.846153 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:45.846137    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fca07606-2755-4cd9-af37-bbd12ba712b7-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-fnrth\" (UID: \"fca07606-2755-4cd9-af37-bbd12ba712b7\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-fnrth"
Jul 22 02:25:45.846204 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:45.846177    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vd5jq\" (UniqueName: \"kubernetes.io/projected/fca07606-2755-4cd9-af37-bbd12ba712b7-kube-api-access-vd5jq\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-fnrth\" (UID: \"fca07606-2755-4cd9-af37-bbd12ba712b7\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-fnrth"
Jul 22 02:25:45.922685 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:45.922650    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:25:45.946844 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:45.946809    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fca07606-2755-4cd9-af37-bbd12ba712b7-manifests\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-fnrth\" (UID: \"fca07606-2755-4cd9-af37-bbd12ba712b7\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-fnrth"
Jul 22 02:25:45.946980 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:45.946853    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fca07606-2755-4cd9-af37-bbd12ba712b7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-fnrth\" (UID: \"fca07606-2755-4cd9-af37-bbd12ba712b7\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-fnrth"
Jul 22 02:25:45.946980 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:45.946906    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fca07606-2755-4cd9-af37-bbd12ba712b7-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-fnrth\" (UID: \"fca07606-2755-4cd9-af37-bbd12ba712b7\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-fnrth"
Jul 22 02:25:45.946980 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:45.946926    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vd5jq\" (UniqueName: \"kubernetes.io/projected/fca07606-2755-4cd9-af37-bbd12ba712b7-kube-api-access-vd5jq\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-fnrth\" (UID: \"fca07606-2755-4cd9-af37-bbd12ba712b7\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-fnrth"
Jul 22 02:25:45.947137 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:45.946979    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fca07606-2755-4cd9-af37-bbd12ba712b7-tmp\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-fnrth\" (UID: \"fca07606-2755-4cd9-af37-bbd12ba712b7\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-fnrth"
Jul 22 02:25:45.947272 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:45.947252    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fca07606-2755-4cd9-af37-bbd12ba712b7-manifests\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-fnrth\" (UID: \"fca07606-2755-4cd9-af37-bbd12ba712b7\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-fnrth"
Jul 22 02:25:45.949405 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:45.949377    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fca07606-2755-4cd9-af37-bbd12ba712b7-tmp\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-fnrth\" (UID: \"fca07606-2755-4cd9-af37-bbd12ba712b7\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-fnrth"
Jul 22 02:25:45.949505 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:45.949489    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fca07606-2755-4cd9-af37-bbd12ba712b7-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-fnrth\" (UID: \"fca07606-2755-4cd9-af37-bbd12ba712b7\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-fnrth"
Jul 22 02:25:45.949551 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:45.949506    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fca07606-2755-4cd9-af37-bbd12ba712b7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-fnrth\" (UID: \"fca07606-2755-4cd9-af37-bbd12ba712b7\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-fnrth"
Jul 22 02:25:45.957923 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:45.957903    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vd5jq\" (UniqueName: \"kubernetes.io/projected/fca07606-2755-4cd9-af37-bbd12ba712b7-kube-api-access-vd5jq\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-fnrth\" (UID: \"fca07606-2755-4cd9-af37-bbd12ba712b7\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-fnrth"
Jul 22 02:25:46.010199 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:46.010164    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-fnrth"
Jul 22 02:25:46.022121 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:46.022091    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:25:46.118989 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:46.118951    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:25:46.159696 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:46.159671    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-fnrth"]
Jul 22 02:25:46.161965 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:25:46.161938    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfca07606_2755_4cd9_af37_bbd12ba712b7.slice/crio-44aa40a006ffde87441d6a741b096a32fde4a94f2865d685c8298e1ac07b2e83 WatchSource:0}: Error finding container 44aa40a006ffde87441d6a741b096a32fde4a94f2865d685c8298e1ac07b2e83: Status 404 returned error can't find the container with id 44aa40a006ffde87441d6a741b096a32fde4a94f2865d685c8298e1ac07b2e83
Jul 22 02:25:46.223297 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:46.223265    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:25:46.332073 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:46.332039    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:25:46.699586 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:46.699536    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-fnrth" event={"ID":"fca07606-2755-4cd9-af37-bbd12ba712b7","Type":"ContainerStarted","Data":"44aa40a006ffde87441d6a741b096a32fde4a94f2865d685c8298e1ac07b2e83"}
Jul 22 02:25:49.718470 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:49.718432    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-fnrth" event={"ID":"fca07606-2755-4cd9-af37-bbd12ba712b7","Type":"ContainerStarted","Data":"b19e6ab353f4bd71fe8f8c5ed9368967ede3968f263c96bd463843b9a866410b"}
Jul 22 02:25:50.724572 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:50.724481    2581 generic.go:356] "Generic (PLEG): container finished" podID="fca07606-2755-4cd9-af37-bbd12ba712b7" containerID="b19e6ab353f4bd71fe8f8c5ed9368967ede3968f263c96bd463843b9a866410b" exitCode=0
Jul 22 02:25:50.724984 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:50.724574    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-fnrth" event={"ID":"fca07606-2755-4cd9-af37-bbd12ba712b7","Type":"ContainerDied","Data":"b19e6ab353f4bd71fe8f8c5ed9368967ede3968f263c96bd463843b9a866410b"}
Jul 22 02:25:51.731022 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:51.730981    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-fnrth" event={"ID":"fca07606-2755-4cd9-af37-bbd12ba712b7","Type":"ContainerStarted","Data":"b1bc9f0b0c6f0e3342099413a3d2a6bc617556490fbb78d7e1b42380a08cd690"}
Jul 22 02:25:51.731413 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:51.731095    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-fnrth"
Jul 22 02:25:51.763076 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:51.763021    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-fnrth" podStartSLOduration=2.324408041 podStartE2EDuration="6.763004368s" totalImagesPullingTime="3.398385445s" totalInitContainerRuntime="1.040210882s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:25:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:25:51.760113471 +0000 UTC m=+373.300907098" watchObservedRunningTime="2026-07-22 02:25:51.763004368 +0000 UTC m=+373.303798033"
Jul 22 02:25:56.701883 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:25:56.701854    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-645b87676-nt9sw"
Jul 22 02:26:02.737524 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:02.737487    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-fnrth"
Jul 22 02:26:05.092297 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:05.092255    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:26:05.292156 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:05.292118    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:26:06.465684 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:06.465643    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:26:06.685304 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:06.685269    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:26:06.750964 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:06.750928    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-65c4bb8965-d2h6v"]
Jul 22 02:26:06.754309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:06.754282    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-65c4bb8965-d2h6v"
Jul 22 02:26:06.763001 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:06.762974    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 22 02:26:06.763122 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:06.763014    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 22 02:26:06.763122 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:06.763028    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-hlsdx\""
Jul 22 02:26:06.763122 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:06.763046    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 22 02:26:06.763122 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:06.763056    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 22 02:26:06.783426 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:06.783374    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-65c4bb8965-d2h6v"]
Jul 22 02:26:06.842977 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:06.842938    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fe58e91-0a3d-4643-b65b-fbd1f3832864-tls-certs\") pod \"kube-auth-proxy-65c4bb8965-d2h6v\" (UID: \"6fe58e91-0a3d-4643-b65b-fbd1f3832864\") " pod="openshift-ingress/kube-auth-proxy-65c4bb8965-d2h6v"
Jul 22 02:26:06.843167 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:06.843006    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djn9s\" (UniqueName: \"kubernetes.io/projected/6fe58e91-0a3d-4643-b65b-fbd1f3832864-kube-api-access-djn9s\") pod \"kube-auth-proxy-65c4bb8965-d2h6v\" (UID: \"6fe58e91-0a3d-4643-b65b-fbd1f3832864\") " pod="openshift-ingress/kube-auth-proxy-65c4bb8965-d2h6v"
Jul 22 02:26:06.843167 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:06.843032    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6fe58e91-0a3d-4643-b65b-fbd1f3832864-tmp\") pod \"kube-auth-proxy-65c4bb8965-d2h6v\" (UID: \"6fe58e91-0a3d-4643-b65b-fbd1f3832864\") " pod="openshift-ingress/kube-auth-proxy-65c4bb8965-d2h6v"
Jul 22 02:26:06.944186 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:06.944148    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fe58e91-0a3d-4643-b65b-fbd1f3832864-tls-certs\") pod \"kube-auth-proxy-65c4bb8965-d2h6v\" (UID: \"6fe58e91-0a3d-4643-b65b-fbd1f3832864\") " pod="openshift-ingress/kube-auth-proxy-65c4bb8965-d2h6v"
Jul 22 02:26:06.944362 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:06.944207    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djn9s\" (UniqueName: \"kubernetes.io/projected/6fe58e91-0a3d-4643-b65b-fbd1f3832864-kube-api-access-djn9s\") pod \"kube-auth-proxy-65c4bb8965-d2h6v\" (UID: \"6fe58e91-0a3d-4643-b65b-fbd1f3832864\") " pod="openshift-ingress/kube-auth-proxy-65c4bb8965-d2h6v"
Jul 22 02:26:06.944362 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:06.944229    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6fe58e91-0a3d-4643-b65b-fbd1f3832864-tmp\") pod \"kube-auth-proxy-65c4bb8965-d2h6v\" (UID: \"6fe58e91-0a3d-4643-b65b-fbd1f3832864\") " pod="openshift-ingress/kube-auth-proxy-65c4bb8965-d2h6v"
Jul 22 02:26:06.946517 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:06.946485    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6fe58e91-0a3d-4643-b65b-fbd1f3832864-tmp\") pod \"kube-auth-proxy-65c4bb8965-d2h6v\" (UID: \"6fe58e91-0a3d-4643-b65b-fbd1f3832864\") " pod="openshift-ingress/kube-auth-proxy-65c4bb8965-d2h6v"
Jul 22 02:26:06.946619 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:06.946589    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fe58e91-0a3d-4643-b65b-fbd1f3832864-tls-certs\") pod \"kube-auth-proxy-65c4bb8965-d2h6v\" (UID: \"6fe58e91-0a3d-4643-b65b-fbd1f3832864\") " pod="openshift-ingress/kube-auth-proxy-65c4bb8965-d2h6v"
Jul 22 02:26:06.986334 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:06.986302    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djn9s\" (UniqueName: \"kubernetes.io/projected/6fe58e91-0a3d-4643-b65b-fbd1f3832864-kube-api-access-djn9s\") pod \"kube-auth-proxy-65c4bb8965-d2h6v\" (UID: \"6fe58e91-0a3d-4643-b65b-fbd1f3832864\") " pod="openshift-ingress/kube-auth-proxy-65c4bb8965-d2h6v"
Jul 22 02:26:07.016526 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:07.016414    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:26:07.064549 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:07.064515    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-65c4bb8965-d2h6v"
Jul 22 02:26:07.178886 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:07.178851    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:26:07.245510 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:07.245486    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-65c4bb8965-d2h6v"]
Jul 22 02:26:07.247133 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:26:07.247107    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fe58e91_0a3d_4643_b65b_fbd1f3832864.slice/crio-7e9260ecb3c41c830e03c7811413a313b424373a4071ee5706412715ad73fc9e WatchSource:0}: Error finding container 7e9260ecb3c41c830e03c7811413a313b424373a4071ee5706412715ad73fc9e: Status 404 returned error can't find the container with id 7e9260ecb3c41c830e03c7811413a313b424373a4071ee5706412715ad73fc9e
Jul 22 02:26:07.309580 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:07.309494    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:26:07.670937 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:07.670848    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:26:07.806915 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:07.806877    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-65c4bb8965-d2h6v" event={"ID":"6fe58e91-0a3d-4643-b65b-fbd1f3832864","Type":"ContainerStarted","Data":"7e9260ecb3c41c830e03c7811413a313b424373a4071ee5706412715ad73fc9e"}
Jul 22 02:26:08.068339 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:08.068304    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:26:08.435343 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:08.435248    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:26:08.604032 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:08.603996    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:26:08.699977 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:08.699870    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:26:09.817375 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:09.817284    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-65c4bb8965-d2h6v" event={"ID":"6fe58e91-0a3d-4643-b65b-fbd1f3832864","Type":"ContainerStarted","Data":"38a1342892854d3be61a3f543234fe6c2ce01c9165c853377ebe1986cd192f23"}
Jul 22 02:26:09.836183 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:09.836128    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-65c4bb8965-d2h6v" podStartSLOduration=1.589627218 podStartE2EDuration="3.836112431s" totalImagesPullingTime="2.246485213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:26:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:26:09.834090234 +0000 UTC m=+391.374883863" watchObservedRunningTime="2026-07-22 02:26:09.836112431 +0000 UTC m=+391.376906097"
Jul 22 02:26:09.901904 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:09.901871    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:26:16.598640 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:16.598604    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:26:16.808871 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:16.808833    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:26:16.933475 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:16.933392    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:26:17.034426 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:17.034385    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:26:17.345450 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:17.345417    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:26:17.615841 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:17.615725    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:26:17.967174 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:17.967098    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:26:18.219008 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:18.218922    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:26:18.388608 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:18.388567    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:26:18.986464 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:18.986430    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:26:19.079974 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:19.079946    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:26:19.141685 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:19.141649    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:26:32.832379 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:32.832341    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:26:32.935399 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:32.935364    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:26:39.262331 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:39.262294    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:26:39.320822 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:39.320786    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:26:41.222791 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:41.222714    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:26:41.295341 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:26:41.295308    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:27:06.492556 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:06.492475    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:27:06.617103 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:06.617061    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:27:06.822359 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:06.822254    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:27:28.421359 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:28.421320    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:27:30.381783 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:30.381710    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:27:30.439408 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:30.439378    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:27:35.265673 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:35.265634    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:27:38.859527 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:38.859493    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:27:41.470765 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:41.470721    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:27:41.723302 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:41.723210    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:27:42.006930 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:42.006892    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:27:43.479431 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:43.479397    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:27:43.757532 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:43.757494    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:27:44.366509 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:44.366467    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:27:46.932936 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:46.932902    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:27:47.175758 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:47.175709    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:27:47.770590 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:47.770550    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:27:49.593606 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:49.593569    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:27:51.101558 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:51.101520    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:27:51.212443 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:51.212399    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:27:51.820760 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:51.820708    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-wknpt"]
Jul 22 02:27:51.824307 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:51.824286    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wknpt"
Jul 22 02:27:51.828301 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:51.828278    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 22 02:27:51.828621 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:51.828607    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 22 02:27:51.828621 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:51.828614    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 22 02:27:51.829821 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:51.829799    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-ss4lb\""
Jul 22 02:27:51.830188 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:51.830165    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 22 02:27:51.841162 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:51.841141    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-wknpt"]
Jul 22 02:27:51.854850 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:51.854832    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:27:51.898120 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:51.898091    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pv5k9\" (UniqueName: \"kubernetes.io/projected/1e9a69b9-4f1c-4673-9fb3-6bc0b7652766-kube-api-access-pv5k9\") pod \"kuadrant-console-plugin-66b5774985-wknpt\" (UID: \"1e9a69b9-4f1c-4673-9fb3-6bc0b7652766\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wknpt"
Jul 22 02:27:51.898273 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:51.898129    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1e9a69b9-4f1c-4673-9fb3-6bc0b7652766-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-wknpt\" (UID: \"1e9a69b9-4f1c-4673-9fb3-6bc0b7652766\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wknpt"
Jul 22 02:27:51.898273 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:51.898195    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1e9a69b9-4f1c-4673-9fb3-6bc0b7652766-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-wknpt\" (UID: \"1e9a69b9-4f1c-4673-9fb3-6bc0b7652766\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wknpt"
Jul 22 02:27:51.999099 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:51.999063    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv5k9\" (UniqueName: \"kubernetes.io/projected/1e9a69b9-4f1c-4673-9fb3-6bc0b7652766-kube-api-access-pv5k9\") pod \"kuadrant-console-plugin-66b5774985-wknpt\" (UID: \"1e9a69b9-4f1c-4673-9fb3-6bc0b7652766\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wknpt"
Jul 22 02:27:51.999289 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:51.999123    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1e9a69b9-4f1c-4673-9fb3-6bc0b7652766-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-wknpt\" (UID: \"1e9a69b9-4f1c-4673-9fb3-6bc0b7652766\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wknpt"
Jul 22 02:27:51.999289 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:51.999175    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1e9a69b9-4f1c-4673-9fb3-6bc0b7652766-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-wknpt\" (UID: \"1e9a69b9-4f1c-4673-9fb3-6bc0b7652766\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wknpt"
Jul 22 02:27:51.999289 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:27:51.999241    2581 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 22 02:27:51.999448 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:27:51.999307    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e9a69b9-4f1c-4673-9fb3-6bc0b7652766-plugin-serving-cert podName:1e9a69b9-4f1c-4673-9fb3-6bc0b7652766 nodeName:}" failed. No retries permitted until 2026-07-22 02:27:52.499286359 +0000 UTC m=+494.040079978 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/1e9a69b9-4f1c-4673-9fb3-6bc0b7652766-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-wknpt" (UID: "1e9a69b9-4f1c-4673-9fb3-6bc0b7652766") : secret "plugin-serving-cert" not found
Jul 22 02:27:51.999996 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:51.999974    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1e9a69b9-4f1c-4673-9fb3-6bc0b7652766-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-wknpt\" (UID: \"1e9a69b9-4f1c-4673-9fb3-6bc0b7652766\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wknpt"
Jul 22 02:27:52.009906 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:52.009887    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pv5k9\" (UniqueName: \"kubernetes.io/projected/1e9a69b9-4f1c-4673-9fb3-6bc0b7652766-kube-api-access-pv5k9\") pod \"kuadrant-console-plugin-66b5774985-wknpt\" (UID: \"1e9a69b9-4f1c-4673-9fb3-6bc0b7652766\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wknpt"
Jul 22 02:27:52.320680 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:52.320645    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:27:52.504850 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:52.504813    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1e9a69b9-4f1c-4673-9fb3-6bc0b7652766-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-wknpt\" (UID: \"1e9a69b9-4f1c-4673-9fb3-6bc0b7652766\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wknpt"
Jul 22 02:27:52.507349 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:52.507325    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1e9a69b9-4f1c-4673-9fb3-6bc0b7652766-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-wknpt\" (UID: \"1e9a69b9-4f1c-4673-9fb3-6bc0b7652766\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wknpt"
Jul 22 02:27:52.733629 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:52.733548    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wknpt"
Jul 22 02:27:52.874862 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:52.874837    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-wknpt"]
Jul 22 02:27:52.877471 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:27:52.877442    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1e9a69b9_4f1c_4673_9fb3_6bc0b7652766.slice/crio-25700aed6d46ff46e97a80bf2cc013f62b2cbf5de1f7089c5ea0160649517cbf WatchSource:0}: Error finding container 25700aed6d46ff46e97a80bf2cc013f62b2cbf5de1f7089c5ea0160649517cbf: Status 404 returned error can't find the container with id 25700aed6d46ff46e97a80bf2cc013f62b2cbf5de1f7089c5ea0160649517cbf
Jul 22 02:27:53.302640 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:53.302604    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wknpt" event={"ID":"1e9a69b9-4f1c-4673-9fb3-6bc0b7652766","Type":"ContainerStarted","Data":"25700aed6d46ff46e97a80bf2cc013f62b2cbf5de1f7089c5ea0160649517cbf"}
Jul 22 02:27:55.160963 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:55.160927    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:27:55.294355 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:55.294313    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:27:56.233491 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:56.233426    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:27:56.398750 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:27:56.398699    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:28:03.651237 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:03.651206    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:28:03.802200 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:03.802166    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:28:06.815837 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:06.815785    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:28:07.020141 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:07.020100    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:28:09.505085 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:09.505039    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:28:09.655054 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:09.655007    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:28:13.035392 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:13.035361    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:28:13.151409 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:13.151373    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:28:14.415075 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:14.414995    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wknpt" event={"ID":"1e9a69b9-4f1c-4673-9fb3-6bc0b7652766","Type":"ContainerStarted","Data":"85353481c96549477babb28dc9a75f7afa1a97ad95784c0b7025fca86969c056"}
Jul 22 02:28:14.434387 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:14.434337    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wknpt" podStartSLOduration=2.221684624 podStartE2EDuration="23.434319771s" totalImagesPullingTime="21.212635147s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:27:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:28:14.432663691 +0000 UTC m=+515.973457321" watchObservedRunningTime="2026-07-22 02:28:14.434319771 +0000 UTC m=+515.975113398"
Jul 22 02:28:23.007675 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:23.007642    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:28:23.124015 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:23.123985    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:28:26.048065 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:26.048030    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:28:26.171023 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:26.170992    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:28:34.289501 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:34.289404    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:28:34.565995 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:34.565908    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:28:36.113752 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:36.113699    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:28:36.282641 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:36.282607    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:28:36.461340 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:36.461255    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:28:36.650683 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:36.650650    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:28:36.654041 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:36.654025    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"
Jul 22 02:28:36.662324 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:36.662302    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 22 02:28:36.686460 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:36.686428    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:28:36.796643 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:36.796610    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:28:36.825241 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:36.825209    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/061682e8-bf94-4a04-9ff4-e877181535ad-config-file\") pod \"limitador-limitador-54d6d988d5-t9zk2\" (UID: \"061682e8-bf94-4a04-9ff4-e877181535ad\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"
Jul 22 02:28:36.825404 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:36.825255    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2gbz\" (UniqueName: \"kubernetes.io/projected/061682e8-bf94-4a04-9ff4-e877181535ad-kube-api-access-c2gbz\") pod \"limitador-limitador-54d6d988d5-t9zk2\" (UID: \"061682e8-bf94-4a04-9ff4-e877181535ad\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"
Jul 22 02:28:36.925810 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:36.925774    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/061682e8-bf94-4a04-9ff4-e877181535ad-config-file\") pod \"limitador-limitador-54d6d988d5-t9zk2\" (UID: \"061682e8-bf94-4a04-9ff4-e877181535ad\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"
Jul 22 02:28:36.925967 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:36.925817    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2gbz\" (UniqueName: \"kubernetes.io/projected/061682e8-bf94-4a04-9ff4-e877181535ad-kube-api-access-c2gbz\") pod \"limitador-limitador-54d6d988d5-t9zk2\" (UID: \"061682e8-bf94-4a04-9ff4-e877181535ad\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"
Jul 22 02:28:36.926463 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:36.926442    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/061682e8-bf94-4a04-9ff4-e877181535ad-config-file\") pod \"limitador-limitador-54d6d988d5-t9zk2\" (UID: \"061682e8-bf94-4a04-9ff4-e877181535ad\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"
Jul 22 02:28:36.953499 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:36.953466    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2gbz\" (UniqueName: \"kubernetes.io/projected/061682e8-bf94-4a04-9ff4-e877181535ad-kube-api-access-c2gbz\") pod \"limitador-limitador-54d6d988d5-t9zk2\" (UID: \"061682e8-bf94-4a04-9ff4-e877181535ad\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"
Jul 22 02:28:36.964347 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:36.964323    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"
Jul 22 02:28:37.126240 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:37.126208    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:28:37.129047 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:28:37.129017    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod061682e8_bf94_4a04_9ff4_e877181535ad.slice/crio-e80292b3c25a7a21bd629da737f104b4e9055bc3bbedcc049c1016cf66225bd8 WatchSource:0}: Error finding container e80292b3c25a7a21bd629da737f104b4e9055bc3bbedcc049c1016cf66225bd8: Status 404 returned error can't find the container with id e80292b3c25a7a21bd629da737f104b4e9055bc3bbedcc049c1016cf66225bd8
Jul 22 02:28:37.317003 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:37.316917    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:28:37.483390 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:37.483340    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:28:37.527928 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:37.527892    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-t9zk2" event={"ID":"061682e8-bf94-4a04-9ff4-e877181535ad","Type":"ContainerStarted","Data":"e80292b3c25a7a21bd629da737f104b4e9055bc3bbedcc049c1016cf66225bd8"}
Jul 22 02:28:37.773521 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:37.773480    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-x49pb"]
Jul 22 02:28:37.777520 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:37.777491    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-x49pb"
Jul 22 02:28:37.786130 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:37.785930    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-htxd8\""
Jul 22 02:28:37.794941 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:37.794917    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-x49pb"]
Jul 22 02:28:37.935428 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:37.935392    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbn6d\" (UniqueName: \"kubernetes.io/projected/da4bf1ec-30e6-4232-a356-f775c0049f2a-kube-api-access-cbn6d\") pod \"authorino-75d556bd77-x49pb\" (UID: \"da4bf1ec-30e6-4232-a356-f775c0049f2a\") " pod="kuadrant-system/authorino-75d556bd77-x49pb"
Jul 22 02:28:38.036165 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:38.036069    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbn6d\" (UniqueName: \"kubernetes.io/projected/da4bf1ec-30e6-4232-a356-f775c0049f2a-kube-api-access-cbn6d\") pod \"authorino-75d556bd77-x49pb\" (UID: \"da4bf1ec-30e6-4232-a356-f775c0049f2a\") " pod="kuadrant-system/authorino-75d556bd77-x49pb"
Jul 22 02:28:38.054381 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:38.054354    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbn6d\" (UniqueName: \"kubernetes.io/projected/da4bf1ec-30e6-4232-a356-f775c0049f2a-kube-api-access-cbn6d\") pod \"authorino-75d556bd77-x49pb\" (UID: \"da4bf1ec-30e6-4232-a356-f775c0049f2a\") " pod="kuadrant-system/authorino-75d556bd77-x49pb"
Jul 22 02:28:38.091987 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:38.091947    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-x49pb"
Jul 22 02:28:38.268495 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:38.268465    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-x49pb"]
Jul 22 02:28:38.815712 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:28:38.815668    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podda4bf1ec_30e6_4232_a356_f775c0049f2a.slice/crio-a124d493b4a7462c6b2ae566e3fe83c408c657e83b58c39e4bc766b4c391b7ee WatchSource:0}: Error finding container a124d493b4a7462c6b2ae566e3fe83c408c657e83b58c39e4bc766b4c391b7ee: Status 404 returned error can't find the container with id a124d493b4a7462c6b2ae566e3fe83c408c657e83b58c39e4bc766b4c391b7ee
Jul 22 02:28:39.542376 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:39.542333    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-x49pb" event={"ID":"da4bf1ec-30e6-4232-a356-f775c0049f2a","Type":"ContainerStarted","Data":"a124d493b4a7462c6b2ae566e3fe83c408c657e83b58c39e4bc766b4c391b7ee"}
Jul 22 02:28:40.550000 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:40.549962    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-t9zk2" event={"ID":"061682e8-bf94-4a04-9ff4-e877181535ad","Type":"ContainerStarted","Data":"f77854f0e59201205b767908add38d6bb5bd6a0c68d6976b75c7b18bacd1c724"}
Jul 22 02:28:40.550445 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:40.550074    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"
Jul 22 02:28:40.584196 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:40.584139    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-t9zk2" podStartSLOduration=2.005670639 podStartE2EDuration="4.584116586s" totalImagesPullingTime="2.578445947s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:28:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:28:40.582497388 +0000 UTC m=+542.123291016" watchObservedRunningTime="2026-07-22 02:28:40.584116586 +0000 UTC m=+542.124910230"
Jul 22 02:28:42.564024 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:42.563985    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-x49pb" event={"ID":"da4bf1ec-30e6-4232-a356-f775c0049f2a","Type":"ContainerStarted","Data":"73121417795f03252b0354c5600ed95e22be06d661bb1f895ac339191aea1d12"}
Jul 22 02:28:42.581075 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:42.581025    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-x49pb" podStartSLOduration=2.34711077 podStartE2EDuration="5.581009667s" totalImagesPullingTime="3.233898897s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:28:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:28:42.579181109 +0000 UTC m=+544.119974727" watchObservedRunningTime="2026-07-22 02:28:42.581009667 +0000 UTC m=+544.121803294"
Jul 22 02:28:50.153983 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:50.153947    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:28:50.208675 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:50.208637    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:28:51.556287 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:51.556256    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"
Jul 22 02:28:57.257715 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:57.257679    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:28:57.350107 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:57.350068    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:28:57.732404 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:28:57.732323    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:29:12.340090 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:12.340053    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:29:12.398843 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:12.398811    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:29:22.915530 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:22.915492    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:29:32.912084 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:32.912048    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:29:33.011834 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:33.011803    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:29:33.730691 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:33.730657    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-866d947867-wbw5n"]
Jul 22 02:29:33.734047 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:33.734030    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-866d947867-wbw5n"
Jul 22 02:29:33.736913 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:33.736894    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 22 02:29:33.745544 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:33.745521    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-866d947867-wbw5n"]
Jul 22 02:29:33.756092 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:33.756068    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwr4v\" (UniqueName: \"kubernetes.io/projected/260bdd76-f1ad-4b09-bf2b-fa30b9b7b8f8-kube-api-access-dwr4v\") pod \"authorino-866d947867-wbw5n\" (UID: \"260bdd76-f1ad-4b09-bf2b-fa30b9b7b8f8\") " pod="kuadrant-system/authorino-866d947867-wbw5n"
Jul 22 02:29:33.756188 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:33.756142    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/260bdd76-f1ad-4b09-bf2b-fa30b9b7b8f8-tls-cert\") pod \"authorino-866d947867-wbw5n\" (UID: \"260bdd76-f1ad-4b09-bf2b-fa30b9b7b8f8\") " pod="kuadrant-system/authorino-866d947867-wbw5n"
Jul 22 02:29:33.857050 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:33.857012    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwr4v\" (UniqueName: \"kubernetes.io/projected/260bdd76-f1ad-4b09-bf2b-fa30b9b7b8f8-kube-api-access-dwr4v\") pod \"authorino-866d947867-wbw5n\" (UID: \"260bdd76-f1ad-4b09-bf2b-fa30b9b7b8f8\") " pod="kuadrant-system/authorino-866d947867-wbw5n"
Jul 22 02:29:33.857219 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:33.857060    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/260bdd76-f1ad-4b09-bf2b-fa30b9b7b8f8-tls-cert\") pod \"authorino-866d947867-wbw5n\" (UID: \"260bdd76-f1ad-4b09-bf2b-fa30b9b7b8f8\") " pod="kuadrant-system/authorino-866d947867-wbw5n"
Jul 22 02:29:33.859479 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:33.859448    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/260bdd76-f1ad-4b09-bf2b-fa30b9b7b8f8-tls-cert\") pod \"authorino-866d947867-wbw5n\" (UID: \"260bdd76-f1ad-4b09-bf2b-fa30b9b7b8f8\") " pod="kuadrant-system/authorino-866d947867-wbw5n"
Jul 22 02:29:33.865933 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:33.865908    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwr4v\" (UniqueName: \"kubernetes.io/projected/260bdd76-f1ad-4b09-bf2b-fa30b9b7b8f8-kube-api-access-dwr4v\") pod \"authorino-866d947867-wbw5n\" (UID: \"260bdd76-f1ad-4b09-bf2b-fa30b9b7b8f8\") " pod="kuadrant-system/authorino-866d947867-wbw5n"
Jul 22 02:29:34.043671 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:34.043624    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-866d947867-wbw5n"
Jul 22 02:29:34.176485 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:34.176334    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-866d947867-wbw5n"]
Jul 22 02:29:34.179398 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:29:34.179360    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod260bdd76_f1ad_4b09_bf2b_fa30b9b7b8f8.slice/crio-afe34c25ccd01e1eb1bc4d6e9808f7b0ac7e5e4dee975211473ac4a7fb7b3bbb WatchSource:0}: Error finding container afe34c25ccd01e1eb1bc4d6e9808f7b0ac7e5e4dee975211473ac4a7fb7b3bbb: Status 404 returned error can't find the container with id afe34c25ccd01e1eb1bc4d6e9808f7b0ac7e5e4dee975211473ac4a7fb7b3bbb
Jul 22 02:29:34.811485 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:34.811444    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-866d947867-wbw5n" event={"ID":"260bdd76-f1ad-4b09-bf2b-fa30b9b7b8f8","Type":"ContainerStarted","Data":"138b101cad9d042eccd4693a41c021de32a88ceefd5ed93a2a45611fe685f4dc"}
Jul 22 02:29:34.811485 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:34.811494    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-866d947867-wbw5n" event={"ID":"260bdd76-f1ad-4b09-bf2b-fa30b9b7b8f8","Type":"ContainerStarted","Data":"afe34c25ccd01e1eb1bc4d6e9808f7b0ac7e5e4dee975211473ac4a7fb7b3bbb"}
Jul 22 02:29:34.855806 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:34.855674    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-866d947867-wbw5n" podStartSLOduration=1.4711971099999999 podStartE2EDuration="1.85565712s" totalImagesPullingTime="384.46001ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:29:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:29:34.828276341 +0000 UTC m=+596.369069968" watchObservedRunningTime="2026-07-22 02:29:34.85565712 +0000 UTC m=+596.396450747"
Jul 22 02:29:34.856552 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:34.856530    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-x49pb"]
Jul 22 02:29:34.856707 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:34.856687    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-x49pb" podUID="da4bf1ec-30e6-4232-a356-f775c0049f2a" containerName="authorino" containerID="cri-o://73121417795f03252b0354c5600ed95e22be06d661bb1f895ac339191aea1d12" gracePeriod=30
Jul 22 02:29:35.103218 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:35.103189    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-x49pb"
Jul 22 02:29:35.168168 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:35.168092    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbn6d\" (UniqueName: \"kubernetes.io/projected/da4bf1ec-30e6-4232-a356-f775c0049f2a-kube-api-access-cbn6d\") pod \"da4bf1ec-30e6-4232-a356-f775c0049f2a\" (UID: \"da4bf1ec-30e6-4232-a356-f775c0049f2a\") "
Jul 22 02:29:35.170294 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:35.170268    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da4bf1ec-30e6-4232-a356-f775c0049f2a-kube-api-access-cbn6d" (OuterVolumeSpecName: "kube-api-access-cbn6d") pod "da4bf1ec-30e6-4232-a356-f775c0049f2a" (UID: "da4bf1ec-30e6-4232-a356-f775c0049f2a"). InnerVolumeSpecName "kube-api-access-cbn6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 02:29:35.268845 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:35.268810    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbn6d\" (UniqueName: \"kubernetes.io/projected/da4bf1ec-30e6-4232-a356-f775c0049f2a-kube-api-access-cbn6d\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:29:35.818390 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:35.818346    2581 generic.go:356] "Generic (PLEG): container finished" podID="da4bf1ec-30e6-4232-a356-f775c0049f2a" containerID="73121417795f03252b0354c5600ed95e22be06d661bb1f895ac339191aea1d12" exitCode=0
Jul 22 02:29:35.818390 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:35.818376    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-x49pb" event={"ID":"da4bf1ec-30e6-4232-a356-f775c0049f2a","Type":"ContainerDied","Data":"73121417795f03252b0354c5600ed95e22be06d661bb1f895ac339191aea1d12"}
Jul 22 02:29:35.818604 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:35.818408    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-x49pb"
Jul 22 02:29:35.818604 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:35.818434    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-x49pb" event={"ID":"da4bf1ec-30e6-4232-a356-f775c0049f2a","Type":"ContainerDied","Data":"a124d493b4a7462c6b2ae566e3fe83c408c657e83b58c39e4bc766b4c391b7ee"}
Jul 22 02:29:35.818604 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:35.818456    2581 scope.go:117] "RemoveContainer" containerID="73121417795f03252b0354c5600ed95e22be06d661bb1f895ac339191aea1d12"
Jul 22 02:29:35.827148 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:35.827125    2581 scope.go:117] "RemoveContainer" containerID="73121417795f03252b0354c5600ed95e22be06d661bb1f895ac339191aea1d12"
Jul 22 02:29:35.827470 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:29:35.827448    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73121417795f03252b0354c5600ed95e22be06d661bb1f895ac339191aea1d12\": container with ID starting with 73121417795f03252b0354c5600ed95e22be06d661bb1f895ac339191aea1d12 not found: ID does not exist" containerID="73121417795f03252b0354c5600ed95e22be06d661bb1f895ac339191aea1d12"
Jul 22 02:29:35.827565 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:35.827483    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73121417795f03252b0354c5600ed95e22be06d661bb1f895ac339191aea1d12"} err="failed to get container status \"73121417795f03252b0354c5600ed95e22be06d661bb1f895ac339191aea1d12\": rpc error: code = NotFound desc = could not find container \"73121417795f03252b0354c5600ed95e22be06d661bb1f895ac339191aea1d12\": container with ID starting with 73121417795f03252b0354c5600ed95e22be06d661bb1f895ac339191aea1d12 not found: ID does not exist"
Jul 22 02:29:35.843770 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:35.843727    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-x49pb"]
Jul 22 02:29:35.845007 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:29:35.844986    2581 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podda4bf1ec_30e6_4232_a356_f775c0049f2a.slice\": RecentStats: unable to find data in memory cache]"
Jul 22 02:29:35.849110 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:35.849090    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-x49pb"]
Jul 22 02:29:36.903305 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:36.903270    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da4bf1ec-30e6-4232-a356-f775c0049f2a" path="/var/lib/kubelet/pods/da4bf1ec-30e6-4232-a356-f775c0049f2a/volumes"
Jul 22 02:29:39.806992 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:39.806960    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:29:39.914697 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:39.914662    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:29:40.017406 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:40.017371    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:29:40.089216 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:40.089141    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:29:42.992970 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:42.992931    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:29:44.109326 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:44.109289    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:29:49.162876 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:49.162842    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:29:54.702803 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:29:54.702767    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:30:16.034021 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:30:16.033943    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:30:16.122166 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:30:16.122131    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:30:16.549876 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:30:16.549842    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:30:16.800117 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:30:16.800035    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:30:17.048660 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:30:17.048629    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:30:17.192854 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:30:17.192769    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:30:17.302438 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:30:17.302390    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:30:26.960518 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:30:26.960484    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:30:27.084291 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:30:27.084257    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:30:37.847024 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:30:37.846988    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:30:37.944611 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:30:37.944580    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:30:41.101443 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:30:41.101411    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:30:41.773148 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:30:41.773109    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:31:02.904619 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:02.904584    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:31:03.360614 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:03.360579    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:31:10.285210 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:10.285174    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:31:11.567024 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:11.566983    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:31:19.092110 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:19.092074    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:31:20.550647 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:20.550613    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:31:21.923982 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:21.923948    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-cwmsw"]
Jul 22 02:31:21.924379 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:21.924309    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da4bf1ec-30e6-4232-a356-f775c0049f2a" containerName="authorino"
Jul 22 02:31:21.924379 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:21.924319    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="da4bf1ec-30e6-4232-a356-f775c0049f2a" containerName="authorino"
Jul 22 02:31:21.924464 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:21.924393    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="da4bf1ec-30e6-4232-a356-f775c0049f2a" containerName="authorino"
Jul 22 02:31:21.927770 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:21.927731    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwmsw"
Jul 22 02:31:21.934695 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:21.934666    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 22 02:31:21.934695 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:21.934675    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 22 02:31:21.934949 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:21.934829    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 22 02:31:21.934949 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:21.934834    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4mjch\""
Jul 22 02:31:21.956654 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:21.956621    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:31:21.979419 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:21.979387    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-cwmsw"]
Jul 22 02:31:22.061870 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:22.061837    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c67de9f-be55-4efa-8fd5-2f6d319e7cd3-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cwmsw\" (UID: \"7c67de9f-be55-4efa-8fd5-2f6d319e7cd3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwmsw"
Jul 22 02:31:22.062039 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:22.061881    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f49bc\" (UniqueName: \"kubernetes.io/projected/7c67de9f-be55-4efa-8fd5-2f6d319e7cd3-kube-api-access-f49bc\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cwmsw\" (UID: \"7c67de9f-be55-4efa-8fd5-2f6d319e7cd3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwmsw"
Jul 22 02:31:22.062224 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:22.062182    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c67de9f-be55-4efa-8fd5-2f6d319e7cd3-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cwmsw\" (UID: \"7c67de9f-be55-4efa-8fd5-2f6d319e7cd3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwmsw"
Jul 22 02:31:22.062224 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:22.062224    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c67de9f-be55-4efa-8fd5-2f6d319e7cd3-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cwmsw\" (UID: \"7c67de9f-be55-4efa-8fd5-2f6d319e7cd3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwmsw"
Jul 22 02:31:22.062422 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:22.062257    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c67de9f-be55-4efa-8fd5-2f6d319e7cd3-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cwmsw\" (UID: \"7c67de9f-be55-4efa-8fd5-2f6d319e7cd3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwmsw"
Jul 22 02:31:22.062422 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:22.062306    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c67de9f-be55-4efa-8fd5-2f6d319e7cd3-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cwmsw\" (UID: \"7c67de9f-be55-4efa-8fd5-2f6d319e7cd3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwmsw"
Jul 22 02:31:22.163134 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:22.163094    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f49bc\" (UniqueName: \"kubernetes.io/projected/7c67de9f-be55-4efa-8fd5-2f6d319e7cd3-kube-api-access-f49bc\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cwmsw\" (UID: \"7c67de9f-be55-4efa-8fd5-2f6d319e7cd3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwmsw"
Jul 22 02:31:22.163327 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:22.163194    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c67de9f-be55-4efa-8fd5-2f6d319e7cd3-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cwmsw\" (UID: \"7c67de9f-be55-4efa-8fd5-2f6d319e7cd3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwmsw"
Jul 22 02:31:22.163327 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:22.163221    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c67de9f-be55-4efa-8fd5-2f6d319e7cd3-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cwmsw\" (UID: \"7c67de9f-be55-4efa-8fd5-2f6d319e7cd3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwmsw"
Jul 22 02:31:22.163327 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:22.163256    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c67de9f-be55-4efa-8fd5-2f6d319e7cd3-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cwmsw\" (UID: \"7c67de9f-be55-4efa-8fd5-2f6d319e7cd3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwmsw"
Jul 22 02:31:22.163327 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:22.163278    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c67de9f-be55-4efa-8fd5-2f6d319e7cd3-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cwmsw\" (UID: \"7c67de9f-be55-4efa-8fd5-2f6d319e7cd3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwmsw"
Jul 22 02:31:22.163592 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:22.163355    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c67de9f-be55-4efa-8fd5-2f6d319e7cd3-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cwmsw\" (UID: \"7c67de9f-be55-4efa-8fd5-2f6d319e7cd3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwmsw"
Jul 22 02:31:22.163658 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:22.163631    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c67de9f-be55-4efa-8fd5-2f6d319e7cd3-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cwmsw\" (UID: \"7c67de9f-be55-4efa-8fd5-2f6d319e7cd3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwmsw"
Jul 22 02:31:22.163658 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:22.163646    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c67de9f-be55-4efa-8fd5-2f6d319e7cd3-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cwmsw\" (UID: \"7c67de9f-be55-4efa-8fd5-2f6d319e7cd3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwmsw"
Jul 22 02:31:22.163801 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:22.163714    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c67de9f-be55-4efa-8fd5-2f6d319e7cd3-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cwmsw\" (UID: \"7c67de9f-be55-4efa-8fd5-2f6d319e7cd3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwmsw"
Jul 22 02:31:22.165987 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:22.165962    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c67de9f-be55-4efa-8fd5-2f6d319e7cd3-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cwmsw\" (UID: \"7c67de9f-be55-4efa-8fd5-2f6d319e7cd3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwmsw"
Jul 22 02:31:22.166111 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:22.166100    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c67de9f-be55-4efa-8fd5-2f6d319e7cd3-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cwmsw\" (UID: \"7c67de9f-be55-4efa-8fd5-2f6d319e7cd3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwmsw"
Jul 22 02:31:22.191402 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:22.191335    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f49bc\" (UniqueName: \"kubernetes.io/projected/7c67de9f-be55-4efa-8fd5-2f6d319e7cd3-kube-api-access-f49bc\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cwmsw\" (UID: \"7c67de9f-be55-4efa-8fd5-2f6d319e7cd3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwmsw"
Jul 22 02:31:22.237917 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:22.237887    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwmsw"
Jul 22 02:31:22.380073 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:22.379891    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-cwmsw"]
Jul 22 02:31:22.382783 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:31:22.382753    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c67de9f_be55_4efa_8fd5_2f6d319e7cd3.slice/crio-de1f3ac0b0a6f285ba8f278bbc8cef538bec56be79250c499b6f91ef3acf6724 WatchSource:0}: Error finding container de1f3ac0b0a6f285ba8f278bbc8cef538bec56be79250c499b6f91ef3acf6724: Status 404 returned error can't find the container with id de1f3ac0b0a6f285ba8f278bbc8cef538bec56be79250c499b6f91ef3acf6724
Jul 22 02:31:22.384087 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:22.384064    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 02:31:23.332016 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:23.331985    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwmsw" event={"ID":"7c67de9f-be55-4efa-8fd5-2f6d319e7cd3","Type":"ContainerStarted","Data":"de1f3ac0b0a6f285ba8f278bbc8cef538bec56be79250c499b6f91ef3acf6724"}
Jul 22 02:31:24.338128 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:24.338091    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwmsw" event={"ID":"7c67de9f-be55-4efa-8fd5-2f6d319e7cd3","Type":"ContainerStarted","Data":"185691848c4c88c213f1c0649a097b8032691b967fcb639f3f816fa267330ae9"}
Jul 22 02:31:24.360245 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:24.360190    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwmsw" podStartSLOduration=2.364784061 podStartE2EDuration="3.360174025s" totalImagesPullingTime="995.389964ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:31:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:31:24.35873197 +0000 UTC m=+705.899525601" watchObservedRunningTime="2026-07-22 02:31:24.360174025 +0000 UTC m=+705.900967651"
Jul 22 02:31:30.141164 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:30.141131    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:31:32.238693 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:32.238608    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwmsw"
Jul 22 02:31:32.239098 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:32.238695    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwmsw"
Jul 22 02:31:32.251874 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:32.251844    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwmsw"
Jul 22 02:31:32.390866 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:32.390829    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwmsw"
Jul 22 02:31:52.459417 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:52.459384    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-77w66"]
Jul 22 02:31:52.463475 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:52.463452    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-77w66"
Jul 22 02:31:52.466880 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:52.466857    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 22 02:31:52.475116 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:52.475093    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-77w66"]
Jul 22 02:31:52.498554 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:52.498526    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:31:52.538373 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:52.538347    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6mwh\" (UniqueName: \"kubernetes.io/projected/c70213dc-ed59-4cb2-bcb7-8a62d938f24d-kube-api-access-z6mwh\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-77w66\" (UID: \"c70213dc-ed59-4cb2-bcb7-8a62d938f24d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-77w66"
Jul 22 02:31:52.538504 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:52.538391    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c70213dc-ed59-4cb2-bcb7-8a62d938f24d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-77w66\" (UID: \"c70213dc-ed59-4cb2-bcb7-8a62d938f24d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-77w66"
Jul 22 02:31:52.538504 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:52.538412    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c70213dc-ed59-4cb2-bcb7-8a62d938f24d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-77w66\" (UID: \"c70213dc-ed59-4cb2-bcb7-8a62d938f24d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-77w66"
Jul 22 02:31:52.538592 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:52.538510    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c70213dc-ed59-4cb2-bcb7-8a62d938f24d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-77w66\" (UID: \"c70213dc-ed59-4cb2-bcb7-8a62d938f24d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-77w66"
Jul 22 02:31:52.538592 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:52.538549    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c70213dc-ed59-4cb2-bcb7-8a62d938f24d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-77w66\" (UID: \"c70213dc-ed59-4cb2-bcb7-8a62d938f24d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-77w66"
Jul 22 02:31:52.538702 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:52.538680    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c70213dc-ed59-4cb2-bcb7-8a62d938f24d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-77w66\" (UID: \"c70213dc-ed59-4cb2-bcb7-8a62d938f24d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-77w66"
Jul 22 02:31:52.639701 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:52.639669    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c70213dc-ed59-4cb2-bcb7-8a62d938f24d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-77w66\" (UID: \"c70213dc-ed59-4cb2-bcb7-8a62d938f24d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-77w66"
Jul 22 02:31:52.639914 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:52.639709    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c70213dc-ed59-4cb2-bcb7-8a62d938f24d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-77w66\" (UID: \"c70213dc-ed59-4cb2-bcb7-8a62d938f24d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-77w66"
Jul 22 02:31:52.639997 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:52.639917    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c70213dc-ed59-4cb2-bcb7-8a62d938f24d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-77w66\" (UID: \"c70213dc-ed59-4cb2-bcb7-8a62d938f24d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-77w66"
Jul 22 02:31:52.639997 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:52.639938    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6mwh\" (UniqueName: \"kubernetes.io/projected/c70213dc-ed59-4cb2-bcb7-8a62d938f24d-kube-api-access-z6mwh\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-77w66\" (UID: \"c70213dc-ed59-4cb2-bcb7-8a62d938f24d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-77w66"
Jul 22 02:31:52.639997 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:52.639968    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c70213dc-ed59-4cb2-bcb7-8a62d938f24d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-77w66\" (UID: \"c70213dc-ed59-4cb2-bcb7-8a62d938f24d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-77w66"
Jul 22 02:31:52.639997 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:52.639988    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c70213dc-ed59-4cb2-bcb7-8a62d938f24d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-77w66\" (UID: \"c70213dc-ed59-4cb2-bcb7-8a62d938f24d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-77w66"
Jul 22 02:31:52.640206 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:52.640146    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c70213dc-ed59-4cb2-bcb7-8a62d938f24d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-77w66\" (UID: \"c70213dc-ed59-4cb2-bcb7-8a62d938f24d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-77w66"
Jul 22 02:31:52.640386 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:52.640358    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c70213dc-ed59-4cb2-bcb7-8a62d938f24d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-77w66\" (UID: \"c70213dc-ed59-4cb2-bcb7-8a62d938f24d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-77w66"
Jul 22 02:31:52.640386 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:52.640370    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c70213dc-ed59-4cb2-bcb7-8a62d938f24d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-77w66\" (UID: \"c70213dc-ed59-4cb2-bcb7-8a62d938f24d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-77w66"
Jul 22 02:31:52.642044 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:52.642023    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c70213dc-ed59-4cb2-bcb7-8a62d938f24d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-77w66\" (UID: \"c70213dc-ed59-4cb2-bcb7-8a62d938f24d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-77w66"
Jul 22 02:31:52.642184 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:52.642168    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c70213dc-ed59-4cb2-bcb7-8a62d938f24d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-77w66\" (UID: \"c70213dc-ed59-4cb2-bcb7-8a62d938f24d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-77w66"
Jul 22 02:31:52.654154 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:52.654128    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6mwh\" (UniqueName: \"kubernetes.io/projected/c70213dc-ed59-4cb2-bcb7-8a62d938f24d-kube-api-access-z6mwh\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-77w66\" (UID: \"c70213dc-ed59-4cb2-bcb7-8a62d938f24d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-77w66"
Jul 22 02:31:52.773834 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:52.773796    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-77w66"
Jul 22 02:31:52.913600 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:52.913576    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-77w66"]
Jul 22 02:31:52.915932 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:31:52.915904    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc70213dc_ed59_4cb2_bcb7_8a62d938f24d.slice/crio-af7590577e466b8a453b752acb018311c34ae9289b78c857c8101e7b45682408 WatchSource:0}: Error finding container af7590577e466b8a453b752acb018311c34ae9289b78c857c8101e7b45682408: Status 404 returned error can't find the container with id af7590577e466b8a453b752acb018311c34ae9289b78c857c8101e7b45682408
Jul 22 02:31:53.480896 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:53.480803    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-77w66" event={"ID":"c70213dc-ed59-4cb2-bcb7-8a62d938f24d","Type":"ContainerStarted","Data":"099285d9b5fe6080b953a9b78f8eb5d888db3c2c80495cb36b1d0558c9592665"}
Jul 22 02:31:53.480896 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:53.480850    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-77w66" event={"ID":"c70213dc-ed59-4cb2-bcb7-8a62d938f24d","Type":"ContainerStarted","Data":"af7590577e466b8a453b752acb018311c34ae9289b78c857c8101e7b45682408"}
Jul 22 02:31:53.506610 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:53.506560    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-77w66" podStartSLOduration=1.331007458 podStartE2EDuration="1.506544521s" totalImagesPullingTime="175.537063ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:31:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:31:53.505253275 +0000 UTC m=+735.046046904" watchObservedRunningTime="2026-07-22 02:31:53.506544521 +0000 UTC m=+735.047338148"
Jul 22 02:31:55.153779 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:31:55.153731    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:32:02.774817 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:32:02.774778    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-77w66"
Jul 22 02:32:02.774817 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:32:02.774827    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-77w66"
Jul 22 02:32:02.787151 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:32:02.787128    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-77w66"
Jul 22 02:32:03.540668 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:32:03.540642    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-77w66"
Jul 22 02:32:34.570828 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:32:34.570792    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:32:34.714020 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:32:34.713978    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:32:35.838356 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:32:35.838323    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:32:35.931826 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:32:35.931789    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:32:36.035389 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:32:36.035356    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-866d947867-wbw5n"]
Jul 22 02:32:36.035571 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:32:36.035550    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-866d947867-wbw5n" podUID="260bdd76-f1ad-4b09-bf2b-fa30b9b7b8f8" containerName="authorino" containerID="cri-o://138b101cad9d042eccd4693a41c021de32a88ceefd5ed93a2a45611fe685f4dc" gracePeriod=30
Jul 22 02:32:36.279028 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:32:36.279007    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-866d947867-wbw5n"
Jul 22 02:32:36.345228 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:32:36.345195    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dwr4v\" (UniqueName: \"kubernetes.io/projected/260bdd76-f1ad-4b09-bf2b-fa30b9b7b8f8-kube-api-access-dwr4v\") pod \"260bdd76-f1ad-4b09-bf2b-fa30b9b7b8f8\" (UID: \"260bdd76-f1ad-4b09-bf2b-fa30b9b7b8f8\") "
Jul 22 02:32:36.345385 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:32:36.345259    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/260bdd76-f1ad-4b09-bf2b-fa30b9b7b8f8-tls-cert\") pod \"260bdd76-f1ad-4b09-bf2b-fa30b9b7b8f8\" (UID: \"260bdd76-f1ad-4b09-bf2b-fa30b9b7b8f8\") "
Jul 22 02:32:36.347277 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:32:36.347235    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/260bdd76-f1ad-4b09-bf2b-fa30b9b7b8f8-kube-api-access-dwr4v" (OuterVolumeSpecName: "kube-api-access-dwr4v") pod "260bdd76-f1ad-4b09-bf2b-fa30b9b7b8f8" (UID: "260bdd76-f1ad-4b09-bf2b-fa30b9b7b8f8"). InnerVolumeSpecName "kube-api-access-dwr4v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 02:32:36.355312 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:32:36.355289    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/260bdd76-f1ad-4b09-bf2b-fa30b9b7b8f8-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "260bdd76-f1ad-4b09-bf2b-fa30b9b7b8f8" (UID: "260bdd76-f1ad-4b09-bf2b-fa30b9b7b8f8"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 02:32:36.447149 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:32:36.447062    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dwr4v\" (UniqueName: \"kubernetes.io/projected/260bdd76-f1ad-4b09-bf2b-fa30b9b7b8f8-kube-api-access-dwr4v\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:32:36.447149 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:32:36.447091    2581 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/260bdd76-f1ad-4b09-bf2b-fa30b9b7b8f8-tls-cert\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:32:36.694376 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:32:36.694341    2581 generic.go:356] "Generic (PLEG): container finished" podID="260bdd76-f1ad-4b09-bf2b-fa30b9b7b8f8" containerID="138b101cad9d042eccd4693a41c021de32a88ceefd5ed93a2a45611fe685f4dc" exitCode=0
Jul 22 02:32:36.694571 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:32:36.694385    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-866d947867-wbw5n"
Jul 22 02:32:36.694571 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:32:36.694395    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-866d947867-wbw5n" event={"ID":"260bdd76-f1ad-4b09-bf2b-fa30b9b7b8f8","Type":"ContainerDied","Data":"138b101cad9d042eccd4693a41c021de32a88ceefd5ed93a2a45611fe685f4dc"}
Jul 22 02:32:36.694571 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:32:36.694432    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-866d947867-wbw5n" event={"ID":"260bdd76-f1ad-4b09-bf2b-fa30b9b7b8f8","Type":"ContainerDied","Data":"afe34c25ccd01e1eb1bc4d6e9808f7b0ac7e5e4dee975211473ac4a7fb7b3bbb"}
Jul 22 02:32:36.694571 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:32:36.694452    2581 scope.go:117] "RemoveContainer" containerID="138b101cad9d042eccd4693a41c021de32a88ceefd5ed93a2a45611fe685f4dc"
Jul 22 02:32:36.704164 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:32:36.704143    2581 scope.go:117] "RemoveContainer" containerID="138b101cad9d042eccd4693a41c021de32a88ceefd5ed93a2a45611fe685f4dc"
Jul 22 02:32:36.704412 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:32:36.704394    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"138b101cad9d042eccd4693a41c021de32a88ceefd5ed93a2a45611fe685f4dc\": container with ID starting with 138b101cad9d042eccd4693a41c021de32a88ceefd5ed93a2a45611fe685f4dc not found: ID does not exist" containerID="138b101cad9d042eccd4693a41c021de32a88ceefd5ed93a2a45611fe685f4dc"
Jul 22 02:32:36.704474 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:32:36.704419    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"138b101cad9d042eccd4693a41c021de32a88ceefd5ed93a2a45611fe685f4dc"} err="failed to get container status \"138b101cad9d042eccd4693a41c021de32a88ceefd5ed93a2a45611fe685f4dc\": rpc error: code = NotFound desc = could not find container \"138b101cad9d042eccd4693a41c021de32a88ceefd5ed93a2a45611fe685f4dc\": container with ID starting with 138b101cad9d042eccd4693a41c021de32a88ceefd5ed93a2a45611fe685f4dc not found: ID does not exist"
Jul 22 02:32:36.752684 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:32:36.752642    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-866d947867-wbw5n"]
Jul 22 02:32:36.756115 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:32:36.756090    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-866d947867-wbw5n"]
Jul 22 02:32:36.905489 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:32:36.905446    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="260bdd76-f1ad-4b09-bf2b-fa30b9b7b8f8" path="/var/lib/kubelet/pods/260bdd76-f1ad-4b09-bf2b-fa30b9b7b8f8/volumes"
Jul 22 02:33:03.961281 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:33:03.961186    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:33:14.446382 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:33:14.446341    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:33:24.755615 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:33:24.755577    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:34:39.567938 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:34:39.567902    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:34:39.655980 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:34:39.655911    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:34:40.701086 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:34:40.701054    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:34:41.949785 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:34:41.949753    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:35:00.316582 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:35:00.316080    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:35:06.341131 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:35:06.341094    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:35:15.451861 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:35:15.451827    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:35:15.781196 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:35:15.781160    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:35:15.872435 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:35:15.872385    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:35:16.009088 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:35:16.009035    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:35:21.865638 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:35:21.865592    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:35:21.935016 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:35:21.934977    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:35:22.027661 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:35:22.027627    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:35:22.229492 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:35:22.229418    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:35:28.261163 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:35:28.261122    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:35:28.354873 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:35:28.354840    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:35:44.749477 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:35:44.749438    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:35:54.745225 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:35:54.745188    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:36:03.647391 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:36:03.647307    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:36:15.149012 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:36:15.148972    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:36:26.647561 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:36:26.647522    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:36:37.963176 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:36:37.963137    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:36:38.751998 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:36:38.751963    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:36:50.238029 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:36:50.237995    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:37:00.435430 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:37:00.435389    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:37:17.039750 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:37:17.039708    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:37:22.250591 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:37:22.250560    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:37:31.342556 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:37:31.342466    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:37:55.744091 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:37:55.744056    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:38:06.161901 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:38:06.161869    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:38:06.231884 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:38:06.231846    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:38:06.333207 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:38:06.333170    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:38:24.447276 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:38:24.447236    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:38:33.852091 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:38:33.852046    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:38:43.856997 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:38:43.856959    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:38:53.845761 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:38:53.845703    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:39:02.953061 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:39:02.952977    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:40:51.352054 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:40:51.352015    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:41:02.745642 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:41:02.745607    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:41:12.043065 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:41:12.043028    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:41:23.865588 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:41:23.865546    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:41:34.968005 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:41:34.967965    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:41:46.272894 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:41:46.272858    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:41:53.958338 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:41:53.958302    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:42:05.950127 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:42:05.950048    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:42:16.877218 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:42:16.877141    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:42:25.762355 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:42:25.762308    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:42:36.451389 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:42:36.451350    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:42:57.970119 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:42:57.970082    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:43:02.056993 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:02.056958    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:43:07.772966 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:07.772931    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:43:10.281026 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:10.280993    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm"]
Jul 22 02:43:10.281404 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:10.281376    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="260bdd76-f1ad-4b09-bf2b-fa30b9b7b8f8" containerName="authorino"
Jul 22 02:43:10.281404 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:10.281387    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="260bdd76-f1ad-4b09-bf2b-fa30b9b7b8f8" containerName="authorino"
Jul 22 02:43:10.281475 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:10.281470    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="260bdd76-f1ad-4b09-bf2b-fa30b9b7b8f8" containerName="authorino"
Jul 22 02:43:10.284495 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:10.284478    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm"
Jul 22 02:43:10.289215 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:10.289194    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-73909a99-kserve-self-signed-certs\""
Jul 22 02:43:10.301663 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:10.301636    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:43:10.330420 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:10.330390    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm"]
Jul 22 02:43:10.352719 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:10.352686    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71065006-9b5f-4323-9178-7358648da265-dshm\") pod \"e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm\" (UID: \"71065006-9b5f-4323-9178-7358648da265\") " pod="llm/e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm"
Jul 22 02:43:10.352897 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:10.352731    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5z86s\" (UniqueName: \"kubernetes.io/projected/71065006-9b5f-4323-9178-7358648da265-kube-api-access-5z86s\") pod \"e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm\" (UID: \"71065006-9b5f-4323-9178-7358648da265\") " pod="llm/e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm"
Jul 22 02:43:10.352897 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:10.352854    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71065006-9b5f-4323-9178-7358648da265-tls-certs\") pod \"e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm\" (UID: \"71065006-9b5f-4323-9178-7358648da265\") " pod="llm/e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm"
Jul 22 02:43:10.353006 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:10.352915    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71065006-9b5f-4323-9178-7358648da265-model-cache\") pod \"e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm\" (UID: \"71065006-9b5f-4323-9178-7358648da265\") " pod="llm/e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm"
Jul 22 02:43:10.353006 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:10.352994    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71065006-9b5f-4323-9178-7358648da265-home\") pod \"e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm\" (UID: \"71065006-9b5f-4323-9178-7358648da265\") " pod="llm/e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm"
Jul 22 02:43:10.353086 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:10.353058    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71065006-9b5f-4323-9178-7358648da265-tmp-dir\") pod \"e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm\" (UID: \"71065006-9b5f-4323-9178-7358648da265\") " pod="llm/e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm"
Jul 22 02:43:10.453566 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:10.453531    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71065006-9b5f-4323-9178-7358648da265-model-cache\") pod \"e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm\" (UID: \"71065006-9b5f-4323-9178-7358648da265\") " pod="llm/e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm"
Jul 22 02:43:10.453791 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:10.453584    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71065006-9b5f-4323-9178-7358648da265-home\") pod \"e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm\" (UID: \"71065006-9b5f-4323-9178-7358648da265\") " pod="llm/e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm"
Jul 22 02:43:10.453791 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:10.453619    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71065006-9b5f-4323-9178-7358648da265-tmp-dir\") pod \"e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm\" (UID: \"71065006-9b5f-4323-9178-7358648da265\") " pod="llm/e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm"
Jul 22 02:43:10.453791 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:10.453645    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71065006-9b5f-4323-9178-7358648da265-dshm\") pod \"e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm\" (UID: \"71065006-9b5f-4323-9178-7358648da265\") " pod="llm/e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm"
Jul 22 02:43:10.453791 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:10.453674    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5z86s\" (UniqueName: \"kubernetes.io/projected/71065006-9b5f-4323-9178-7358648da265-kube-api-access-5z86s\") pod \"e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm\" (UID: \"71065006-9b5f-4323-9178-7358648da265\") " pod="llm/e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm"
Jul 22 02:43:10.453791 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:10.453708    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71065006-9b5f-4323-9178-7358648da265-tls-certs\") pod \"e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm\" (UID: \"71065006-9b5f-4323-9178-7358648da265\") " pod="llm/e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm"
Jul 22 02:43:10.454084 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:10.453984    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71065006-9b5f-4323-9178-7358648da265-model-cache\") pod \"e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm\" (UID: \"71065006-9b5f-4323-9178-7358648da265\") " pod="llm/e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm"
Jul 22 02:43:10.454084 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:10.454053    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71065006-9b5f-4323-9178-7358648da265-tmp-dir\") pod \"e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm\" (UID: \"71065006-9b5f-4323-9178-7358648da265\") " pod="llm/e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm"
Jul 22 02:43:10.454160 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:10.454112    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71065006-9b5f-4323-9178-7358648da265-home\") pod \"e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm\" (UID: \"71065006-9b5f-4323-9178-7358648da265\") " pod="llm/e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm"
Jul 22 02:43:10.456026 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:10.455991    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71065006-9b5f-4323-9178-7358648da265-dshm\") pod \"e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm\" (UID: \"71065006-9b5f-4323-9178-7358648da265\") " pod="llm/e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm"
Jul 22 02:43:10.456272 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:10.456255    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71065006-9b5f-4323-9178-7358648da265-tls-certs\") pod \"e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm\" (UID: \"71065006-9b5f-4323-9178-7358648da265\") " pod="llm/e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm"
Jul 22 02:43:10.462493 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:10.462470    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5z86s\" (UniqueName: \"kubernetes.io/projected/71065006-9b5f-4323-9178-7358648da265-kube-api-access-5z86s\") pod \"e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm\" (UID: \"71065006-9b5f-4323-9178-7358648da265\") " pod="llm/e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm"
Jul 22 02:43:10.593774 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:10.593658    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm"
Jul 22 02:43:10.733602 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:10.733577    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm"]
Jul 22 02:43:10.735685 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:43:10.735654    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod71065006_9b5f_4323_9178_7358648da265.slice/crio-d5daaf76bd3264f6f70ba14bfa346e9e7ed0a60ed4f699ddc048a86acde8dc88 WatchSource:0}: Error finding container d5daaf76bd3264f6f70ba14bfa346e9e7ed0a60ed4f699ddc048a86acde8dc88: Status 404 returned error can't find the container with id d5daaf76bd3264f6f70ba14bfa346e9e7ed0a60ed4f699ddc048a86acde8dc88
Jul 22 02:43:10.755686 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:10.755648    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm" event={"ID":"71065006-9b5f-4323-9178-7358648da265","Type":"ContainerStarted","Data":"d5daaf76bd3264f6f70ba14bfa346e9e7ed0a60ed4f699ddc048a86acde8dc88"}
Jul 22 02:43:11.764671 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:11.764636    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm" event={"ID":"71065006-9b5f-4323-9178-7358648da265","Type":"ContainerStarted","Data":"c64243bf985684a538f5e3ee810e3176b2d1c93aa02b8723256c354fe471c885"}
Jul 22 02:43:11.794623 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:11.794576    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm" podStartSLOduration=1.794562396 podStartE2EDuration="1.794562396s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:43:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:43:11.793568348 +0000 UTC m=+1413.334361974" watchObservedRunningTime="2026-07-22 02:43:11.794562396 +0000 UTC m=+1413.335356023"
Jul 22 02:43:17.120127 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:17.120093    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j"]
Jul 22 02:43:17.123730 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:17.123706    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j"
Jul 22 02:43:17.128208 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:17.128189    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-73909a99-kserve-self-signed-certs\""
Jul 22 02:43:17.147556 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:17.147521    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j"]
Jul 22 02:43:17.216245 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:17.216204    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d679c7d-825d-4c8f-8476-1664c93931b3-model-cache\") pod \"e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j\" (UID: \"7d679c7d-825d-4c8f-8476-1664c93931b3\") " pod="llm/e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j"
Jul 22 02:43:17.216245 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:17.216247    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d679c7d-825d-4c8f-8476-1664c93931b3-tls-certs\") pod \"e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j\" (UID: \"7d679c7d-825d-4c8f-8476-1664c93931b3\") " pod="llm/e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j"
Jul 22 02:43:17.216453 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:17.216276    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d679c7d-825d-4c8f-8476-1664c93931b3-dshm\") pod \"e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j\" (UID: \"7d679c7d-825d-4c8f-8476-1664c93931b3\") " pod="llm/e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j"
Jul 22 02:43:17.216453 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:17.216408    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnc8w\" (UniqueName: \"kubernetes.io/projected/7d679c7d-825d-4c8f-8476-1664c93931b3-kube-api-access-xnc8w\") pod \"e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j\" (UID: \"7d679c7d-825d-4c8f-8476-1664c93931b3\") " pod="llm/e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j"
Jul 22 02:43:17.216524 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:17.216453    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d679c7d-825d-4c8f-8476-1664c93931b3-home\") pod \"e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j\" (UID: \"7d679c7d-825d-4c8f-8476-1664c93931b3\") " pod="llm/e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j"
Jul 22 02:43:17.216524 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:17.216479    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d679c7d-825d-4c8f-8476-1664c93931b3-tmp-dir\") pod \"e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j\" (UID: \"7d679c7d-825d-4c8f-8476-1664c93931b3\") " pod="llm/e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j"
Jul 22 02:43:17.251885 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:17.251844    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:43:17.317238 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:17.317198    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d679c7d-825d-4c8f-8476-1664c93931b3-home\") pod \"e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j\" (UID: \"7d679c7d-825d-4c8f-8476-1664c93931b3\") " pod="llm/e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j"
Jul 22 02:43:17.317238 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:17.317238    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d679c7d-825d-4c8f-8476-1664c93931b3-tmp-dir\") pod \"e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j\" (UID: \"7d679c7d-825d-4c8f-8476-1664c93931b3\") " pod="llm/e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j"
Jul 22 02:43:17.317490 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:17.317278    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d679c7d-825d-4c8f-8476-1664c93931b3-model-cache\") pod \"e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j\" (UID: \"7d679c7d-825d-4c8f-8476-1664c93931b3\") " pod="llm/e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j"
Jul 22 02:43:17.317490 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:17.317299    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d679c7d-825d-4c8f-8476-1664c93931b3-tls-certs\") pod \"e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j\" (UID: \"7d679c7d-825d-4c8f-8476-1664c93931b3\") " pod="llm/e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j"
Jul 22 02:43:17.317490 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:17.317322    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d679c7d-825d-4c8f-8476-1664c93931b3-dshm\") pod \"e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j\" (UID: \"7d679c7d-825d-4c8f-8476-1664c93931b3\") " pod="llm/e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j"
Jul 22 02:43:17.317490 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:17.317368    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnc8w\" (UniqueName: \"kubernetes.io/projected/7d679c7d-825d-4c8f-8476-1664c93931b3-kube-api-access-xnc8w\") pod \"e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j\" (UID: \"7d679c7d-825d-4c8f-8476-1664c93931b3\") " pod="llm/e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j"
Jul 22 02:43:17.317718 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:17.317659    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d679c7d-825d-4c8f-8476-1664c93931b3-home\") pod \"e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j\" (UID: \"7d679c7d-825d-4c8f-8476-1664c93931b3\") " pod="llm/e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j"
Jul 22 02:43:17.317718 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:17.317671    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d679c7d-825d-4c8f-8476-1664c93931b3-model-cache\") pod \"e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j\" (UID: \"7d679c7d-825d-4c8f-8476-1664c93931b3\") " pod="llm/e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j"
Jul 22 02:43:17.317718 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:17.317700    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d679c7d-825d-4c8f-8476-1664c93931b3-tmp-dir\") pod \"e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j\" (UID: \"7d679c7d-825d-4c8f-8476-1664c93931b3\") " pod="llm/e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j"
Jul 22 02:43:17.319692 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:17.319670    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d679c7d-825d-4c8f-8476-1664c93931b3-dshm\") pod \"e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j\" (UID: \"7d679c7d-825d-4c8f-8476-1664c93931b3\") " pod="llm/e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j"
Jul 22 02:43:17.319937 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:17.319917    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d679c7d-825d-4c8f-8476-1664c93931b3-tls-certs\") pod \"e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j\" (UID: \"7d679c7d-825d-4c8f-8476-1664c93931b3\") " pod="llm/e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j"
Jul 22 02:43:17.331462 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:17.331438    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnc8w\" (UniqueName: \"kubernetes.io/projected/7d679c7d-825d-4c8f-8476-1664c93931b3-kube-api-access-xnc8w\") pod \"e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j\" (UID: \"7d679c7d-825d-4c8f-8476-1664c93931b3\") " pod="llm/e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j"
Jul 22 02:43:17.435206 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:17.435111    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j"
Jul 22 02:43:17.585578 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:17.585555    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j"]
Jul 22 02:43:17.588109 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:43:17.588077    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7d679c7d_825d_4c8f_8476_1664c93931b3.slice/crio-424b9fbf1da15bc16c8174a24f210029dde75e7c8b09941ad4ad438f1674c5f7 WatchSource:0}: Error finding container 424b9fbf1da15bc16c8174a24f210029dde75e7c8b09941ad4ad438f1674c5f7: Status 404 returned error can't find the container with id 424b9fbf1da15bc16c8174a24f210029dde75e7c8b09941ad4ad438f1674c5f7
Jul 22 02:43:17.798909 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:17.798869    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j" event={"ID":"7d679c7d-825d-4c8f-8476-1664c93931b3","Type":"ContainerStarted","Data":"9e1742994eae095c5f9ca0ae93d68159a20b8a6d99df83578cffac23de571055"}
Jul 22 02:43:17.798909 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:17.798914    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j" event={"ID":"7d679c7d-825d-4c8f-8476-1664c93931b3","Type":"ContainerStarted","Data":"424b9fbf1da15bc16c8174a24f210029dde75e7c8b09941ad4ad438f1674c5f7"}
Jul 22 02:43:17.853413 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:17.853361    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j" podStartSLOduration=0.853344678 podStartE2EDuration="853.344678ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:43:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:43:17.852084769 +0000 UTC m=+1419.392878397" watchObservedRunningTime="2026-07-22 02:43:17.853344678 +0000 UTC m=+1419.394138356"
Jul 22 02:43:20.594219 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:20.594183    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm"
Jul 22 02:43:20.594656 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:20.594270    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm"
Jul 22 02:43:20.606362 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:20.606339    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm"
Jul 22 02:43:20.825898 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:20.825867    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm"
Jul 22 02:43:23.556023 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:23.555992    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:43:27.435749 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:27.435704    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j"
Jul 22 02:43:27.436132 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:27.435776    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j"
Jul 22 02:43:27.448605 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:27.448580    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j"
Jul 22 02:43:27.861027 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:27.860997    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j"
Jul 22 02:43:40.516165 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:40.516127    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm"]
Jul 22 02:43:40.572642 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:40.572606    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:43:41.619723 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:41.619687    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j"]
Jul 22 02:43:41.653190 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:41.653155    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:43:44.551535 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:44.551499    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:43:55.536393 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:55.536341    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm" podUID="71065006-9b5f-4323-9178-7358648da265" containerName="main" containerID="cri-o://c64243bf985684a538f5e3ee810e3176b2d1c93aa02b8723256c354fe471c885" gracePeriod=45
Jul 22 02:43:55.784562 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:55.784535    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm"
Jul 22 02:43:55.880731 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:55.880641    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71065006-9b5f-4323-9178-7358648da265-tls-certs\") pod \"71065006-9b5f-4323-9178-7358648da265\" (UID: \"71065006-9b5f-4323-9178-7358648da265\") "
Jul 22 02:43:55.880731 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:55.880709    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71065006-9b5f-4323-9178-7358648da265-home\") pod \"71065006-9b5f-4323-9178-7358648da265\" (UID: \"71065006-9b5f-4323-9178-7358648da265\") "
Jul 22 02:43:55.880989 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:55.880783    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71065006-9b5f-4323-9178-7358648da265-dshm\") pod \"71065006-9b5f-4323-9178-7358648da265\" (UID: \"71065006-9b5f-4323-9178-7358648da265\") "
Jul 22 02:43:55.880989 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:55.880833    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71065006-9b5f-4323-9178-7358648da265-tmp-dir\") pod \"71065006-9b5f-4323-9178-7358648da265\" (UID: \"71065006-9b5f-4323-9178-7358648da265\") "
Jul 22 02:43:55.880989 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:55.880868    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5z86s\" (UniqueName: \"kubernetes.io/projected/71065006-9b5f-4323-9178-7358648da265-kube-api-access-5z86s\") pod \"71065006-9b5f-4323-9178-7358648da265\" (UID: \"71065006-9b5f-4323-9178-7358648da265\") "
Jul 22 02:43:55.880989 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:55.880947    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71065006-9b5f-4323-9178-7358648da265-model-cache\") pod \"71065006-9b5f-4323-9178-7358648da265\" (UID: \"71065006-9b5f-4323-9178-7358648da265\") "
Jul 22 02:43:55.881200 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:55.880996    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71065006-9b5f-4323-9178-7358648da265-home" (OuterVolumeSpecName: "home") pod "71065006-9b5f-4323-9178-7358648da265" (UID: "71065006-9b5f-4323-9178-7358648da265"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:43:55.881200 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:55.881079    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71065006-9b5f-4323-9178-7358648da265-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "71065006-9b5f-4323-9178-7358648da265" (UID: "71065006-9b5f-4323-9178-7358648da265"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:43:55.881306 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:55.881284    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71065006-9b5f-4323-9178-7358648da265-model-cache" (OuterVolumeSpecName: "model-cache") pod "71065006-9b5f-4323-9178-7358648da265" (UID: "71065006-9b5f-4323-9178-7358648da265"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:43:55.881375 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:55.881355    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71065006-9b5f-4323-9178-7358648da265-home\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:43:55.881433 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:55.881382    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71065006-9b5f-4323-9178-7358648da265-tmp-dir\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:43:55.883131 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:55.883105    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71065006-9b5f-4323-9178-7358648da265-dshm" (OuterVolumeSpecName: "dshm") pod "71065006-9b5f-4323-9178-7358648da265" (UID: "71065006-9b5f-4323-9178-7358648da265"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:43:55.883131 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:55.883114    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71065006-9b5f-4323-9178-7358648da265-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "71065006-9b5f-4323-9178-7358648da265" (UID: "71065006-9b5f-4323-9178-7358648da265"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 02:43:55.883289 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:55.883129    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71065006-9b5f-4323-9178-7358648da265-kube-api-access-5z86s" (OuterVolumeSpecName: "kube-api-access-5z86s") pod "71065006-9b5f-4323-9178-7358648da265" (UID: "71065006-9b5f-4323-9178-7358648da265"). InnerVolumeSpecName "kube-api-access-5z86s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 02:43:55.982540 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:55.982464    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71065006-9b5f-4323-9178-7358648da265-model-cache\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:43:55.982540 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:55.982497    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71065006-9b5f-4323-9178-7358648da265-tls-certs\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:43:55.982540 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:55.982507    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71065006-9b5f-4323-9178-7358648da265-dshm\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:43:55.982540 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:55.982518    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5z86s\" (UniqueName: \"kubernetes.io/projected/71065006-9b5f-4323-9178-7358648da265-kube-api-access-5z86s\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:43:55.983629 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:55.983597    2581 generic.go:356] "Generic (PLEG): container finished" podID="71065006-9b5f-4323-9178-7358648da265" containerID="c64243bf985684a538f5e3ee810e3176b2d1c93aa02b8723256c354fe471c885" exitCode=0
Jul 22 02:43:55.983766 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:55.983642    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm"
Jul 22 02:43:55.983766 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:55.983651    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm" event={"ID":"71065006-9b5f-4323-9178-7358648da265","Type":"ContainerDied","Data":"c64243bf985684a538f5e3ee810e3176b2d1c93aa02b8723256c354fe471c885"}
Jul 22 02:43:55.983766 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:55.983683    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm" event={"ID":"71065006-9b5f-4323-9178-7358648da265","Type":"ContainerDied","Data":"d5daaf76bd3264f6f70ba14bfa346e9e7ed0a60ed4f699ddc048a86acde8dc88"}
Jul 22 02:43:55.983766 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:55.983700    2581 scope.go:117] "RemoveContainer" containerID="c64243bf985684a538f5e3ee810e3176b2d1c93aa02b8723256c354fe471c885"
Jul 22 02:43:55.993174 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:55.993155    2581 scope.go:117] "RemoveContainer" containerID="c64243bf985684a538f5e3ee810e3176b2d1c93aa02b8723256c354fe471c885"
Jul 22 02:43:55.993441 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:43:55.993421    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c64243bf985684a538f5e3ee810e3176b2d1c93aa02b8723256c354fe471c885\": container with ID starting with c64243bf985684a538f5e3ee810e3176b2d1c93aa02b8723256c354fe471c885 not found: ID does not exist" containerID="c64243bf985684a538f5e3ee810e3176b2d1c93aa02b8723256c354fe471c885"
Jul 22 02:43:55.993487 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:55.993451    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c64243bf985684a538f5e3ee810e3176b2d1c93aa02b8723256c354fe471c885"} err="failed to get container status \"c64243bf985684a538f5e3ee810e3176b2d1c93aa02b8723256c354fe471c885\": rpc error: code = NotFound desc = could not find container \"c64243bf985684a538f5e3ee810e3176b2d1c93aa02b8723256c354fe471c885\": container with ID starting with c64243bf985684a538f5e3ee810e3176b2d1c93aa02b8723256c354fe471c885 not found: ID does not exist"
Jul 22 02:43:56.029265 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:56.029235    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm"]
Jul 22 02:43:56.051908 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:56.051877    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-73909a99-kserve-86498989b7-rkcnm"]
Jul 22 02:43:56.639705 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:56.639663    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j" podUID="7d679c7d-825d-4c8f-8476-1664c93931b3" containerName="main" containerID="cri-o://9e1742994eae095c5f9ca0ae93d68159a20b8a6d99df83578cffac23de571055" gracePeriod=45
Jul 22 02:43:56.892419 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:56.892361    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j"
Jul 22 02:43:56.903104 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:56.903077    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71065006-9b5f-4323-9178-7358648da265" path="/var/lib/kubelet/pods/71065006-9b5f-4323-9178-7358648da265/volumes"
Jul 22 02:43:56.990817 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:56.990779    2581 generic.go:356] "Generic (PLEG): container finished" podID="7d679c7d-825d-4c8f-8476-1664c93931b3" containerID="9e1742994eae095c5f9ca0ae93d68159a20b8a6d99df83578cffac23de571055" exitCode=0
Jul 22 02:43:56.990996 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:56.990827    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j"
Jul 22 02:43:56.990996 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:56.990863    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j" event={"ID":"7d679c7d-825d-4c8f-8476-1664c93931b3","Type":"ContainerDied","Data":"9e1742994eae095c5f9ca0ae93d68159a20b8a6d99df83578cffac23de571055"}
Jul 22 02:43:56.990996 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:56.990909    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j" event={"ID":"7d679c7d-825d-4c8f-8476-1664c93931b3","Type":"ContainerDied","Data":"424b9fbf1da15bc16c8174a24f210029dde75e7c8b09941ad4ad438f1674c5f7"}
Jul 22 02:43:56.990996 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:56.990926    2581 scope.go:117] "RemoveContainer" containerID="9e1742994eae095c5f9ca0ae93d68159a20b8a6d99df83578cffac23de571055"
Jul 22 02:43:56.991669 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:56.991648    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d679c7d-825d-4c8f-8476-1664c93931b3-tls-certs\") pod \"7d679c7d-825d-4c8f-8476-1664c93931b3\" (UID: \"7d679c7d-825d-4c8f-8476-1664c93931b3\") "
Jul 22 02:43:56.991801 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:56.991763    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xnc8w\" (UniqueName: \"kubernetes.io/projected/7d679c7d-825d-4c8f-8476-1664c93931b3-kube-api-access-xnc8w\") pod \"7d679c7d-825d-4c8f-8476-1664c93931b3\" (UID: \"7d679c7d-825d-4c8f-8476-1664c93931b3\") "
Jul 22 02:43:56.991942 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:56.991853    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d679c7d-825d-4c8f-8476-1664c93931b3-dshm\") pod \"7d679c7d-825d-4c8f-8476-1664c93931b3\" (UID: \"7d679c7d-825d-4c8f-8476-1664c93931b3\") "
Jul 22 02:43:56.991942 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:56.991899    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d679c7d-825d-4c8f-8476-1664c93931b3-tmp-dir\") pod \"7d679c7d-825d-4c8f-8476-1664c93931b3\" (UID: \"7d679c7d-825d-4c8f-8476-1664c93931b3\") "
Jul 22 02:43:56.992053 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:56.991967    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d679c7d-825d-4c8f-8476-1664c93931b3-model-cache\") pod \"7d679c7d-825d-4c8f-8476-1664c93931b3\" (UID: \"7d679c7d-825d-4c8f-8476-1664c93931b3\") "
Jul 22 02:43:56.992053 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:56.992002    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d679c7d-825d-4c8f-8476-1664c93931b3-home\") pod \"7d679c7d-825d-4c8f-8476-1664c93931b3\" (UID: \"7d679c7d-825d-4c8f-8476-1664c93931b3\") "
Jul 22 02:43:56.992241 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:56.992156    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d679c7d-825d-4c8f-8476-1664c93931b3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7d679c7d-825d-4c8f-8476-1664c93931b3" (UID: "7d679c7d-825d-4c8f-8476-1664c93931b3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:43:56.992381 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:56.992256    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d679c7d-825d-4c8f-8476-1664c93931b3-model-cache" (OuterVolumeSpecName: "model-cache") pod "7d679c7d-825d-4c8f-8476-1664c93931b3" (UID: "7d679c7d-825d-4c8f-8476-1664c93931b3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:43:56.992381 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:56.992299    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d679c7d-825d-4c8f-8476-1664c93931b3-home" (OuterVolumeSpecName: "home") pod "7d679c7d-825d-4c8f-8476-1664c93931b3" (UID: "7d679c7d-825d-4c8f-8476-1664c93931b3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:43:56.992538 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:56.992515    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d679c7d-825d-4c8f-8476-1664c93931b3-tmp-dir\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:43:56.992637 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:56.992542    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d679c7d-825d-4c8f-8476-1664c93931b3-model-cache\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:43:56.992637 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:56.992554    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d679c7d-825d-4c8f-8476-1664c93931b3-home\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:43:56.994025 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:56.993964    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d679c7d-825d-4c8f-8476-1664c93931b3-kube-api-access-xnc8w" (OuterVolumeSpecName: "kube-api-access-xnc8w") pod "7d679c7d-825d-4c8f-8476-1664c93931b3" (UID: "7d679c7d-825d-4c8f-8476-1664c93931b3"). InnerVolumeSpecName "kube-api-access-xnc8w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 02:43:56.994370 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:56.994330    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d679c7d-825d-4c8f-8476-1664c93931b3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7d679c7d-825d-4c8f-8476-1664c93931b3" (UID: "7d679c7d-825d-4c8f-8476-1664c93931b3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 02:43:56.994454 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:56.994365    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d679c7d-825d-4c8f-8476-1664c93931b3-dshm" (OuterVolumeSpecName: "dshm") pod "7d679c7d-825d-4c8f-8476-1664c93931b3" (UID: "7d679c7d-825d-4c8f-8476-1664c93931b3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:43:57.012362 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:57.012339    2581 scope.go:117] "RemoveContainer" containerID="9e1742994eae095c5f9ca0ae93d68159a20b8a6d99df83578cffac23de571055"
Jul 22 02:43:57.012647 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:43:57.012626    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e1742994eae095c5f9ca0ae93d68159a20b8a6d99df83578cffac23de571055\": container with ID starting with 9e1742994eae095c5f9ca0ae93d68159a20b8a6d99df83578cffac23de571055 not found: ID does not exist" containerID="9e1742994eae095c5f9ca0ae93d68159a20b8a6d99df83578cffac23de571055"
Jul 22 02:43:57.012709 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:57.012656    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e1742994eae095c5f9ca0ae93d68159a20b8a6d99df83578cffac23de571055"} err="failed to get container status \"9e1742994eae095c5f9ca0ae93d68159a20b8a6d99df83578cffac23de571055\": rpc error: code = NotFound desc = could not find container \"9e1742994eae095c5f9ca0ae93d68159a20b8a6d99df83578cffac23de571055\": container with ID starting with 9e1742994eae095c5f9ca0ae93d68159a20b8a6d99df83578cffac23de571055 not found: ID does not exist"
Jul 22 02:43:57.093375 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:57.093338    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnc8w\" (UniqueName: \"kubernetes.io/projected/7d679c7d-825d-4c8f-8476-1664c93931b3-kube-api-access-xnc8w\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:43:57.093375 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:57.093367    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d679c7d-825d-4c8f-8476-1664c93931b3-dshm\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:43:57.093375 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:57.093377    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d679c7d-825d-4c8f-8476-1664c93931b3-tls-certs\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:43:57.314646 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:57.314608    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j"]
Jul 22 02:43:57.325843 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:57.325816    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-73909a99-kserve-5f449b86dc-jhh8j"]
Jul 22 02:43:58.903861 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:58.903829    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d679c7d-825d-4c8f-8476-1664c93931b3" path="/var/lib/kubelet/pods/7d679c7d-825d-4c8f-8476-1664c93931b3/volumes"
Jul 22 02:43:59.454825 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:43:59.454788    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:44:01.570683 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:44:01.570648    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:44:09.602931 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:44:09.602896    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:44:22.553103 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:44:22.553060    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:44:23.671419 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:44:23.671380    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:44:23.772087 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:44:23.772052    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:44:32.261663 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:44:32.261622    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:44:42.358221 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:44:42.358182    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:44:52.548077 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:44:52.548036    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:45:02.049473 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:45:02.049385    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:45:11.266771 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:45:11.266720    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:45:25.345846 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:45:25.345807    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:45:34.854120 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:45:34.854082    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:45:50.646565 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:45:50.646521    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:45:58.574106 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:45:58.574066    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:46:11.258281 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:46:11.258244    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:46:27.154377 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:46:27.154342    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:46:39.337634 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:46:39.337603    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:46:51.040584 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:46:51.040547    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:46:53.449125 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:46:53.449081    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:47:04.440775 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:47:04.440723    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:47:14.754591 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:47:14.754536    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:47:17.168720 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:47:17.168687    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:47:17.700342 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:47:17.700286    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:47:17.796142 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:47:17.796105    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:47:18.134039 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:47:18.133993    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:47:24.151011 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:47:24.150955    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:47:24.251007 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:47:24.250954    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:47:24.440536 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:47:24.440449    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:47:24.693989 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:47:24.693901    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:47:24.926707 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:47:24.926673    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:47:27.653954 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:47:27.653914    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:47:27.808944 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:47:27.808914    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:47:27.893849 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:47:27.893813    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:47:28.452487 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:47:28.452453    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:47:28.550798 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:47:28.550761    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:47:33.046842 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:47:33.046806    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:47:55.433253 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:47:55.433219    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:48:03.781639 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:48:03.781560    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:48:06.871816 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:48:06.871772    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:48:08.265246 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:48:08.265214    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:48:08.466213 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:48:08.466181    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:48:08.759338 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:48:08.759305    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:48:09.045154 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:48:09.045074    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:48:09.251883 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:48:09.251850    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:48:09.501512 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:48:09.501476    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:48:10.009573 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:48:10.009539    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:48:10.219470 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:48:10.219438    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:48:13.040641 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:48:13.040604    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:48:13.479765 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:48:13.479649    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:48:13.694491 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:48:13.694456    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:48:14.277510 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:48:14.277474    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:48:14.402511 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:48:14.402477    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:48:18.484976 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:48:18.484937    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:48:48.909367 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:48:48.909336    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:48:51.995275 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:48:51.995231    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:48:53.249216 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:48:53.249186    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:48:53.614603 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:48:53.614518    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:48:53.869270 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:48:53.869190    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:48:54.006276 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:48:54.006246    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:48:54.443485 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:48:54.443454    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:48:54.758491 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:48:54.758460    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:48:57.356920 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:48:57.356887    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:48:57.678175 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:48:57.678090    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:48:57.865354 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:48:57.865323    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:48:58.185944 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:48:58.185918    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:48:59.426866 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:48:59.426834    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:48:59.943193 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:48:59.943162    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:49:04.008311 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:49:04.008270    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:49:23.438930 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:49:23.438888    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:49:33.484999 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:49:33.484923    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:49:35.479730 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:49:35.479697    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:49:38.151384 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:49:38.151345    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:49:39.372721 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:49:39.372685    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:49:39.628778 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:49:39.628680    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:49:39.858513 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:49:39.858483    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:49:40.261695 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:49:40.261651    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:49:40.695044 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:49:40.694958    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:49:42.281803 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:49:42.281772    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:49:42.376880 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:49:42.376847    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:49:42.638234 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:49:42.638153    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:49:42.940164 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:49:42.940085    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:49:43.290818 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:49:43.290789    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:49:43.392816 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:49:43.392781    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:49:47.559341 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:49:47.559306    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:50:03.328228 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:50:03.328192    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:50:03.483858 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:50:03.483825    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:50:04.123164 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:50:04.123132    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:50:04.223011 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:50:04.222978    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:50:17.656080 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:50:17.656044    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:50:20.641327 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:50:20.641286    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:50:22.263582 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:50:22.263546    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:50:22.428147 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:50:22.428116    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:50:22.671695 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:50:22.671609    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:50:23.211815 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:50:23.211781    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:50:23.310815 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:50:23.310778    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:50:24.896229 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:50:24.896197    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:50:31.657518 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:50:31.657446    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:50:31.789190 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:50:31.789157    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:50:31.913472 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:50:31.913387    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:50:32.015087 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:50:32.015052    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:50:32.557313 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:50:32.557285    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:50:36.130542 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:50:36.130508    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:50:58.460691 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:50:58.460659    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:51:05.622790 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:51:05.622688    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:51:05.734706 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:51:05.734672    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:51:05.838191 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:51:05.838157    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:51:21.756560 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:51:21.756522    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:51:27.659310 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:51:27.659276    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:51:27.761706 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:51:27.761674    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:51:27.860072 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:51:27.860040    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:51:38.436706 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:51:38.436667    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:51:39.249138 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:51:39.249101    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:51:46.345008 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:51:46.344976    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:51:46.452600 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:51:46.452564    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:51:46.557255 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:51:46.557223    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:51:46.651035 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:51:46.650947    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:51:47.468762 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:51:47.468694    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:51:47.569593 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:51:47.569562    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:51:47.674328 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:51:47.674292    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:51:49.395347 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:51:49.395312    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:51:51.198161 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:51:51.198124    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:51:55.600942 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:51:55.600907    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:51:55.701897 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:51:55.701859    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:51:56.336503 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:51:56.336469    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:52:00.839941 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:52:00.839904    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:52:01.170763 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:52:01.170660    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:52:02.724213 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:52:02.724175    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:52:02.826182 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:52:02.826144    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:52:02.973448 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:52:02.973416    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:52:03.311989 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:52:03.311952    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:52:03.440430 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:52:03.440392    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:52:06.560000 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:52:06.559965    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:52:06.650194 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:52:06.650161    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:52:06.808974 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:52:06.808934    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:52:07.214717 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:52:07.214678    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:52:07.504102 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:52:07.504067    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:52:07.613614 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:52:07.613582    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:52:11.045685 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:52:11.045651    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:52:11.767433 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:52:11.767397    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:52:11.880571 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:52:11.880529    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:52:11.882944 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:52:11.882919    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:52:12.220727 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:52:12.220641    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:52:18.086308 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:52:18.086272    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:52:18.187497 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:52:18.187460    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:52:29.072432 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:52:29.072398    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:52:32.140284 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:52:32.140193    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:52:33.499989 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:52:33.499955    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:52:33.900123 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:52:33.900046    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:52:39.565022 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:52:39.564983    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:52:39.665060 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:52:39.665030    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:52:39.829419 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:52:39.829327    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:52:40.093055 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:52:40.092968    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:52:40.200436 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:52:40.200401    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:52:40.305515 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:52:40.305477    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:52:40.436484 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:52:40.436405    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:52:46.750764 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:52:46.750707    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:52:46.865022 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:52:46.864981    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:52:46.972160 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:52:46.972126    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:52:47.294631 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:52:47.294600    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:52:53.507199 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:52:53.507166    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:52:53.577274 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:52:53.577244    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:53:00.287273 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:00.287221    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:53:00.802444 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:00.802400    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:53:00.908328 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:00.908295    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:53:03.863360 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:03.863293    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:53:04.058102 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:04.058068    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:53:04.148453 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:04.148369    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:53:04.551580 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:04.551550    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:53:04.651574 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:04.651542    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:53:08.304134 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:08.304100    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:53:08.957453 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:08.957402    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:53:11.688322 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:11.688286    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:53:11.755398 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:11.755362    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:53:21.100040 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:21.100005    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:53:24.379658 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:24.379616    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-6a145fe4/e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d"]
Jul 22 02:53:24.380064 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:24.380040    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71065006-9b5f-4323-9178-7358648da265" containerName="main"
Jul 22 02:53:24.380064 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:24.380054    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="71065006-9b5f-4323-9178-7358648da265" containerName="main"
Jul 22 02:53:24.380064 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:24.380061    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d679c7d-825d-4c8f-8476-1664c93931b3" containerName="main"
Jul 22 02:53:24.380064 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:24.380066    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d679c7d-825d-4c8f-8476-1664c93931b3" containerName="main"
Jul 22 02:53:24.380197 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:24.380127    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="71065006-9b5f-4323-9178-7358648da265" containerName="main"
Jul 22 02:53:24.380197 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:24.380138    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d679c7d-825d-4c8f-8476-1664c93931b3" containerName="main"
Jul 22 02:53:24.383136 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:24.383119    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6a145fe4/e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d"
Jul 22 02:53:24.386710 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:24.386688    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-6a145fe4\"/\"openshift-service-ca.crt\""
Jul 22 02:53:24.386900 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:24.386824    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-6a145fe4\"/\"kube-root-ca.crt\""
Jul 22 02:53:24.387805 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:24.387787    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-6a145fe4\"/\"default-dockercfg-wghvp\""
Jul 22 02:53:24.387869 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:24.387806    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-6a145fe4\"/\"e2e-lifecycle-model-6a145fe4-kserve-self-signed-certs\""
Jul 22 02:53:24.397796 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:24.397767    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-6a145fe4/e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d"]
Jul 22 02:53:24.398823 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:24.398801    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:53:24.499028 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:24.498998    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrft2\" (UniqueName: \"kubernetes.io/projected/946c100a-c171-4bf0-9d33-3ef62efd8456-kube-api-access-lrft2\") pod \"e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d\" (UID: \"946c100a-c171-4bf0-9d33-3ef62efd8456\") " pod="ai-tenant-e2e-mt-6a145fe4/e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d"
Jul 22 02:53:24.499218 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:24.499043    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/946c100a-c171-4bf0-9d33-3ef62efd8456-dshm\") pod \"e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d\" (UID: \"946c100a-c171-4bf0-9d33-3ef62efd8456\") " pod="ai-tenant-e2e-mt-6a145fe4/e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d"
Jul 22 02:53:24.499218 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:24.499066    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/946c100a-c171-4bf0-9d33-3ef62efd8456-model-cache\") pod \"e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d\" (UID: \"946c100a-c171-4bf0-9d33-3ef62efd8456\") " pod="ai-tenant-e2e-mt-6a145fe4/e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d"
Jul 22 02:53:24.499218 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:24.499166    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/946c100a-c171-4bf0-9d33-3ef62efd8456-home\") pod \"e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d\" (UID: \"946c100a-c171-4bf0-9d33-3ef62efd8456\") " pod="ai-tenant-e2e-mt-6a145fe4/e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d"
Jul 22 02:53:24.499218 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:24.499203    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/946c100a-c171-4bf0-9d33-3ef62efd8456-tmp-dir\") pod \"e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d\" (UID: \"946c100a-c171-4bf0-9d33-3ef62efd8456\") " pod="ai-tenant-e2e-mt-6a145fe4/e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d"
Jul 22 02:53:24.499387 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:24.499266    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/946c100a-c171-4bf0-9d33-3ef62efd8456-tls-certs\") pod \"e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d\" (UID: \"946c100a-c171-4bf0-9d33-3ef62efd8456\") " pod="ai-tenant-e2e-mt-6a145fe4/e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d"
Jul 22 02:53:24.600787 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:24.600729    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrft2\" (UniqueName: \"kubernetes.io/projected/946c100a-c171-4bf0-9d33-3ef62efd8456-kube-api-access-lrft2\") pod \"e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d\" (UID: \"946c100a-c171-4bf0-9d33-3ef62efd8456\") " pod="ai-tenant-e2e-mt-6a145fe4/e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d"
Jul 22 02:53:24.600963 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:24.600809    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/946c100a-c171-4bf0-9d33-3ef62efd8456-dshm\") pod \"e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d\" (UID: \"946c100a-c171-4bf0-9d33-3ef62efd8456\") " pod="ai-tenant-e2e-mt-6a145fe4/e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d"
Jul 22 02:53:24.600963 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:24.600839    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/946c100a-c171-4bf0-9d33-3ef62efd8456-model-cache\") pod \"e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d\" (UID: \"946c100a-c171-4bf0-9d33-3ef62efd8456\") " pod="ai-tenant-e2e-mt-6a145fe4/e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d"
Jul 22 02:53:24.600963 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:24.600898    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/946c100a-c171-4bf0-9d33-3ef62efd8456-home\") pod \"e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d\" (UID: \"946c100a-c171-4bf0-9d33-3ef62efd8456\") " pod="ai-tenant-e2e-mt-6a145fe4/e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d"
Jul 22 02:53:24.600963 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:24.600927    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/946c100a-c171-4bf0-9d33-3ef62efd8456-tmp-dir\") pod \"e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d\" (UID: \"946c100a-c171-4bf0-9d33-3ef62efd8456\") " pod="ai-tenant-e2e-mt-6a145fe4/e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d"
Jul 22 02:53:24.601115 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:24.600991    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/946c100a-c171-4bf0-9d33-3ef62efd8456-tls-certs\") pod \"e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d\" (UID: \"946c100a-c171-4bf0-9d33-3ef62efd8456\") " pod="ai-tenant-e2e-mt-6a145fe4/e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d"
Jul 22 02:53:24.601351 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:24.601318    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/946c100a-c171-4bf0-9d33-3ef62efd8456-home\") pod \"e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d\" (UID: \"946c100a-c171-4bf0-9d33-3ef62efd8456\") " pod="ai-tenant-e2e-mt-6a145fe4/e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d"
Jul 22 02:53:24.601433 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:24.601367    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/946c100a-c171-4bf0-9d33-3ef62efd8456-model-cache\") pod \"e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d\" (UID: \"946c100a-c171-4bf0-9d33-3ef62efd8456\") " pod="ai-tenant-e2e-mt-6a145fe4/e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d"
Jul 22 02:53:24.601433 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:24.601398    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/946c100a-c171-4bf0-9d33-3ef62efd8456-tmp-dir\") pod \"e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d\" (UID: \"946c100a-c171-4bf0-9d33-3ef62efd8456\") " pod="ai-tenant-e2e-mt-6a145fe4/e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d"
Jul 22 02:53:24.603197 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:24.603175    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/946c100a-c171-4bf0-9d33-3ef62efd8456-dshm\") pod \"e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d\" (UID: \"946c100a-c171-4bf0-9d33-3ef62efd8456\") " pod="ai-tenant-e2e-mt-6a145fe4/e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d"
Jul 22 02:53:24.603706 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:24.603677    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/946c100a-c171-4bf0-9d33-3ef62efd8456-tls-certs\") pod \"e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d\" (UID: \"946c100a-c171-4bf0-9d33-3ef62efd8456\") " pod="ai-tenant-e2e-mt-6a145fe4/e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d"
Jul 22 02:53:24.609762 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:24.609715    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrft2\" (UniqueName: \"kubernetes.io/projected/946c100a-c171-4bf0-9d33-3ef62efd8456-kube-api-access-lrft2\") pod \"e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d\" (UID: \"946c100a-c171-4bf0-9d33-3ef62efd8456\") " pod="ai-tenant-e2e-mt-6a145fe4/e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d"
Jul 22 02:53:24.692831 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:24.692721    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6a145fe4/e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d"
Jul 22 02:53:24.834866 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:24.834839    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-6a145fe4/e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d"]
Jul 22 02:53:24.836683 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:53:24.836651    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod946c100a_c171_4bf0_9d33_3ef62efd8456.slice/crio-f18ad216ebbd1b80d8270ea9d4df4c39420b5c2f016a3714015fafd4f71ea193 WatchSource:0}: Error finding container f18ad216ebbd1b80d8270ea9d4df4c39420b5c2f016a3714015fafd4f71ea193: Status 404 returned error can't find the container with id f18ad216ebbd1b80d8270ea9d4df4c39420b5c2f016a3714015fafd4f71ea193
Jul 22 02:53:25.758926 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:25.758893    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6a145fe4/e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d" event={"ID":"946c100a-c171-4bf0-9d33-3ef62efd8456","Type":"ContainerStarted","Data":"fc3782f556d1a1ff03d4d1d25535c2ccfa57eef1808fdaa49a2a2e9f93731603"}
Jul 22 02:53:25.758926 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:25.758933    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6a145fe4/e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d" event={"ID":"946c100a-c171-4bf0-9d33-3ef62efd8456","Type":"ContainerStarted","Data":"f18ad216ebbd1b80d8270ea9d4df4c39420b5c2f016a3714015fafd4f71ea193"}
Jul 22 02:53:25.777481 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:25.777426    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-6a145fe4/e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d" podStartSLOduration=1.777410951 podStartE2EDuration="1.777410951s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:53:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:53:25.77629227 +0000 UTC m=+2027.317085898" watchObservedRunningTime="2026-07-22 02:53:25.777410951 +0000 UTC m=+2027.318204578"
Jul 22 02:53:27.642483 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:27.642450    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:53:34.693039 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:34.693000    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-6a145fe4/e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d"
Jul 22 02:53:34.693531 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:34.693064    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-6a145fe4/e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d"
Jul 22 02:53:34.705375 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:34.705343    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-6a145fe4/e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d"
Jul 22 02:53:34.813412 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:34.813376    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-6a145fe4/e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d"
Jul 22 02:53:43.159217 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:53:43.159177    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:54:00.963399 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:00.963361    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:54:02.374709 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:02.374659    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:54:06.868065 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:06.868031    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:54:09.176626 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:09.176594    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:54:09.646900 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:09.646868    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:54:14.123234 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:14.123197    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-6a145fe4/e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d"]
Jul 22 02:54:14.175916 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:14.175886    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:54:29.143698 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:29.143647    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-6a145fe4/e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d" podUID="946c100a-c171-4bf0-9d33-3ef62efd8456" containerName="main" containerID="cri-o://fc3782f556d1a1ff03d4d1d25535c2ccfa57eef1808fdaa49a2a2e9f93731603" gracePeriod=45
Jul 22 02:54:29.400617 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:29.400554    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6a145fe4/e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d"
Jul 22 02:54:29.532097 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:29.532062    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/946c100a-c171-4bf0-9d33-3ef62efd8456-dshm\") pod \"946c100a-c171-4bf0-9d33-3ef62efd8456\" (UID: \"946c100a-c171-4bf0-9d33-3ef62efd8456\") "
Jul 22 02:54:29.532307 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:29.532131    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/946c100a-c171-4bf0-9d33-3ef62efd8456-tls-certs\") pod \"946c100a-c171-4bf0-9d33-3ef62efd8456\" (UID: \"946c100a-c171-4bf0-9d33-3ef62efd8456\") "
Jul 22 02:54:29.532307 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:29.532190    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrft2\" (UniqueName: \"kubernetes.io/projected/946c100a-c171-4bf0-9d33-3ef62efd8456-kube-api-access-lrft2\") pod \"946c100a-c171-4bf0-9d33-3ef62efd8456\" (UID: \"946c100a-c171-4bf0-9d33-3ef62efd8456\") "
Jul 22 02:54:29.532307 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:29.532253    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/946c100a-c171-4bf0-9d33-3ef62efd8456-tmp-dir\") pod \"946c100a-c171-4bf0-9d33-3ef62efd8456\" (UID: \"946c100a-c171-4bf0-9d33-3ef62efd8456\") "
Jul 22 02:54:29.532307 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:29.532273    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/946c100a-c171-4bf0-9d33-3ef62efd8456-model-cache\") pod \"946c100a-c171-4bf0-9d33-3ef62efd8456\" (UID: \"946c100a-c171-4bf0-9d33-3ef62efd8456\") "
Jul 22 02:54:29.532307 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:29.532299    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/946c100a-c171-4bf0-9d33-3ef62efd8456-home\") pod \"946c100a-c171-4bf0-9d33-3ef62efd8456\" (UID: \"946c100a-c171-4bf0-9d33-3ef62efd8456\") "
Jul 22 02:54:29.532611 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:29.532542    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/946c100a-c171-4bf0-9d33-3ef62efd8456-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "946c100a-c171-4bf0-9d33-3ef62efd8456" (UID: "946c100a-c171-4bf0-9d33-3ef62efd8456"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:54:29.532689 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:29.532657    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/946c100a-c171-4bf0-9d33-3ef62efd8456-home" (OuterVolumeSpecName: "home") pod "946c100a-c171-4bf0-9d33-3ef62efd8456" (UID: "946c100a-c171-4bf0-9d33-3ef62efd8456"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:54:29.532850 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:29.532691    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/946c100a-c171-4bf0-9d33-3ef62efd8456-model-cache" (OuterVolumeSpecName: "model-cache") pod "946c100a-c171-4bf0-9d33-3ef62efd8456" (UID: "946c100a-c171-4bf0-9d33-3ef62efd8456"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:54:29.534519 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:29.534489    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/946c100a-c171-4bf0-9d33-3ef62efd8456-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "946c100a-c171-4bf0-9d33-3ef62efd8456" (UID: "946c100a-c171-4bf0-9d33-3ef62efd8456"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 02:54:29.534610 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:29.534529    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/946c100a-c171-4bf0-9d33-3ef62efd8456-kube-api-access-lrft2" (OuterVolumeSpecName: "kube-api-access-lrft2") pod "946c100a-c171-4bf0-9d33-3ef62efd8456" (UID: "946c100a-c171-4bf0-9d33-3ef62efd8456"). InnerVolumeSpecName "kube-api-access-lrft2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 02:54:29.534610 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:29.534570    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/946c100a-c171-4bf0-9d33-3ef62efd8456-dshm" (OuterVolumeSpecName: "dshm") pod "946c100a-c171-4bf0-9d33-3ef62efd8456" (UID: "946c100a-c171-4bf0-9d33-3ef62efd8456"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:54:29.633440 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:29.633404    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/946c100a-c171-4bf0-9d33-3ef62efd8456-tmp-dir\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:54:29.633440 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:29.633434    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/946c100a-c171-4bf0-9d33-3ef62efd8456-model-cache\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:54:29.633440 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:29.633446    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/946c100a-c171-4bf0-9d33-3ef62efd8456-home\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:54:29.633724 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:29.633453    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/946c100a-c171-4bf0-9d33-3ef62efd8456-dshm\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:54:29.633724 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:29.633462    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/946c100a-c171-4bf0-9d33-3ef62efd8456-tls-certs\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:54:29.633724 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:29.633470    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrft2\" (UniqueName: \"kubernetes.io/projected/946c100a-c171-4bf0-9d33-3ef62efd8456-kube-api-access-lrft2\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:54:30.066244 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:30.066206    2581 generic.go:356] "Generic (PLEG): container finished" podID="946c100a-c171-4bf0-9d33-3ef62efd8456" containerID="fc3782f556d1a1ff03d4d1d25535c2ccfa57eef1808fdaa49a2a2e9f93731603" exitCode=0
Jul 22 02:54:30.066244 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:30.066244    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6a145fe4/e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d"
Jul 22 02:54:30.066495 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:30.066283    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6a145fe4/e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d" event={"ID":"946c100a-c171-4bf0-9d33-3ef62efd8456","Type":"ContainerDied","Data":"fc3782f556d1a1ff03d4d1d25535c2ccfa57eef1808fdaa49a2a2e9f93731603"}
Jul 22 02:54:30.066495 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:30.066325    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6a145fe4/e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d" event={"ID":"946c100a-c171-4bf0-9d33-3ef62efd8456","Type":"ContainerDied","Data":"f18ad216ebbd1b80d8270ea9d4df4c39420b5c2f016a3714015fafd4f71ea193"}
Jul 22 02:54:30.066495 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:30.066342    2581 scope.go:117] "RemoveContainer" containerID="fc3782f556d1a1ff03d4d1d25535c2ccfa57eef1808fdaa49a2a2e9f93731603"
Jul 22 02:54:30.075421 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:30.075401    2581 scope.go:117] "RemoveContainer" containerID="fc3782f556d1a1ff03d4d1d25535c2ccfa57eef1808fdaa49a2a2e9f93731603"
Jul 22 02:54:30.075660 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:54:30.075642    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc3782f556d1a1ff03d4d1d25535c2ccfa57eef1808fdaa49a2a2e9f93731603\": container with ID starting with fc3782f556d1a1ff03d4d1d25535c2ccfa57eef1808fdaa49a2a2e9f93731603 not found: ID does not exist" containerID="fc3782f556d1a1ff03d4d1d25535c2ccfa57eef1808fdaa49a2a2e9f93731603"
Jul 22 02:54:30.075713 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:30.075668    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc3782f556d1a1ff03d4d1d25535c2ccfa57eef1808fdaa49a2a2e9f93731603"} err="failed to get container status \"fc3782f556d1a1ff03d4d1d25535c2ccfa57eef1808fdaa49a2a2e9f93731603\": rpc error: code = NotFound desc = could not find container \"fc3782f556d1a1ff03d4d1d25535c2ccfa57eef1808fdaa49a2a2e9f93731603\": container with ID starting with fc3782f556d1a1ff03d4d1d25535c2ccfa57eef1808fdaa49a2a2e9f93731603 not found: ID does not exist"
Jul 22 02:54:30.088583 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:30.088557    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-6a145fe4/e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d"]
Jul 22 02:54:30.092653 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:30.092628    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-6a145fe4/e2e-lifecycle-model-6a145fe4-kserve-5f4d4975-4xv5d"]
Jul 22 02:54:30.903533 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:30.903498    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="946c100a-c171-4bf0-9d33-3ef62efd8456" path="/var/lib/kubelet/pods/946c100a-c171-4bf0-9d33-3ef62efd8456/volumes"
Jul 22 02:54:50.742954 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:50.742892    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:54:51.232651 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:51.232548    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:54:57.077719 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:54:57.077675    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:55:03.022533 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:55:03.022456    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:55:03.122880 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:55:03.122853    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:55:03.231792 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:55:03.231754    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:55:03.466608 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:55:03.466502    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:55:04.112924 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:55:04.112885    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:55:04.214076 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:55:04.214042    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:55:04.523115 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:55:04.523082    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:55:04.637005 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:55:04.636971    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:55:04.822621 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:55:04.822499    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:55:05.564945 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:55:05.564909    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:55:05.664208 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:55:05.664160    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:55:05.764506 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:55:05.764451    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:55:07.155513 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:55:07.155465    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:55:07.165812 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:55:07.165782    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:55:14.273336 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:55:14.273304    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:55:14.378702 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:55:14.378663    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:55:14.974227 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:55:14.974190    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:55:17.652988 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:55:17.652944    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:55:21.422842 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:55:21.422794    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:55:21.528752 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:55:21.528704    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:55:21.639256 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:55:21.639224    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:55:32.186783 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:55:32.186693    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:55:32.561998 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:55:32.561955    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:55:53.497330 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:55:53.497278    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:56:00.379959 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:00.379920    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:56:00.480376 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:00.480341    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:56:00.732484 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:00.732398    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:56:01.090645 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:01.090604    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:56:01.494699 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:01.494670    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:56:03.802429 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:03.802395    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:56:03.893358 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:03.893324    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:56:03.998040 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:03.998002    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:56:04.096624 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:04.096538    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:56:04.471186 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:04.471096    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:56:04.572314 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:04.572279    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:56:08.204003 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:08.203971    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:56:08.538570 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:08.538540    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:56:09.056854 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:09.056808    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:56:09.252533 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:09.252481    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:56:09.855089 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:09.855051    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:56:11.714099 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:11.714060    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:56:11.828600 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:11.828567    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:56:11.927042 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:11.927007    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:56:12.183663 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:12.183632    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:56:12.574528 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:12.574495    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:56:12.674986 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:12.674938    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:56:16.181076 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:16.181032    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:56:25.257215 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:25.257167    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:56:25.358134 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:25.358095    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:56:25.744212 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:25.744175    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:56:33.326080 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:33.326037    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:56:33.426236 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:33.426174    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:56:39.142919 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:39.142877    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-fbb5cc/test-model-fbb5cc-kserve-6ccf4558b7-g8jj4"]
Jul 22 02:56:39.143369 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:39.143304    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="946c100a-c171-4bf0-9d33-3ef62efd8456" containerName="main"
Jul 22 02:56:39.143369 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:39.143319    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="946c100a-c171-4bf0-9d33-3ef62efd8456" containerName="main"
Jul 22 02:56:39.143456 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:39.143407    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="946c100a-c171-4bf0-9d33-3ef62efd8456" containerName="main"
Jul 22 02:56:39.147033 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:39.147010    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-fbb5cc/test-model-fbb5cc-kserve-6ccf4558b7-g8jj4"
Jul 22 02:56:39.152330 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:39.152116    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-fbb5cc\"/\"test-model-fbb5cc-kserve-self-signed-certs\""
Jul 22 02:56:39.152330 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:39.152129    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-fbb5cc\"/\"openshift-service-ca.crt\""
Jul 22 02:56:39.152330 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:39.152229    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-fbb5cc\"/\"kube-root-ca.crt\""
Jul 22 02:56:39.152597 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:39.152477    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-fbb5cc\"/\"default-dockercfg-5gd2f\""
Jul 22 02:56:39.159460 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:39.159436    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-fbb5cc/test-model-fbb5cc-kserve-6ccf4558b7-g8jj4"]
Jul 22 02:56:39.186075 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:39.186042    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dcb7ba03-f407-4bde-81cd-214ea8309daf-home\") pod \"test-model-fbb5cc-kserve-6ccf4558b7-g8jj4\" (UID: \"dcb7ba03-f407-4bde-81cd-214ea8309daf\") " pod="ai-tenant-e2e-inf-a-fbb5cc/test-model-fbb5cc-kserve-6ccf4558b7-g8jj4"
Jul 22 02:56:39.186228 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:39.186085    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dcb7ba03-f407-4bde-81cd-214ea8309daf-tmp-dir\") pod \"test-model-fbb5cc-kserve-6ccf4558b7-g8jj4\" (UID: \"dcb7ba03-f407-4bde-81cd-214ea8309daf\") " pod="ai-tenant-e2e-inf-a-fbb5cc/test-model-fbb5cc-kserve-6ccf4558b7-g8jj4"
Jul 22 02:56:39.186228 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:39.186123    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dcb7ba03-f407-4bde-81cd-214ea8309daf-tls-certs\") pod \"test-model-fbb5cc-kserve-6ccf4558b7-g8jj4\" (UID: \"dcb7ba03-f407-4bde-81cd-214ea8309daf\") " pod="ai-tenant-e2e-inf-a-fbb5cc/test-model-fbb5cc-kserve-6ccf4558b7-g8jj4"
Jul 22 02:56:39.186228 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:39.186161    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwnsn\" (UniqueName: \"kubernetes.io/projected/dcb7ba03-f407-4bde-81cd-214ea8309daf-kube-api-access-vwnsn\") pod \"test-model-fbb5cc-kserve-6ccf4558b7-g8jj4\" (UID: \"dcb7ba03-f407-4bde-81cd-214ea8309daf\") " pod="ai-tenant-e2e-inf-a-fbb5cc/test-model-fbb5cc-kserve-6ccf4558b7-g8jj4"
Jul 22 02:56:39.186228 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:39.186190    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dcb7ba03-f407-4bde-81cd-214ea8309daf-model-cache\") pod \"test-model-fbb5cc-kserve-6ccf4558b7-g8jj4\" (UID: \"dcb7ba03-f407-4bde-81cd-214ea8309daf\") " pod="ai-tenant-e2e-inf-a-fbb5cc/test-model-fbb5cc-kserve-6ccf4558b7-g8jj4"
Jul 22 02:56:39.186377 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:39.186242    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dcb7ba03-f407-4bde-81cd-214ea8309daf-dshm\") pod \"test-model-fbb5cc-kserve-6ccf4558b7-g8jj4\" (UID: \"dcb7ba03-f407-4bde-81cd-214ea8309daf\") " pod="ai-tenant-e2e-inf-a-fbb5cc/test-model-fbb5cc-kserve-6ccf4558b7-g8jj4"
Jul 22 02:56:39.287554 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:39.287509    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dcb7ba03-f407-4bde-81cd-214ea8309daf-dshm\") pod \"test-model-fbb5cc-kserve-6ccf4558b7-g8jj4\" (UID: \"dcb7ba03-f407-4bde-81cd-214ea8309daf\") " pod="ai-tenant-e2e-inf-a-fbb5cc/test-model-fbb5cc-kserve-6ccf4558b7-g8jj4"
Jul 22 02:56:39.287765 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:39.287615    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dcb7ba03-f407-4bde-81cd-214ea8309daf-home\") pod \"test-model-fbb5cc-kserve-6ccf4558b7-g8jj4\" (UID: \"dcb7ba03-f407-4bde-81cd-214ea8309daf\") " pod="ai-tenant-e2e-inf-a-fbb5cc/test-model-fbb5cc-kserve-6ccf4558b7-g8jj4"
Jul 22 02:56:39.287765 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:39.287651    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dcb7ba03-f407-4bde-81cd-214ea8309daf-tmp-dir\") pod \"test-model-fbb5cc-kserve-6ccf4558b7-g8jj4\" (UID: \"dcb7ba03-f407-4bde-81cd-214ea8309daf\") " pod="ai-tenant-e2e-inf-a-fbb5cc/test-model-fbb5cc-kserve-6ccf4558b7-g8jj4"
Jul 22 02:56:39.287765 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:39.287705    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dcb7ba03-f407-4bde-81cd-214ea8309daf-tls-certs\") pod \"test-model-fbb5cc-kserve-6ccf4558b7-g8jj4\" (UID: \"dcb7ba03-f407-4bde-81cd-214ea8309daf\") " pod="ai-tenant-e2e-inf-a-fbb5cc/test-model-fbb5cc-kserve-6ccf4558b7-g8jj4"
Jul 22 02:56:39.287958 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:39.287772    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwnsn\" (UniqueName: \"kubernetes.io/projected/dcb7ba03-f407-4bde-81cd-214ea8309daf-kube-api-access-vwnsn\") pod \"test-model-fbb5cc-kserve-6ccf4558b7-g8jj4\" (UID: \"dcb7ba03-f407-4bde-81cd-214ea8309daf\") " pod="ai-tenant-e2e-inf-a-fbb5cc/test-model-fbb5cc-kserve-6ccf4558b7-g8jj4"
Jul 22 02:56:39.287958 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:39.287805    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dcb7ba03-f407-4bde-81cd-214ea8309daf-model-cache\") pod \"test-model-fbb5cc-kserve-6ccf4558b7-g8jj4\" (UID: \"dcb7ba03-f407-4bde-81cd-214ea8309daf\") " pod="ai-tenant-e2e-inf-a-fbb5cc/test-model-fbb5cc-kserve-6ccf4558b7-g8jj4"
Jul 22 02:56:39.288109 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:39.288083    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dcb7ba03-f407-4bde-81cd-214ea8309daf-tmp-dir\") pod \"test-model-fbb5cc-kserve-6ccf4558b7-g8jj4\" (UID: \"dcb7ba03-f407-4bde-81cd-214ea8309daf\") " pod="ai-tenant-e2e-inf-a-fbb5cc/test-model-fbb5cc-kserve-6ccf4558b7-g8jj4"
Jul 22 02:56:39.288294 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:39.288133    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dcb7ba03-f407-4bde-81cd-214ea8309daf-home\") pod \"test-model-fbb5cc-kserve-6ccf4558b7-g8jj4\" (UID: \"dcb7ba03-f407-4bde-81cd-214ea8309daf\") " pod="ai-tenant-e2e-inf-a-fbb5cc/test-model-fbb5cc-kserve-6ccf4558b7-g8jj4"
Jul 22 02:56:39.288294 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:39.288229    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dcb7ba03-f407-4bde-81cd-214ea8309daf-model-cache\") pod \"test-model-fbb5cc-kserve-6ccf4558b7-g8jj4\" (UID: \"dcb7ba03-f407-4bde-81cd-214ea8309daf\") " pod="ai-tenant-e2e-inf-a-fbb5cc/test-model-fbb5cc-kserve-6ccf4558b7-g8jj4"
Jul 22 02:56:39.289953 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:39.289926    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dcb7ba03-f407-4bde-81cd-214ea8309daf-dshm\") pod \"test-model-fbb5cc-kserve-6ccf4558b7-g8jj4\" (UID: \"dcb7ba03-f407-4bde-81cd-214ea8309daf\") " pod="ai-tenant-e2e-inf-a-fbb5cc/test-model-fbb5cc-kserve-6ccf4558b7-g8jj4"
Jul 22 02:56:39.290361 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:39.290343    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dcb7ba03-f407-4bde-81cd-214ea8309daf-tls-certs\") pod \"test-model-fbb5cc-kserve-6ccf4558b7-g8jj4\" (UID: \"dcb7ba03-f407-4bde-81cd-214ea8309daf\") " pod="ai-tenant-e2e-inf-a-fbb5cc/test-model-fbb5cc-kserve-6ccf4558b7-g8jj4"
Jul 22 02:56:39.297242 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:39.297214    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwnsn\" (UniqueName: \"kubernetes.io/projected/dcb7ba03-f407-4bde-81cd-214ea8309daf-kube-api-access-vwnsn\") pod \"test-model-fbb5cc-kserve-6ccf4558b7-g8jj4\" (UID: \"dcb7ba03-f407-4bde-81cd-214ea8309daf\") " pod="ai-tenant-e2e-inf-a-fbb5cc/test-model-fbb5cc-kserve-6ccf4558b7-g8jj4"
Jul 22 02:56:39.303243 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:39.303205    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:56:39.458483 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:39.458368    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-fbb5cc/test-model-fbb5cc-kserve-6ccf4558b7-g8jj4"
Jul 22 02:56:39.601424 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:39.601395    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-fbb5cc/test-model-fbb5cc-kserve-6ccf4558b7-g8jj4"]
Jul 22 02:56:39.605314 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:56:39.605282    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddcb7ba03_f407_4bde_81cd_214ea8309daf.slice/crio-cb663f945d90c7286890064c627fa9087121d8c949a898a5af778cc4ae827406 WatchSource:0}: Error finding container cb663f945d90c7286890064c627fa9087121d8c949a898a5af778cc4ae827406: Status 404 returned error can't find the container with id cb663f945d90c7286890064c627fa9087121d8c949a898a5af778cc4ae827406
Jul 22 02:56:39.698633 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:39.698595    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-fbb5cc/test-model-fbb5cc-kserve-6ccf4558b7-g8jj4" event={"ID":"dcb7ba03-f407-4bde-81cd-214ea8309daf","Type":"ContainerStarted","Data":"b82e22e248f19d19f014b31b303f7f6bb7c0969fa78420e59b3593cc0640599f"}
Jul 22 02:56:39.698633 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:39.698642    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-fbb5cc/test-model-fbb5cc-kserve-6ccf4558b7-g8jj4" event={"ID":"dcb7ba03-f407-4bde-81cd-214ea8309daf","Type":"ContainerStarted","Data":"cb663f945d90c7286890064c627fa9087121d8c949a898a5af778cc4ae827406"}
Jul 22 02:56:39.720213 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:39.720096    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-fbb5cc/test-model-fbb5cc-kserve-6ccf4558b7-g8jj4" podStartSLOduration=0.720077773 podStartE2EDuration="720.077773ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:56:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:56:39.718204174 +0000 UTC m=+2221.258997802" watchObservedRunningTime="2026-07-22 02:56:39.720077773 +0000 UTC m=+2221.260871399"
Jul 22 02:56:42.551291 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:42.551253    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:56:49.458980 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:49.458945    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-fbb5cc/test-model-fbb5cc-kserve-6ccf4558b7-g8jj4"
Jul 22 02:56:49.459467 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:49.458992    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-fbb5cc/test-model-fbb5cc-kserve-6ccf4558b7-g8jj4"
Jul 22 02:56:49.472102 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:49.472073    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-fbb5cc/test-model-fbb5cc-kserve-6ccf4558b7-g8jj4"
Jul 22 02:56:49.756642 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:49.756613    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-fbb5cc/test-model-fbb5cc-kserve-6ccf4558b7-g8jj4"
Jul 22 02:56:53.945520 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:56:53.945484    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:57:08.042800 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:08.042694    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-6873d4/test-model-6873d4-kserve-55d8c9c849-4lwnj"]
Jul 22 02:57:08.046547 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:08.046525    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-6873d4/test-model-6873d4-kserve-55d8c9c849-4lwnj"
Jul 22 02:57:08.050684 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:08.050662    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-6873d4\"/\"openshift-service-ca.crt\""
Jul 22 02:57:08.050684 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:08.050672    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-6873d4\"/\"kube-root-ca.crt\""
Jul 22 02:57:08.050903 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:08.050672    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-6873d4\"/\"test-model-6873d4-kserve-self-signed-certs\""
Jul 22 02:57:08.050903 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:08.050662    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-6873d4\"/\"default-dockercfg-bxsqp\""
Jul 22 02:57:08.057659 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:08.057635    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-6873d4/test-model-6873d4-kserve-55d8c9c849-4lwnj"]
Jul 22 02:57:08.068868 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:08.068844    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:57:08.071076 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:08.071054    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59985662-39b4-4f17-9cff-d6c05936d0ae-home\") pod \"test-model-6873d4-kserve-55d8c9c849-4lwnj\" (UID: \"59985662-39b4-4f17-9cff-d6c05936d0ae\") " pod="ai-tenant-e2e-inf-b-6873d4/test-model-6873d4-kserve-55d8c9c849-4lwnj"
Jul 22 02:57:08.071146 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:08.071092    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpp7h\" (UniqueName: \"kubernetes.io/projected/59985662-39b4-4f17-9cff-d6c05936d0ae-kube-api-access-gpp7h\") pod \"test-model-6873d4-kserve-55d8c9c849-4lwnj\" (UID: \"59985662-39b4-4f17-9cff-d6c05936d0ae\") " pod="ai-tenant-e2e-inf-b-6873d4/test-model-6873d4-kserve-55d8c9c849-4lwnj"
Jul 22 02:57:08.071146 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:08.071125    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59985662-39b4-4f17-9cff-d6c05936d0ae-model-cache\") pod \"test-model-6873d4-kserve-55d8c9c849-4lwnj\" (UID: \"59985662-39b4-4f17-9cff-d6c05936d0ae\") " pod="ai-tenant-e2e-inf-b-6873d4/test-model-6873d4-kserve-55d8c9c849-4lwnj"
Jul 22 02:57:08.071251 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:08.071191    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59985662-39b4-4f17-9cff-d6c05936d0ae-tmp-dir\") pod \"test-model-6873d4-kserve-55d8c9c849-4lwnj\" (UID: \"59985662-39b4-4f17-9cff-d6c05936d0ae\") " pod="ai-tenant-e2e-inf-b-6873d4/test-model-6873d4-kserve-55d8c9c849-4lwnj"
Jul 22 02:57:08.071251 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:08.071228    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59985662-39b4-4f17-9cff-d6c05936d0ae-dshm\") pod \"test-model-6873d4-kserve-55d8c9c849-4lwnj\" (UID: \"59985662-39b4-4f17-9cff-d6c05936d0ae\") " pod="ai-tenant-e2e-inf-b-6873d4/test-model-6873d4-kserve-55d8c9c849-4lwnj"
Jul 22 02:57:08.071373 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:08.071344    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59985662-39b4-4f17-9cff-d6c05936d0ae-tls-certs\") pod \"test-model-6873d4-kserve-55d8c9c849-4lwnj\" (UID: \"59985662-39b4-4f17-9cff-d6c05936d0ae\") " pod="ai-tenant-e2e-inf-b-6873d4/test-model-6873d4-kserve-55d8c9c849-4lwnj"
Jul 22 02:57:08.172862 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:08.172818    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59985662-39b4-4f17-9cff-d6c05936d0ae-tmp-dir\") pod \"test-model-6873d4-kserve-55d8c9c849-4lwnj\" (UID: \"59985662-39b4-4f17-9cff-d6c05936d0ae\") " pod="ai-tenant-e2e-inf-b-6873d4/test-model-6873d4-kserve-55d8c9c849-4lwnj"
Jul 22 02:57:08.173071 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:08.172890    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59985662-39b4-4f17-9cff-d6c05936d0ae-dshm\") pod \"test-model-6873d4-kserve-55d8c9c849-4lwnj\" (UID: \"59985662-39b4-4f17-9cff-d6c05936d0ae\") " pod="ai-tenant-e2e-inf-b-6873d4/test-model-6873d4-kserve-55d8c9c849-4lwnj"
Jul 22 02:57:08.173071 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:08.172951    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59985662-39b4-4f17-9cff-d6c05936d0ae-tls-certs\") pod \"test-model-6873d4-kserve-55d8c9c849-4lwnj\" (UID: \"59985662-39b4-4f17-9cff-d6c05936d0ae\") " pod="ai-tenant-e2e-inf-b-6873d4/test-model-6873d4-kserve-55d8c9c849-4lwnj"
Jul 22 02:57:08.173071 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:08.173040    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59985662-39b4-4f17-9cff-d6c05936d0ae-home\") pod \"test-model-6873d4-kserve-55d8c9c849-4lwnj\" (UID: \"59985662-39b4-4f17-9cff-d6c05936d0ae\") " pod="ai-tenant-e2e-inf-b-6873d4/test-model-6873d4-kserve-55d8c9c849-4lwnj"
Jul 22 02:57:08.173234 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:08.173074    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpp7h\" (UniqueName: \"kubernetes.io/projected/59985662-39b4-4f17-9cff-d6c05936d0ae-kube-api-access-gpp7h\") pod \"test-model-6873d4-kserve-55d8c9c849-4lwnj\" (UID: \"59985662-39b4-4f17-9cff-d6c05936d0ae\") " pod="ai-tenant-e2e-inf-b-6873d4/test-model-6873d4-kserve-55d8c9c849-4lwnj"
Jul 22 02:57:08.173234 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:08.173108    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59985662-39b4-4f17-9cff-d6c05936d0ae-model-cache\") pod \"test-model-6873d4-kserve-55d8c9c849-4lwnj\" (UID: \"59985662-39b4-4f17-9cff-d6c05936d0ae\") " pod="ai-tenant-e2e-inf-b-6873d4/test-model-6873d4-kserve-55d8c9c849-4lwnj"
Jul 22 02:57:08.173330 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:08.173296    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59985662-39b4-4f17-9cff-d6c05936d0ae-tmp-dir\") pod \"test-model-6873d4-kserve-55d8c9c849-4lwnj\" (UID: \"59985662-39b4-4f17-9cff-d6c05936d0ae\") " pod="ai-tenant-e2e-inf-b-6873d4/test-model-6873d4-kserve-55d8c9c849-4lwnj"
Jul 22 02:57:08.173464 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:08.173443    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59985662-39b4-4f17-9cff-d6c05936d0ae-model-cache\") pod \"test-model-6873d4-kserve-55d8c9c849-4lwnj\" (UID: \"59985662-39b4-4f17-9cff-d6c05936d0ae\") " pod="ai-tenant-e2e-inf-b-6873d4/test-model-6873d4-kserve-55d8c9c849-4lwnj"
Jul 22 02:57:08.173559 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:08.173516    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59985662-39b4-4f17-9cff-d6c05936d0ae-home\") pod \"test-model-6873d4-kserve-55d8c9c849-4lwnj\" (UID: \"59985662-39b4-4f17-9cff-d6c05936d0ae\") " pod="ai-tenant-e2e-inf-b-6873d4/test-model-6873d4-kserve-55d8c9c849-4lwnj"
Jul 22 02:57:08.175286 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:08.175264    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59985662-39b4-4f17-9cff-d6c05936d0ae-dshm\") pod \"test-model-6873d4-kserve-55d8c9c849-4lwnj\" (UID: \"59985662-39b4-4f17-9cff-d6c05936d0ae\") " pod="ai-tenant-e2e-inf-b-6873d4/test-model-6873d4-kserve-55d8c9c849-4lwnj"
Jul 22 02:57:08.175527 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:08.175510    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59985662-39b4-4f17-9cff-d6c05936d0ae-tls-certs\") pod \"test-model-6873d4-kserve-55d8c9c849-4lwnj\" (UID: \"59985662-39b4-4f17-9cff-d6c05936d0ae\") " pod="ai-tenant-e2e-inf-b-6873d4/test-model-6873d4-kserve-55d8c9c849-4lwnj"
Jul 22 02:57:08.182207 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:08.182184    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpp7h\" (UniqueName: \"kubernetes.io/projected/59985662-39b4-4f17-9cff-d6c05936d0ae-kube-api-access-gpp7h\") pod \"test-model-6873d4-kserve-55d8c9c849-4lwnj\" (UID: \"59985662-39b4-4f17-9cff-d6c05936d0ae\") " pod="ai-tenant-e2e-inf-b-6873d4/test-model-6873d4-kserve-55d8c9c849-4lwnj"
Jul 22 02:57:08.356350 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:08.356252    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-6873d4/test-model-6873d4-kserve-55d8c9c849-4lwnj"
Jul 22 02:57:08.501557 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:08.501517    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-6873d4/test-model-6873d4-kserve-55d8c9c849-4lwnj"]
Jul 22 02:57:08.504913 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:57:08.504881    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod59985662_39b4_4f17_9cff_d6c05936d0ae.slice/crio-0b100f2b69a08fde23127a981eca86faad5127a6d2c7026f2964deb300b3f716 WatchSource:0}: Error finding container 0b100f2b69a08fde23127a981eca86faad5127a6d2c7026f2964deb300b3f716: Status 404 returned error can't find the container with id 0b100f2b69a08fde23127a981eca86faad5127a6d2c7026f2964deb300b3f716
Jul 22 02:57:08.839619 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:08.839579    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-6873d4/test-model-6873d4-kserve-55d8c9c849-4lwnj" event={"ID":"59985662-39b4-4f17-9cff-d6c05936d0ae","Type":"ContainerStarted","Data":"752c6f56c743a9744e0b74c582f958de105a1d298be5656fcd6f023c22149795"}
Jul 22 02:57:08.839831 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:08.839634    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-6873d4/test-model-6873d4-kserve-55d8c9c849-4lwnj" event={"ID":"59985662-39b4-4f17-9cff-d6c05936d0ae","Type":"ContainerStarted","Data":"0b100f2b69a08fde23127a981eca86faad5127a6d2c7026f2964deb300b3f716"}
Jul 22 02:57:08.862650 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:08.862594    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-6873d4/test-model-6873d4-kserve-55d8c9c849-4lwnj" podStartSLOduration=0.862578166 podStartE2EDuration="862.578166ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:57:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:57:08.8611402 +0000 UTC m=+2250.401933848" watchObservedRunningTime="2026-07-22 02:57:08.862578166 +0000 UTC m=+2250.403371792"
Jul 22 02:57:12.045074 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:12.045040    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:57:18.356383 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:18.356340    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-6873d4/test-model-6873d4-kserve-55d8c9c849-4lwnj"
Jul 22 02:57:18.356905 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:18.356408    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-6873d4/test-model-6873d4-kserve-55d8c9c849-4lwnj"
Jul 22 02:57:18.368949 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:18.368920    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-6873d4/test-model-6873d4-kserve-55d8c9c849-4lwnj"
Jul 22 02:57:18.908568 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:18.908539    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-6873d4/test-model-6873d4-kserve-55d8c9c849-4lwnj"
Jul 22 02:57:39.395460 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:39.395430    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:57:51.163077 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:51.163041    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-fbb5cc/test-model-fbb5cc-kserve-6ccf4558b7-g8jj4"]
Jul 22 02:57:51.191391 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:51.191355    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:57:52.204920 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:52.204888    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-6873d4/test-model-6873d4-kserve-55d8c9c849-4lwnj"]
Jul 22 02:57:52.218293 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:52.218240    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:57:52.831971 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:52.831920    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:57:52.955037 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:52.954999    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:57:56.247483 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:57:56.247444    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:58:02.717172 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:02.717110    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:58:06.183402 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:06.183363    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-fbb5cc/test-model-fbb5cc-kserve-6ccf4558b7-g8jj4" podUID="dcb7ba03-f407-4bde-81cd-214ea8309daf" containerName="main" containerID="cri-o://b82e22e248f19d19f014b31b303f7f6bb7c0969fa78420e59b3593cc0640599f" gracePeriod=45
Jul 22 02:58:06.442295 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:06.442232    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-fbb5cc/test-model-fbb5cc-kserve-6ccf4558b7-g8jj4"
Jul 22 02:58:06.528390 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:06.528342    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dcb7ba03-f407-4bde-81cd-214ea8309daf-tls-certs\") pod \"dcb7ba03-f407-4bde-81cd-214ea8309daf\" (UID: \"dcb7ba03-f407-4bde-81cd-214ea8309daf\") "
Jul 22 02:58:06.528390 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:06.528394    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dcb7ba03-f407-4bde-81cd-214ea8309daf-home\") pod \"dcb7ba03-f407-4bde-81cd-214ea8309daf\" (UID: \"dcb7ba03-f407-4bde-81cd-214ea8309daf\") "
Jul 22 02:58:06.528665 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:06.528422    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dcb7ba03-f407-4bde-81cd-214ea8309daf-dshm\") pod \"dcb7ba03-f407-4bde-81cd-214ea8309daf\" (UID: \"dcb7ba03-f407-4bde-81cd-214ea8309daf\") "
Jul 22 02:58:06.528665 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:06.528455    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwnsn\" (UniqueName: \"kubernetes.io/projected/dcb7ba03-f407-4bde-81cd-214ea8309daf-kube-api-access-vwnsn\") pod \"dcb7ba03-f407-4bde-81cd-214ea8309daf\" (UID: \"dcb7ba03-f407-4bde-81cd-214ea8309daf\") "
Jul 22 02:58:06.528665 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:06.528520    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dcb7ba03-f407-4bde-81cd-214ea8309daf-tmp-dir\") pod \"dcb7ba03-f407-4bde-81cd-214ea8309daf\" (UID: \"dcb7ba03-f407-4bde-81cd-214ea8309daf\") "
Jul 22 02:58:06.528665 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:06.528542    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dcb7ba03-f407-4bde-81cd-214ea8309daf-model-cache\") pod \"dcb7ba03-f407-4bde-81cd-214ea8309daf\" (UID: \"dcb7ba03-f407-4bde-81cd-214ea8309daf\") "
Jul 22 02:58:06.528916 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:06.528706    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcb7ba03-f407-4bde-81cd-214ea8309daf-home" (OuterVolumeSpecName: "home") pod "dcb7ba03-f407-4bde-81cd-214ea8309daf" (UID: "dcb7ba03-f407-4bde-81cd-214ea8309daf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:58:06.528916 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:06.528859    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcb7ba03-f407-4bde-81cd-214ea8309daf-model-cache" (OuterVolumeSpecName: "model-cache") pod "dcb7ba03-f407-4bde-81cd-214ea8309daf" (UID: "dcb7ba03-f407-4bde-81cd-214ea8309daf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:58:06.528916 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:06.528858    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcb7ba03-f407-4bde-81cd-214ea8309daf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "dcb7ba03-f407-4bde-81cd-214ea8309daf" (UID: "dcb7ba03-f407-4bde-81cd-214ea8309daf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:58:06.528916 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:06.528909    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dcb7ba03-f407-4bde-81cd-214ea8309daf-home\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:58:06.530881 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:06.530857    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dcb7ba03-f407-4bde-81cd-214ea8309daf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dcb7ba03-f407-4bde-81cd-214ea8309daf" (UID: "dcb7ba03-f407-4bde-81cd-214ea8309daf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 02:58:06.530881 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:06.530864    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcb7ba03-f407-4bde-81cd-214ea8309daf-dshm" (OuterVolumeSpecName: "dshm") pod "dcb7ba03-f407-4bde-81cd-214ea8309daf" (UID: "dcb7ba03-f407-4bde-81cd-214ea8309daf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:58:06.531069 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:06.530896    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcb7ba03-f407-4bde-81cd-214ea8309daf-kube-api-access-vwnsn" (OuterVolumeSpecName: "kube-api-access-vwnsn") pod "dcb7ba03-f407-4bde-81cd-214ea8309daf" (UID: "dcb7ba03-f407-4bde-81cd-214ea8309daf"). InnerVolumeSpecName "kube-api-access-vwnsn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 02:58:06.629567 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:06.629527    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dcb7ba03-f407-4bde-81cd-214ea8309daf-tmp-dir\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:58:06.629567 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:06.629560    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dcb7ba03-f407-4bde-81cd-214ea8309daf-model-cache\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:58:06.629567 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:06.629569    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dcb7ba03-f407-4bde-81cd-214ea8309daf-tls-certs\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:58:06.629567 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:06.629577    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dcb7ba03-f407-4bde-81cd-214ea8309daf-dshm\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:58:06.629879 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:06.629585    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwnsn\" (UniqueName: \"kubernetes.io/projected/dcb7ba03-f407-4bde-81cd-214ea8309daf-kube-api-access-vwnsn\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:58:07.132527 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:07.132492    2581 generic.go:356] "Generic (PLEG): container finished" podID="dcb7ba03-f407-4bde-81cd-214ea8309daf" containerID="b82e22e248f19d19f014b31b303f7f6bb7c0969fa78420e59b3593cc0640599f" exitCode=0
Jul 22 02:58:07.132687 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:07.132532    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-fbb5cc/test-model-fbb5cc-kserve-6ccf4558b7-g8jj4"
Jul 22 02:58:07.132687 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:07.132571    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-fbb5cc/test-model-fbb5cc-kserve-6ccf4558b7-g8jj4" event={"ID":"dcb7ba03-f407-4bde-81cd-214ea8309daf","Type":"ContainerDied","Data":"b82e22e248f19d19f014b31b303f7f6bb7c0969fa78420e59b3593cc0640599f"}
Jul 22 02:58:07.132687 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:07.132616    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-fbb5cc/test-model-fbb5cc-kserve-6ccf4558b7-g8jj4" event={"ID":"dcb7ba03-f407-4bde-81cd-214ea8309daf","Type":"ContainerDied","Data":"cb663f945d90c7286890064c627fa9087121d8c949a898a5af778cc4ae827406"}
Jul 22 02:58:07.132687 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:07.132637    2581 scope.go:117] "RemoveContainer" containerID="b82e22e248f19d19f014b31b303f7f6bb7c0969fa78420e59b3593cc0640599f"
Jul 22 02:58:07.142640 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:07.142158    2581 scope.go:117] "RemoveContainer" containerID="b82e22e248f19d19f014b31b303f7f6bb7c0969fa78420e59b3593cc0640599f"
Jul 22 02:58:07.142933 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:58:07.142903    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b82e22e248f19d19f014b31b303f7f6bb7c0969fa78420e59b3593cc0640599f\": container with ID starting with b82e22e248f19d19f014b31b303f7f6bb7c0969fa78420e59b3593cc0640599f not found: ID does not exist" containerID="b82e22e248f19d19f014b31b303f7f6bb7c0969fa78420e59b3593cc0640599f"
Jul 22 02:58:07.143087 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:07.142943    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b82e22e248f19d19f014b31b303f7f6bb7c0969fa78420e59b3593cc0640599f"} err="failed to get container status \"b82e22e248f19d19f014b31b303f7f6bb7c0969fa78420e59b3593cc0640599f\": rpc error: code = NotFound desc = could not find container \"b82e22e248f19d19f014b31b303f7f6bb7c0969fa78420e59b3593cc0640599f\": container with ID starting with b82e22e248f19d19f014b31b303f7f6bb7c0969fa78420e59b3593cc0640599f not found: ID does not exist"
Jul 22 02:58:07.162103 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:07.162022    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-fbb5cc/test-model-fbb5cc-kserve-6ccf4558b7-g8jj4"]
Jul 22 02:58:07.176780 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:07.176753    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-fbb5cc/test-model-fbb5cc-kserve-6ccf4558b7-g8jj4"]
Jul 22 02:58:07.224988 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:07.224952    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-6873d4/test-model-6873d4-kserve-55d8c9c849-4lwnj" podUID="59985662-39b4-4f17-9cff-d6c05936d0ae" containerName="main" containerID="cri-o://752c6f56c743a9744e0b74c582f958de105a1d298be5656fcd6f023c22149795" gracePeriod=45
Jul 22 02:58:07.443905 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:07.443817    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:58:07.489761 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:07.489687    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-6873d4/test-model-6873d4-kserve-55d8c9c849-4lwnj"
Jul 22 02:58:07.538284 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:07.538254    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gpp7h\" (UniqueName: \"kubernetes.io/projected/59985662-39b4-4f17-9cff-d6c05936d0ae-kube-api-access-gpp7h\") pod \"59985662-39b4-4f17-9cff-d6c05936d0ae\" (UID: \"59985662-39b4-4f17-9cff-d6c05936d0ae\") "
Jul 22 02:58:07.538472 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:07.538316    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59985662-39b4-4f17-9cff-d6c05936d0ae-model-cache\") pod \"59985662-39b4-4f17-9cff-d6c05936d0ae\" (UID: \"59985662-39b4-4f17-9cff-d6c05936d0ae\") "
Jul 22 02:58:07.538472 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:07.538345    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59985662-39b4-4f17-9cff-d6c05936d0ae-tmp-dir\") pod \"59985662-39b4-4f17-9cff-d6c05936d0ae\" (UID: \"59985662-39b4-4f17-9cff-d6c05936d0ae\") "
Jul 22 02:58:07.538472 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:07.538425    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59985662-39b4-4f17-9cff-d6c05936d0ae-dshm\") pod \"59985662-39b4-4f17-9cff-d6c05936d0ae\" (UID: \"59985662-39b4-4f17-9cff-d6c05936d0ae\") "
Jul 22 02:58:07.538472 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:07.538467    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59985662-39b4-4f17-9cff-d6c05936d0ae-home\") pod \"59985662-39b4-4f17-9cff-d6c05936d0ae\" (UID: \"59985662-39b4-4f17-9cff-d6c05936d0ae\") "
Jul 22 02:58:07.538690 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:07.538517    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59985662-39b4-4f17-9cff-d6c05936d0ae-tls-certs\") pod \"59985662-39b4-4f17-9cff-d6c05936d0ae\" (UID: \"59985662-39b4-4f17-9cff-d6c05936d0ae\") "
Jul 22 02:58:07.538690 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:07.538647    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59985662-39b4-4f17-9cff-d6c05936d0ae-model-cache" (OuterVolumeSpecName: "model-cache") pod "59985662-39b4-4f17-9cff-d6c05936d0ae" (UID: "59985662-39b4-4f17-9cff-d6c05936d0ae"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:58:07.538832 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:07.538685    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59985662-39b4-4f17-9cff-d6c05936d0ae-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "59985662-39b4-4f17-9cff-d6c05936d0ae" (UID: "59985662-39b4-4f17-9cff-d6c05936d0ae"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:58:07.538832 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:07.538761    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59985662-39b4-4f17-9cff-d6c05936d0ae-home" (OuterVolumeSpecName: "home") pod "59985662-39b4-4f17-9cff-d6c05936d0ae" (UID: "59985662-39b4-4f17-9cff-d6c05936d0ae"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:58:07.538929 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:07.538909    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59985662-39b4-4f17-9cff-d6c05936d0ae-home\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:58:07.538983 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:07.538928    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59985662-39b4-4f17-9cff-d6c05936d0ae-model-cache\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:58:07.538983 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:07.538943    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59985662-39b4-4f17-9cff-d6c05936d0ae-tmp-dir\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:58:07.540728 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:07.540702    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59985662-39b4-4f17-9cff-d6c05936d0ae-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "59985662-39b4-4f17-9cff-d6c05936d0ae" (UID: "59985662-39b4-4f17-9cff-d6c05936d0ae"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 02:58:07.540897 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:07.540766    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59985662-39b4-4f17-9cff-d6c05936d0ae-kube-api-access-gpp7h" (OuterVolumeSpecName: "kube-api-access-gpp7h") pod "59985662-39b4-4f17-9cff-d6c05936d0ae" (UID: "59985662-39b4-4f17-9cff-d6c05936d0ae"). InnerVolumeSpecName "kube-api-access-gpp7h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 02:58:07.540897 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:07.540829    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59985662-39b4-4f17-9cff-d6c05936d0ae-dshm" (OuterVolumeSpecName: "dshm") pod "59985662-39b4-4f17-9cff-d6c05936d0ae" (UID: "59985662-39b4-4f17-9cff-d6c05936d0ae"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:58:07.640477 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:07.640434    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59985662-39b4-4f17-9cff-d6c05936d0ae-dshm\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:58:07.640477 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:07.640470    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59985662-39b4-4f17-9cff-d6c05936d0ae-tls-certs\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:58:07.640477 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:07.640486    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gpp7h\" (UniqueName: \"kubernetes.io/projected/59985662-39b4-4f17-9cff-d6c05936d0ae-kube-api-access-gpp7h\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:58:07.909732 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:07.909690    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:58:08.139724 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:08.139690    2581 generic.go:356] "Generic (PLEG): container finished" podID="59985662-39b4-4f17-9cff-d6c05936d0ae" containerID="752c6f56c743a9744e0b74c582f958de105a1d298be5656fcd6f023c22149795" exitCode=0
Jul 22 02:58:08.139913 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:08.139730    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-6873d4/test-model-6873d4-kserve-55d8c9c849-4lwnj"
Jul 22 02:58:08.139913 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:08.139756    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-6873d4/test-model-6873d4-kserve-55d8c9c849-4lwnj" event={"ID":"59985662-39b4-4f17-9cff-d6c05936d0ae","Type":"ContainerDied","Data":"752c6f56c743a9744e0b74c582f958de105a1d298be5656fcd6f023c22149795"}
Jul 22 02:58:08.139913 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:08.139789    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-6873d4/test-model-6873d4-kserve-55d8c9c849-4lwnj" event={"ID":"59985662-39b4-4f17-9cff-d6c05936d0ae","Type":"ContainerDied","Data":"0b100f2b69a08fde23127a981eca86faad5127a6d2c7026f2964deb300b3f716"}
Jul 22 02:58:08.139913 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:08.139806    2581 scope.go:117] "RemoveContainer" containerID="752c6f56c743a9744e0b74c582f958de105a1d298be5656fcd6f023c22149795"
Jul 22 02:58:08.148851 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:08.148831    2581 scope.go:117] "RemoveContainer" containerID="752c6f56c743a9744e0b74c582f958de105a1d298be5656fcd6f023c22149795"
Jul 22 02:58:08.149118 ip-10-0-138-202 kubenswrapper[2581]: E0722 02:58:08.149100    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"752c6f56c743a9744e0b74c582f958de105a1d298be5656fcd6f023c22149795\": container with ID starting with 752c6f56c743a9744e0b74c582f958de105a1d298be5656fcd6f023c22149795 not found: ID does not exist" containerID="752c6f56c743a9744e0b74c582f958de105a1d298be5656fcd6f023c22149795"
Jul 22 02:58:08.149189 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:08.149128    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"752c6f56c743a9744e0b74c582f958de105a1d298be5656fcd6f023c22149795"} err="failed to get container status \"752c6f56c743a9744e0b74c582f958de105a1d298be5656fcd6f023c22149795\": rpc error: code = NotFound desc = could not find container \"752c6f56c743a9744e0b74c582f958de105a1d298be5656fcd6f023c22149795\": container with ID starting with 752c6f56c743a9744e0b74c582f958de105a1d298be5656fcd6f023c22149795 not found: ID does not exist"
Jul 22 02:58:08.171033 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:08.170968    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-6873d4/test-model-6873d4-kserve-55d8c9c849-4lwnj"]
Jul 22 02:58:08.177010 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:08.176988    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-6873d4/test-model-6873d4-kserve-55d8c9c849-4lwnj"]
Jul 22 02:58:08.904190 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:08.904153    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59985662-39b4-4f17-9cff-d6c05936d0ae" path="/var/lib/kubelet/pods/59985662-39b4-4f17-9cff-d6c05936d0ae/volumes"
Jul 22 02:58:08.904600 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:08.904585    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dcb7ba03-f407-4bde-81cd-214ea8309daf" path="/var/lib/kubelet/pods/dcb7ba03-f407-4bde-81cd-214ea8309daf/volumes"
Jul 22 02:58:11.256981 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:11.256947    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:58:14.519770 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:14.519719    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:58:15.978648 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:15.978609    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:58:16.081563 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:16.081532    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:58:25.622544 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:25.622507    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:58:28.349432 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:28.349390    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:58:30.288597 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:30.288562    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:58:30.691650 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:30.691570    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:58:36.978551 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:36.978478    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:58:37.060610 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:37.060573    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:58:37.296009 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:37.295974    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:58:37.522011 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:37.521981    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:58:37.864010 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:37.863971    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:58:38.221692 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:38.221615    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:58:38.614165 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:38.614132    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:58:41.477071 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:41.477036    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:58:41.612969 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:41.612933    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:58:41.834470 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:41.834434    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:58:42.107432 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:42.107347    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:58:42.368422 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:42.368342    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:58:42.469839 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:42.469809    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:58:45.983885 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:45.983847    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:58:46.488120 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:46.488084    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:58:46.803360 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:46.803275    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:58:47.250448 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:47.250417    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:58:48.457830 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:48.457795    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:58:49.560726 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:49.560697    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:58:49.654828 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:49.654797    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:58:49.845159 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:49.845067    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:58:50.021809 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:50.021763    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:58:50.505545 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:50.505513    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:58:50.606657 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:50.606625    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:58:54.076155 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:58:54.076101    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:59:02.113761 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:02.113714    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3f8fef-mg7pr"]
Jul 22 02:59:02.114231 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:02.114117    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcb7ba03-f407-4bde-81cd-214ea8309daf" containerName="main"
Jul 22 02:59:02.114231 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:02.114129    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcb7ba03-f407-4bde-81cd-214ea8309daf" containerName="main"
Jul 22 02:59:02.114231 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:02.114143    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59985662-39b4-4f17-9cff-d6c05936d0ae" containerName="main"
Jul 22 02:59:02.114231 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:02.114149    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="59985662-39b4-4f17-9cff-d6c05936d0ae" containerName="main"
Jul 22 02:59:02.114231 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:02.114207    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcb7ba03-f407-4bde-81cd-214ea8309daf" containerName="main"
Jul 22 02:59:02.114231 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:02.114218    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="59985662-39b4-4f17-9cff-d6c05936d0ae" containerName="main"
Jul 22 02:59:02.117454 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:02.117438    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3f8fef-mg7pr"
Jul 22 02:59:02.120804 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:02.120784    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 02:59:02.121085 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:02.121067    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rd6jc\""
Jul 22 02:59:02.125943 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:02.125924    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3f8fef-mg7pr"]
Jul 22 02:59:02.263560 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:02.263523    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b24d9b5-374e-4742-a11b-c1f13195ea03-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-3f8fef-mg7pr\" (UID: \"4b24d9b5-374e-4742-a11b-c1f13195ea03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3f8fef-mg7pr"
Jul 22 02:59:02.364712 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:02.364630    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b24d9b5-374e-4742-a11b-c1f13195ea03-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-3f8fef-mg7pr\" (UID: \"4b24d9b5-374e-4742-a11b-c1f13195ea03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3f8fef-mg7pr"
Jul 22 02:59:02.365289 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:02.365269    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b24d9b5-374e-4742-a11b-c1f13195ea03-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-3f8fef-mg7pr\" (UID: \"4b24d9b5-374e-4742-a11b-c1f13195ea03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3f8fef-mg7pr"
Jul 22 02:59:02.429021 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:02.428988    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3f8fef-mg7pr"
Jul 22 02:59:02.567282 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:02.567259    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3f8fef-mg7pr"]
Jul 22 02:59:02.569612 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:59:02.569587    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b24d9b5_374e_4742_a11b_c1f13195ea03.slice/crio-a5db8d25a7b51c3f0a9c0d038884ad6d2526733ec0839b02152bca5907a7c445 WatchSource:0}: Error finding container a5db8d25a7b51c3f0a9c0d038884ad6d2526733ec0839b02152bca5907a7c445: Status 404 returned error can't find the container with id a5db8d25a7b51c3f0a9c0d038884ad6d2526733ec0839b02152bca5907a7c445
Jul 22 02:59:02.571586 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:02.571570    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 02:59:02.749513 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:02.749478    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:59:02.849300 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:02.849260    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:59:03.415527 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:03.415493    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3f8fef-mg7pr" event={"ID":"4b24d9b5-374e-4742-a11b-c1f13195ea03","Type":"ContainerStarted","Data":"a5db8d25a7b51c3f0a9c0d038884ad6d2526733ec0839b02152bca5907a7c445"}
Jul 22 02:59:06.432460 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:06.432417    2581 generic.go:356] "Generic (PLEG): container finished" podID="4b24d9b5-374e-4742-a11b-c1f13195ea03" containerID="da7b4647cce7485a8bbf62569189e8e8302cc0f2352be8f4a33fa1a1573234fb" exitCode=0
Jul 22 02:59:06.432881 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:06.432505    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3f8fef-mg7pr" event={"ID":"4b24d9b5-374e-4742-a11b-c1f13195ea03","Type":"ContainerDied","Data":"da7b4647cce7485a8bbf62569189e8e8302cc0f2352be8f4a33fa1a1573234fb"}
Jul 22 02:59:06.972556 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:06.972522    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:59:07.566504 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:07.566475    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3f8fef-mg7pr"
Jul 22 02:59:07.612919 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:07.612885    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b24d9b5-374e-4742-a11b-c1f13195ea03-maas-api-service-ca\") pod \"4b24d9b5-374e-4742-a11b-c1f13195ea03\" (UID: \"4b24d9b5-374e-4742-a11b-c1f13195ea03\") "
Jul 22 02:59:07.613245 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:07.613224    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b24d9b5-374e-4742-a11b-c1f13195ea03-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4b24d9b5-374e-4742-a11b-c1f13195ea03" (UID: "4b24d9b5-374e-4742-a11b-c1f13195ea03"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 02:59:07.713688 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:07.713596    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b24d9b5-374e-4742-a11b-c1f13195ea03-maas-api-service-ca\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:59:08.443107 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:08.443078    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3f8fef-mg7pr"
Jul 22 02:59:08.443309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:08.443108    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3f8fef-mg7pr" event={"ID":"4b24d9b5-374e-4742-a11b-c1f13195ea03","Type":"ContainerDied","Data":"a5db8d25a7b51c3f0a9c0d038884ad6d2526733ec0839b02152bca5907a7c445"}
Jul 22 02:59:08.443309 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:08.443152    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a5db8d25a7b51c3f0a9c0d038884ad6d2526733ec0839b02152bca5907a7c445"
Jul 22 02:59:11.414758 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:11.414707    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:59:11.515637 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:11.515584    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:59:12.796894 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:12.796861    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:59:12.893970 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:12.893936    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:59:18.107034 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:18.106963    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:59:22.215698 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:22.215657    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3f8fef-mg7pr"]
Jul 22 02:59:22.216313 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:22.216274    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:59:22.218946 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:22.218919    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3f8fef-mg7pr"]
Jul 22 02:59:22.327218 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:22.327187    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:59:22.606370 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:22.606335    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:59:22.904266 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:22.904173    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b24d9b5-374e-4742-a11b-c1f13195ea03" path="/var/lib/kubelet/pods/4b24d9b5-374e-4742-a11b-c1f13195ea03/volumes"
Jul 22 02:59:23.373036 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:23.372998    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:59:29.074792 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:29.074717    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:59:29.155180 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:29.155145    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:59:29.361011 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:29.360896    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:59:29.584451 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:29.584389    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:59:29.708852 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:29.708733    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dac71a-j4p2x"]
Jul 22 02:59:29.709279 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:29.709249    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b24d9b5-374e-4742-a11b-c1f13195ea03" containerName="revoke-keys"
Jul 22 02:59:29.709279 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:29.709275    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b24d9b5-374e-4742-a11b-c1f13195ea03" containerName="revoke-keys"
Jul 22 02:59:29.709471 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:29.709372    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b24d9b5-374e-4742-a11b-c1f13195ea03" containerName="revoke-keys"
Jul 22 02:59:29.712728 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:29.712709    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dac71a-j4p2x"
Jul 22 02:59:29.716554 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:29.716532    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 02:59:29.716648 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:29.716589    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rd6jc\""
Jul 22 02:59:29.721169 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:29.721148    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dac71a-j4p2x"]
Jul 22 02:59:29.831318 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:29.831284    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2efb6d04-5b6e-4889-ad2c-9d8f06c77421-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-dac71a-j4p2x\" (UID: \"2efb6d04-5b6e-4889-ad2c-9d8f06c77421\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dac71a-j4p2x"
Jul 22 02:59:29.932750 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:29.932709    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2efb6d04-5b6e-4889-ad2c-9d8f06c77421-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-dac71a-j4p2x\" (UID: \"2efb6d04-5b6e-4889-ad2c-9d8f06c77421\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dac71a-j4p2x"
Jul 22 02:59:29.933361 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:29.933339    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2efb6d04-5b6e-4889-ad2c-9d8f06c77421-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-dac71a-j4p2x\" (UID: \"2efb6d04-5b6e-4889-ad2c-9d8f06c77421\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dac71a-j4p2x"
Jul 22 02:59:30.024256 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:30.024212    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dac71a-j4p2x"
Jul 22 02:59:30.167141 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:30.167116    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dac71a-j4p2x"]
Jul 22 02:59:30.168794 ip-10-0-138-202 kubenswrapper[2581]: W0722 02:59:30.168767    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2efb6d04_5b6e_4889_ad2c_9d8f06c77421.slice/crio-e77cddc0525087300f035d8307ec10947a5b952ee3f8634f588743de7f03e92a WatchSource:0}: Error finding container e77cddc0525087300f035d8307ec10947a5b952ee3f8634f588743de7f03e92a: Status 404 returned error can't find the container with id e77cddc0525087300f035d8307ec10947a5b952ee3f8634f588743de7f03e92a
Jul 22 02:59:30.264699 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:30.264650    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:59:30.365619 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:30.365578    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:59:30.552347 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:30.552314    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dac71a-j4p2x" event={"ID":"2efb6d04-5b6e-4889-ad2c-9d8f06c77421","Type":"ContainerStarted","Data":"640e8e7c69a164ad6b5291e98ee1fe97feb78509c6afa8f68b6fe86ff1e67242"}
Jul 22 02:59:30.552522 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:30.552369    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dac71a-j4p2x" event={"ID":"2efb6d04-5b6e-4889-ad2c-9d8f06c77421","Type":"ContainerStarted","Data":"e77cddc0525087300f035d8307ec10947a5b952ee3f8634f588743de7f03e92a"}
Jul 22 02:59:30.571638 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:30.571584    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dac71a-j4p2x" podStartSLOduration=1.571568746 podStartE2EDuration="1.571568746s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:59:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:59:30.571405109 +0000 UTC m=+2392.112198737" watchObservedRunningTime="2026-07-22 02:59:30.571568746 +0000 UTC m=+2392.112362399"
Jul 22 02:59:31.558782 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:31.558727    2581 generic.go:356] "Generic (PLEG): container finished" podID="2efb6d04-5b6e-4889-ad2c-9d8f06c77421" containerID="640e8e7c69a164ad6b5291e98ee1fe97feb78509c6afa8f68b6fe86ff1e67242" exitCode=0
Jul 22 02:59:31.559170 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:31.558778    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dac71a-j4p2x" event={"ID":"2efb6d04-5b6e-4889-ad2c-9d8f06c77421","Type":"ContainerDied","Data":"640e8e7c69a164ad6b5291e98ee1fe97feb78509c6afa8f68b6fe86ff1e67242"}
Jul 22 02:59:32.701472 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:32.701444    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dac71a-j4p2x"
Jul 22 02:59:32.862905 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:32.862814    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2efb6d04-5b6e-4889-ad2c-9d8f06c77421-maas-api-service-ca\") pod \"2efb6d04-5b6e-4889-ad2c-9d8f06c77421\" (UID: \"2efb6d04-5b6e-4889-ad2c-9d8f06c77421\") "
Jul 22 02:59:32.863179 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:32.863157    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2efb6d04-5b6e-4889-ad2c-9d8f06c77421-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2efb6d04-5b6e-4889-ad2c-9d8f06c77421" (UID: "2efb6d04-5b6e-4889-ad2c-9d8f06c77421"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 02:59:32.963565 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:32.963528    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2efb6d04-5b6e-4889-ad2c-9d8f06c77421-maas-api-service-ca\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 02:59:33.570197 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:33.570168    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dac71a-j4p2x"
Jul 22 02:59:33.570382 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:33.570207    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dac71a-j4p2x" event={"ID":"2efb6d04-5b6e-4889-ad2c-9d8f06c77421","Type":"ContainerDied","Data":"e77cddc0525087300f035d8307ec10947a5b952ee3f8634f588743de7f03e92a"}
Jul 22 02:59:33.570382 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:33.570254    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e77cddc0525087300f035d8307ec10947a5b952ee3f8634f588743de7f03e92a"
Jul 22 02:59:43.877250 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:43.877210    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:59:45.237048 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:45.237011    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:59:49.786621 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:49.786589    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dac71a-j4p2x"]
Jul 22 02:59:49.788677 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:49.788651    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dac71a-j4p2x"]
Jul 22 02:59:49.791080 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:49.791060    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:59:50.003524 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:50.003487    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 02:59:50.207012 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:50.206927    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:59:50.904420 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:50.904388    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2efb6d04-5b6e-4889-ad2c-9d8f06c77421" path="/var/lib/kubelet/pods/2efb6d04-5b6e-4889-ad2c-9d8f06c77421/volumes"
Jul 22 02:59:57.602145 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:57.602111    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:59:57.734954 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:57.734920    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:59:58.064330 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:58.064295    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:59:58.280767 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:58.280689    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:59:58.580749 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:58.580716    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:59:58.974590 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:58.974508    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 02:59:59.329873 ip-10-0-138-202 kubenswrapper[2581]: I0722 02:59:59.329844    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:00:01.955547 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:01.955467    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:00:02.228875 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:02.228788    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:00:02.481118 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:02.481034    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:00:02.821263 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:02.821230    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:00:03.157001 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:03.156914    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:00:03.244231 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:03.244203    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:00:05.623492 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:05.623451    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:00:05.718274 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:05.718243    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:00:06.408371 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:06.408340    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:00:06.800273 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:06.800240    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:00:07.431917 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:07.431882    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:00:07.556091 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:07.556041    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:00:08.091969 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:08.091936    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 03:00:10.216864 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:10.216823    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:00:10.359945 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:10.359914    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:00:10.457995 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:10.457967    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:00:10.759810 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:10.759778    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:00:11.279819 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:11.279787    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:00:11.472167 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:11.472135    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:00:14.512992 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:14.512957    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:00:18.480526 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:18.480489    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 03:00:25.017460 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:25.017426    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:00:25.129800 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:25.129766    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:00:28.086628 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:28.086594    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-995f8f/auth-test-model-995f8f-kserve-bc6d9d84c-28k5n"]
Jul 22 03:00:28.087116 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:28.087093    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2efb6d04-5b6e-4889-ad2c-9d8f06c77421" containerName="revoke-keys"
Jul 22 03:00:28.087116 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:28.087116    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2efb6d04-5b6e-4889-ad2c-9d8f06c77421" containerName="revoke-keys"
Jul 22 03:00:28.087535 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:28.087205    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2efb6d04-5b6e-4889-ad2c-9d8f06c77421" containerName="revoke-keys"
Jul 22 03:00:28.094055 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:28.092582    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-995f8f/auth-test-model-995f8f-kserve-bc6d9d84c-28k5n"
Jul 22 03:00:28.095985 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:28.095959    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-995f8f\"/\"auth-test-model-995f8f-kserve-self-signed-certs\""
Jul 22 03:00:28.096992 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:28.096972    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-995f8f\"/\"kube-root-ca.crt\""
Jul 22 03:00:28.097128 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:28.097071    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-995f8f\"/\"default-dockercfg-mkrhb\""
Jul 22 03:00:28.097254 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:28.097153    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-995f8f\"/\"openshift-service-ca.crt\""
Jul 22 03:00:28.103536 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:28.103513    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-995f8f/auth-test-model-995f8f-kserve-bc6d9d84c-28k5n"]
Jul 22 03:00:28.117388 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:28.117352    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:00:28.166645 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:28.166614    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c540d47f-4cb7-443b-84d9-efcf7f731b35-model-cache\") pod \"auth-test-model-995f8f-kserve-bc6d9d84c-28k5n\" (UID: \"c540d47f-4cb7-443b-84d9-efcf7f731b35\") " pod="ai-tenant-e2e-shared-a-995f8f/auth-test-model-995f8f-kserve-bc6d9d84c-28k5n"
Jul 22 03:00:28.166800 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:28.166654    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c540d47f-4cb7-443b-84d9-efcf7f731b35-dshm\") pod \"auth-test-model-995f8f-kserve-bc6d9d84c-28k5n\" (UID: \"c540d47f-4cb7-443b-84d9-efcf7f731b35\") " pod="ai-tenant-e2e-shared-a-995f8f/auth-test-model-995f8f-kserve-bc6d9d84c-28k5n"
Jul 22 03:00:28.166800 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:28.166669    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c540d47f-4cb7-443b-84d9-efcf7f731b35-tls-certs\") pod \"auth-test-model-995f8f-kserve-bc6d9d84c-28k5n\" (UID: \"c540d47f-4cb7-443b-84d9-efcf7f731b35\") " pod="ai-tenant-e2e-shared-a-995f8f/auth-test-model-995f8f-kserve-bc6d9d84c-28k5n"
Jul 22 03:00:28.166800 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:28.166768    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c540d47f-4cb7-443b-84d9-efcf7f731b35-tmp-dir\") pod \"auth-test-model-995f8f-kserve-bc6d9d84c-28k5n\" (UID: \"c540d47f-4cb7-443b-84d9-efcf7f731b35\") " pod="ai-tenant-e2e-shared-a-995f8f/auth-test-model-995f8f-kserve-bc6d9d84c-28k5n"
Jul 22 03:00:28.166913 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:28.166808    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptcdz\" (UniqueName: \"kubernetes.io/projected/c540d47f-4cb7-443b-84d9-efcf7f731b35-kube-api-access-ptcdz\") pod \"auth-test-model-995f8f-kserve-bc6d9d84c-28k5n\" (UID: \"c540d47f-4cb7-443b-84d9-efcf7f731b35\") " pod="ai-tenant-e2e-shared-a-995f8f/auth-test-model-995f8f-kserve-bc6d9d84c-28k5n"
Jul 22 03:00:28.166913 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:28.166853    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c540d47f-4cb7-443b-84d9-efcf7f731b35-home\") pod \"auth-test-model-995f8f-kserve-bc6d9d84c-28k5n\" (UID: \"c540d47f-4cb7-443b-84d9-efcf7f731b35\") " pod="ai-tenant-e2e-shared-a-995f8f/auth-test-model-995f8f-kserve-bc6d9d84c-28k5n"
Jul 22 03:00:28.267903 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:28.267849    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptcdz\" (UniqueName: \"kubernetes.io/projected/c540d47f-4cb7-443b-84d9-efcf7f731b35-kube-api-access-ptcdz\") pod \"auth-test-model-995f8f-kserve-bc6d9d84c-28k5n\" (UID: \"c540d47f-4cb7-443b-84d9-efcf7f731b35\") " pod="ai-tenant-e2e-shared-a-995f8f/auth-test-model-995f8f-kserve-bc6d9d84c-28k5n"
Jul 22 03:00:28.268086 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:28.267937    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c540d47f-4cb7-443b-84d9-efcf7f731b35-home\") pod \"auth-test-model-995f8f-kserve-bc6d9d84c-28k5n\" (UID: \"c540d47f-4cb7-443b-84d9-efcf7f731b35\") " pod="ai-tenant-e2e-shared-a-995f8f/auth-test-model-995f8f-kserve-bc6d9d84c-28k5n"
Jul 22 03:00:28.268086 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:28.268022    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c540d47f-4cb7-443b-84d9-efcf7f731b35-model-cache\") pod \"auth-test-model-995f8f-kserve-bc6d9d84c-28k5n\" (UID: \"c540d47f-4cb7-443b-84d9-efcf7f731b35\") " pod="ai-tenant-e2e-shared-a-995f8f/auth-test-model-995f8f-kserve-bc6d9d84c-28k5n"
Jul 22 03:00:28.268086 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:28.268049    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c540d47f-4cb7-443b-84d9-efcf7f731b35-dshm\") pod \"auth-test-model-995f8f-kserve-bc6d9d84c-28k5n\" (UID: \"c540d47f-4cb7-443b-84d9-efcf7f731b35\") " pod="ai-tenant-e2e-shared-a-995f8f/auth-test-model-995f8f-kserve-bc6d9d84c-28k5n"
Jul 22 03:00:28.268086 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:28.268065    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c540d47f-4cb7-443b-84d9-efcf7f731b35-tls-certs\") pod \"auth-test-model-995f8f-kserve-bc6d9d84c-28k5n\" (UID: \"c540d47f-4cb7-443b-84d9-efcf7f731b35\") " pod="ai-tenant-e2e-shared-a-995f8f/auth-test-model-995f8f-kserve-bc6d9d84c-28k5n"
Jul 22 03:00:28.268254 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:28.268104    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c540d47f-4cb7-443b-84d9-efcf7f731b35-tmp-dir\") pod \"auth-test-model-995f8f-kserve-bc6d9d84c-28k5n\" (UID: \"c540d47f-4cb7-443b-84d9-efcf7f731b35\") " pod="ai-tenant-e2e-shared-a-995f8f/auth-test-model-995f8f-kserve-bc6d9d84c-28k5n"
Jul 22 03:00:28.268379 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:28.268360    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c540d47f-4cb7-443b-84d9-efcf7f731b35-home\") pod \"auth-test-model-995f8f-kserve-bc6d9d84c-28k5n\" (UID: \"c540d47f-4cb7-443b-84d9-efcf7f731b35\") " pod="ai-tenant-e2e-shared-a-995f8f/auth-test-model-995f8f-kserve-bc6d9d84c-28k5n"
Jul 22 03:00:28.268448 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:28.268427    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c540d47f-4cb7-443b-84d9-efcf7f731b35-model-cache\") pod \"auth-test-model-995f8f-kserve-bc6d9d84c-28k5n\" (UID: \"c540d47f-4cb7-443b-84d9-efcf7f731b35\") " pod="ai-tenant-e2e-shared-a-995f8f/auth-test-model-995f8f-kserve-bc6d9d84c-28k5n"
Jul 22 03:00:28.268526 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:28.268502    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c540d47f-4cb7-443b-84d9-efcf7f731b35-tmp-dir\") pod \"auth-test-model-995f8f-kserve-bc6d9d84c-28k5n\" (UID: \"c540d47f-4cb7-443b-84d9-efcf7f731b35\") " pod="ai-tenant-e2e-shared-a-995f8f/auth-test-model-995f8f-kserve-bc6d9d84c-28k5n"
Jul 22 03:00:28.270358 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:28.270340    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c540d47f-4cb7-443b-84d9-efcf7f731b35-dshm\") pod \"auth-test-model-995f8f-kserve-bc6d9d84c-28k5n\" (UID: \"c540d47f-4cb7-443b-84d9-efcf7f731b35\") " pod="ai-tenant-e2e-shared-a-995f8f/auth-test-model-995f8f-kserve-bc6d9d84c-28k5n"
Jul 22 03:00:28.270519 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:28.270501    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c540d47f-4cb7-443b-84d9-efcf7f731b35-tls-certs\") pod \"auth-test-model-995f8f-kserve-bc6d9d84c-28k5n\" (UID: \"c540d47f-4cb7-443b-84d9-efcf7f731b35\") " pod="ai-tenant-e2e-shared-a-995f8f/auth-test-model-995f8f-kserve-bc6d9d84c-28k5n"
Jul 22 03:00:28.278049 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:28.278027    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptcdz\" (UniqueName: \"kubernetes.io/projected/c540d47f-4cb7-443b-84d9-efcf7f731b35-kube-api-access-ptcdz\") pod \"auth-test-model-995f8f-kserve-bc6d9d84c-28k5n\" (UID: \"c540d47f-4cb7-443b-84d9-efcf7f731b35\") " pod="ai-tenant-e2e-shared-a-995f8f/auth-test-model-995f8f-kserve-bc6d9d84c-28k5n"
Jul 22 03:00:28.405319 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:28.405232    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-995f8f/auth-test-model-995f8f-kserve-bc6d9d84c-28k5n"
Jul 22 03:00:28.548124 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:28.548100    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-995f8f/auth-test-model-995f8f-kserve-bc6d9d84c-28k5n"]
Jul 22 03:00:28.550547 ip-10-0-138-202 kubenswrapper[2581]: W0722 03:00:28.550519    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc540d47f_4cb7_443b_84d9_efcf7f731b35.slice/crio-f1fa89f2bf3942bb1442c7ca80343309e35b443502a5c27d6aab02a3c8c898b9 WatchSource:0}: Error finding container f1fa89f2bf3942bb1442c7ca80343309e35b443502a5c27d6aab02a3c8c898b9: Status 404 returned error can't find the container with id f1fa89f2bf3942bb1442c7ca80343309e35b443502a5c27d6aab02a3c8c898b9
Jul 22 03:00:28.840619 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:28.840584    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-995f8f/auth-test-model-995f8f-kserve-bc6d9d84c-28k5n" event={"ID":"c540d47f-4cb7-443b-84d9-efcf7f731b35","Type":"ContainerStarted","Data":"6bec6e20c6b1386790c1963feb6fd700a81f8c98aa98f19df1d9723d921dd2c1"}
Jul 22 03:00:28.840619 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:28.840625    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-995f8f/auth-test-model-995f8f-kserve-bc6d9d84c-28k5n" event={"ID":"c540d47f-4cb7-443b-84d9-efcf7f731b35","Type":"ContainerStarted","Data":"f1fa89f2bf3942bb1442c7ca80343309e35b443502a5c27d6aab02a3c8c898b9"}
Jul 22 03:00:28.860927 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:28.860880    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-995f8f/auth-test-model-995f8f-kserve-bc6d9d84c-28k5n" podStartSLOduration=0.860865806 podStartE2EDuration="860.865806ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:00:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:00:28.858981327 +0000 UTC m=+2450.399774966" watchObservedRunningTime="2026-07-22 03:00:28.860865806 +0000 UTC m=+2450.401659439"
Jul 22 03:00:32.495876 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:32.495834    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:00:32.646965 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:32.646915    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:00:34.956585 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:34.956549    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 03:00:38.405935 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:38.405891    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-995f8f/auth-test-model-995f8f-kserve-bc6d9d84c-28k5n"
Jul 22 03:00:38.406332 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:38.405950    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-995f8f/auth-test-model-995f8f-kserve-bc6d9d84c-28k5n"
Jul 22 03:00:38.418415 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:38.418383    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-995f8f/auth-test-model-995f8f-kserve-bc6d9d84c-28k5n"
Jul 22 03:00:38.904160 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:00:38.904131    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-995f8f/auth-test-model-995f8f-kserve-bc6d9d84c-28k5n"
Jul 22 03:01:10.459643 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:01:10.459608    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-bb8c9d/auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q"]
Jul 22 03:01:10.463244 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:01:10.463229    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bb8c9d/auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q"
Jul 22 03:01:10.467416 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:01:10.467385    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-bb8c9d\"/\"auth-test-model-bb8c9d-kserve-self-signed-certs\""
Jul 22 03:01:10.467533 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:01:10.467431    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-bb8c9d\"/\"default-dockercfg-mbfrf\""
Jul 22 03:01:10.467533 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:01:10.467431    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-bb8c9d\"/\"openshift-service-ca.crt\""
Jul 22 03:01:10.467533 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:01:10.467400    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-bb8c9d\"/\"kube-root-ca.crt\""
Jul 22 03:01:10.477224 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:01:10.477201    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-bb8c9d/auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q"]
Jul 22 03:01:10.489435 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:01:10.489413    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:01:10.564526 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:01:10.564491    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6b13fd7-13c4-497c-a1da-21f576f4d6de-tls-certs\") pod \"auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q\" (UID: \"f6b13fd7-13c4-497c-a1da-21f576f4d6de\") " pod="ai-tenant-e2e-shared-b-bb8c9d/auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q"
Jul 22 03:01:10.564694 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:01:10.564550    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc977\" (UniqueName: \"kubernetes.io/projected/f6b13fd7-13c4-497c-a1da-21f576f4d6de-kube-api-access-wc977\") pod \"auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q\" (UID: \"f6b13fd7-13c4-497c-a1da-21f576f4d6de\") " pod="ai-tenant-e2e-shared-b-bb8c9d/auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q"
Jul 22 03:01:10.564694 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:01:10.564577    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6b13fd7-13c4-497c-a1da-21f576f4d6de-dshm\") pod \"auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q\" (UID: \"f6b13fd7-13c4-497c-a1da-21f576f4d6de\") " pod="ai-tenant-e2e-shared-b-bb8c9d/auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q"
Jul 22 03:01:10.564694 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:01:10.564593    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6b13fd7-13c4-497c-a1da-21f576f4d6de-tmp-dir\") pod \"auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q\" (UID: \"f6b13fd7-13c4-497c-a1da-21f576f4d6de\") " pod="ai-tenant-e2e-shared-b-bb8c9d/auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q"
Jul 22 03:01:10.564694 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:01:10.564619    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6b13fd7-13c4-497c-a1da-21f576f4d6de-home\") pod \"auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q\" (UID: \"f6b13fd7-13c4-497c-a1da-21f576f4d6de\") " pod="ai-tenant-e2e-shared-b-bb8c9d/auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q"
Jul 22 03:01:10.564694 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:01:10.564682    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6b13fd7-13c4-497c-a1da-21f576f4d6de-model-cache\") pod \"auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q\" (UID: \"f6b13fd7-13c4-497c-a1da-21f576f4d6de\") " pod="ai-tenant-e2e-shared-b-bb8c9d/auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q"
Jul 22 03:01:10.665908 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:01:10.665869    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6b13fd7-13c4-497c-a1da-21f576f4d6de-tls-certs\") pod \"auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q\" (UID: \"f6b13fd7-13c4-497c-a1da-21f576f4d6de\") " pod="ai-tenant-e2e-shared-b-bb8c9d/auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q"
Jul 22 03:01:10.666080 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:01:10.665939    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc977\" (UniqueName: \"kubernetes.io/projected/f6b13fd7-13c4-497c-a1da-21f576f4d6de-kube-api-access-wc977\") pod \"auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q\" (UID: \"f6b13fd7-13c4-497c-a1da-21f576f4d6de\") " pod="ai-tenant-e2e-shared-b-bb8c9d/auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q"
Jul 22 03:01:10.666080 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:01:10.665966    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6b13fd7-13c4-497c-a1da-21f576f4d6de-dshm\") pod \"auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q\" (UID: \"f6b13fd7-13c4-497c-a1da-21f576f4d6de\") " pod="ai-tenant-e2e-shared-b-bb8c9d/auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q"
Jul 22 03:01:10.666080 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:01:10.665989    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6b13fd7-13c4-497c-a1da-21f576f4d6de-tmp-dir\") pod \"auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q\" (UID: \"f6b13fd7-13c4-497c-a1da-21f576f4d6de\") " pod="ai-tenant-e2e-shared-b-bb8c9d/auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q"
Jul 22 03:01:10.666080 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:01:10.666024    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6b13fd7-13c4-497c-a1da-21f576f4d6de-home\") pod \"auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q\" (UID: \"f6b13fd7-13c4-497c-a1da-21f576f4d6de\") " pod="ai-tenant-e2e-shared-b-bb8c9d/auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q"
Jul 22 03:01:10.666080 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:01:10.666057    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6b13fd7-13c4-497c-a1da-21f576f4d6de-model-cache\") pod \"auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q\" (UID: \"f6b13fd7-13c4-497c-a1da-21f576f4d6de\") " pod="ai-tenant-e2e-shared-b-bb8c9d/auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q"
Jul 22 03:01:10.666471 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:01:10.666452    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6b13fd7-13c4-497c-a1da-21f576f4d6de-home\") pod \"auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q\" (UID: \"f6b13fd7-13c4-497c-a1da-21f576f4d6de\") " pod="ai-tenant-e2e-shared-b-bb8c9d/auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q"
Jul 22 03:01:10.666534 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:01:10.666484    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6b13fd7-13c4-497c-a1da-21f576f4d6de-model-cache\") pod \"auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q\" (UID: \"f6b13fd7-13c4-497c-a1da-21f576f4d6de\") " pod="ai-tenant-e2e-shared-b-bb8c9d/auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q"
Jul 22 03:01:10.666587 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:01:10.666542    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6b13fd7-13c4-497c-a1da-21f576f4d6de-tmp-dir\") pod \"auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q\" (UID: \"f6b13fd7-13c4-497c-a1da-21f576f4d6de\") " pod="ai-tenant-e2e-shared-b-bb8c9d/auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q"
Jul 22 03:01:10.668229 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:01:10.668209    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6b13fd7-13c4-497c-a1da-21f576f4d6de-dshm\") pod \"auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q\" (UID: \"f6b13fd7-13c4-497c-a1da-21f576f4d6de\") " pod="ai-tenant-e2e-shared-b-bb8c9d/auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q"
Jul 22 03:01:10.668536 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:01:10.668518    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6b13fd7-13c4-497c-a1da-21f576f4d6de-tls-certs\") pod \"auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q\" (UID: \"f6b13fd7-13c4-497c-a1da-21f576f4d6de\") " pod="ai-tenant-e2e-shared-b-bb8c9d/auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q"
Jul 22 03:01:10.676935 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:01:10.676915    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc977\" (UniqueName: \"kubernetes.io/projected/f6b13fd7-13c4-497c-a1da-21f576f4d6de-kube-api-access-wc977\") pod \"auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q\" (UID: \"f6b13fd7-13c4-497c-a1da-21f576f4d6de\") " pod="ai-tenant-e2e-shared-b-bb8c9d/auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q"
Jul 22 03:01:10.773878 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:01:10.773847    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bb8c9d/auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q"
Jul 22 03:01:10.907087 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:01:10.907052    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-bb8c9d/auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q"]
Jul 22 03:01:10.909918 ip-10-0-138-202 kubenswrapper[2581]: W0722 03:01:10.909893    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6b13fd7_13c4_497c_a1da_21f576f4d6de.slice/crio-ac77ba4bcb903fab6abfb795254634f5147fbaeedb42b6500900b1b63b0552cb WatchSource:0}: Error finding container ac77ba4bcb903fab6abfb795254634f5147fbaeedb42b6500900b1b63b0552cb: Status 404 returned error can't find the container with id ac77ba4bcb903fab6abfb795254634f5147fbaeedb42b6500900b1b63b0552cb
Jul 22 03:01:11.045286 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:01:11.045194    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bb8c9d/auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q" event={"ID":"f6b13fd7-13c4-497c-a1da-21f576f4d6de","Type":"ContainerStarted","Data":"c41e2ab03d0985cacc33b7e82f404ebe0a8d003114c3ae1c3f97b723b89d6395"}
Jul 22 03:01:11.045286 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:01:11.045240    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bb8c9d/auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q" event={"ID":"f6b13fd7-13c4-497c-a1da-21f576f4d6de","Type":"ContainerStarted","Data":"ac77ba4bcb903fab6abfb795254634f5147fbaeedb42b6500900b1b63b0552cb"}
Jul 22 03:01:11.066513 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:01:11.066463    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-bb8c9d/auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q" podStartSLOduration=1.066447881 podStartE2EDuration="1.066447881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:01:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:01:11.065262194 +0000 UTC m=+2492.606055820" watchObservedRunningTime="2026-07-22 03:01:11.066447881 +0000 UTC m=+2492.607241506"
Jul 22 03:01:14.045146 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:01:14.045114    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 03:01:20.774991 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:01:20.774955    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-bb8c9d/auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q"
Jul 22 03:01:20.775382 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:01:20.775006    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-bb8c9d/auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q"
Jul 22 03:01:20.787580 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:01:20.787553    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-bb8c9d/auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q"
Jul 22 03:01:21.103478 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:01:21.103398    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-bb8c9d/auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q"
Jul 22 03:01:32.552125 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:01:32.552040    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 03:01:48.744717 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:01:48.744676    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 03:02:07.439753 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:02:07.439701    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 03:02:26.148278 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:02:26.148245    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 03:02:38.008099 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:02:38.008065    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:02:38.110311 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:02:38.110278    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:02:45.047936 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:02:45.047902    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 03:03:00.140195 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:00.140154    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 03:03:11.555426 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:11.555348    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-995f8f/auth-test-model-995f8f-kserve-bc6d9d84c-28k5n"]
Jul 22 03:03:11.578954 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:11.578922    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:03:12.567196 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:12.567159    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-bb8c9d/auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q"]
Jul 22 03:03:12.585572 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:12.585546    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:03:16.297157 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:16.297126    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 03:03:24.453574 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:24.453541    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-995f8f/sub-test-model-995f8f-kserve-7754f69997-sbtmz"]
Jul 22 03:03:24.456992 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:24.456975    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-995f8f/sub-test-model-995f8f-kserve-7754f69997-sbtmz"
Jul 22 03:03:24.459789 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:24.459770    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-995f8f\"/\"sub-test-model-995f8f-kserve-self-signed-certs\""
Jul 22 03:03:24.467369 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:24.467343    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-995f8f/sub-test-model-995f8f-kserve-7754f69997-sbtmz"]
Jul 22 03:03:24.535996 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:24.535967    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:03:24.539162 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:24.539141    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43404426-b4aa-4af7-9624-d7df9402df36-home\") pod \"sub-test-model-995f8f-kserve-7754f69997-sbtmz\" (UID: \"43404426-b4aa-4af7-9624-d7df9402df36\") " pod="ai-tenant-e2e-shared-a-995f8f/sub-test-model-995f8f-kserve-7754f69997-sbtmz"
Jul 22 03:03:24.539222 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:24.539191    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4w44h\" (UniqueName: \"kubernetes.io/projected/43404426-b4aa-4af7-9624-d7df9402df36-kube-api-access-4w44h\") pod \"sub-test-model-995f8f-kserve-7754f69997-sbtmz\" (UID: \"43404426-b4aa-4af7-9624-d7df9402df36\") " pod="ai-tenant-e2e-shared-a-995f8f/sub-test-model-995f8f-kserve-7754f69997-sbtmz"
Jul 22 03:03:24.539290 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:24.539277    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43404426-b4aa-4af7-9624-d7df9402df36-tmp-dir\") pod \"sub-test-model-995f8f-kserve-7754f69997-sbtmz\" (UID: \"43404426-b4aa-4af7-9624-d7df9402df36\") " pod="ai-tenant-e2e-shared-a-995f8f/sub-test-model-995f8f-kserve-7754f69997-sbtmz"
Jul 22 03:03:24.539326 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:24.539302    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43404426-b4aa-4af7-9624-d7df9402df36-dshm\") pod \"sub-test-model-995f8f-kserve-7754f69997-sbtmz\" (UID: \"43404426-b4aa-4af7-9624-d7df9402df36\") " pod="ai-tenant-e2e-shared-a-995f8f/sub-test-model-995f8f-kserve-7754f69997-sbtmz"
Jul 22 03:03:24.539326 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:24.539323    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43404426-b4aa-4af7-9624-d7df9402df36-tls-certs\") pod \"sub-test-model-995f8f-kserve-7754f69997-sbtmz\" (UID: \"43404426-b4aa-4af7-9624-d7df9402df36\") " pod="ai-tenant-e2e-shared-a-995f8f/sub-test-model-995f8f-kserve-7754f69997-sbtmz"
Jul 22 03:03:24.539435 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:24.539341    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43404426-b4aa-4af7-9624-d7df9402df36-model-cache\") pod \"sub-test-model-995f8f-kserve-7754f69997-sbtmz\" (UID: \"43404426-b4aa-4af7-9624-d7df9402df36\") " pod="ai-tenant-e2e-shared-a-995f8f/sub-test-model-995f8f-kserve-7754f69997-sbtmz"
Jul 22 03:03:24.640382 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:24.640343    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43404426-b4aa-4af7-9624-d7df9402df36-tmp-dir\") pod \"sub-test-model-995f8f-kserve-7754f69997-sbtmz\" (UID: \"43404426-b4aa-4af7-9624-d7df9402df36\") " pod="ai-tenant-e2e-shared-a-995f8f/sub-test-model-995f8f-kserve-7754f69997-sbtmz"
Jul 22 03:03:24.640382 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:24.640384    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43404426-b4aa-4af7-9624-d7df9402df36-dshm\") pod \"sub-test-model-995f8f-kserve-7754f69997-sbtmz\" (UID: \"43404426-b4aa-4af7-9624-d7df9402df36\") " pod="ai-tenant-e2e-shared-a-995f8f/sub-test-model-995f8f-kserve-7754f69997-sbtmz"
Jul 22 03:03:24.640610 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:24.640403    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43404426-b4aa-4af7-9624-d7df9402df36-tls-certs\") pod \"sub-test-model-995f8f-kserve-7754f69997-sbtmz\" (UID: \"43404426-b4aa-4af7-9624-d7df9402df36\") " pod="ai-tenant-e2e-shared-a-995f8f/sub-test-model-995f8f-kserve-7754f69997-sbtmz"
Jul 22 03:03:24.640610 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:24.640421    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43404426-b4aa-4af7-9624-d7df9402df36-model-cache\") pod \"sub-test-model-995f8f-kserve-7754f69997-sbtmz\" (UID: \"43404426-b4aa-4af7-9624-d7df9402df36\") " pod="ai-tenant-e2e-shared-a-995f8f/sub-test-model-995f8f-kserve-7754f69997-sbtmz"
Jul 22 03:03:24.640610 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:24.640473    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43404426-b4aa-4af7-9624-d7df9402df36-home\") pod \"sub-test-model-995f8f-kserve-7754f69997-sbtmz\" (UID: \"43404426-b4aa-4af7-9624-d7df9402df36\") " pod="ai-tenant-e2e-shared-a-995f8f/sub-test-model-995f8f-kserve-7754f69997-sbtmz"
Jul 22 03:03:24.640610 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:24.640513    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4w44h\" (UniqueName: \"kubernetes.io/projected/43404426-b4aa-4af7-9624-d7df9402df36-kube-api-access-4w44h\") pod \"sub-test-model-995f8f-kserve-7754f69997-sbtmz\" (UID: \"43404426-b4aa-4af7-9624-d7df9402df36\") " pod="ai-tenant-e2e-shared-a-995f8f/sub-test-model-995f8f-kserve-7754f69997-sbtmz"
Jul 22 03:03:24.640913 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:24.640887    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43404426-b4aa-4af7-9624-d7df9402df36-tmp-dir\") pod \"sub-test-model-995f8f-kserve-7754f69997-sbtmz\" (UID: \"43404426-b4aa-4af7-9624-d7df9402df36\") " pod="ai-tenant-e2e-shared-a-995f8f/sub-test-model-995f8f-kserve-7754f69997-sbtmz"
Jul 22 03:03:24.641040 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:24.640929    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43404426-b4aa-4af7-9624-d7df9402df36-model-cache\") pod \"sub-test-model-995f8f-kserve-7754f69997-sbtmz\" (UID: \"43404426-b4aa-4af7-9624-d7df9402df36\") " pod="ai-tenant-e2e-shared-a-995f8f/sub-test-model-995f8f-kserve-7754f69997-sbtmz"
Jul 22 03:03:24.641040 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:24.640965    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43404426-b4aa-4af7-9624-d7df9402df36-home\") pod \"sub-test-model-995f8f-kserve-7754f69997-sbtmz\" (UID: \"43404426-b4aa-4af7-9624-d7df9402df36\") " pod="ai-tenant-e2e-shared-a-995f8f/sub-test-model-995f8f-kserve-7754f69997-sbtmz"
Jul 22 03:03:24.642678 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:24.642656    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43404426-b4aa-4af7-9624-d7df9402df36-dshm\") pod \"sub-test-model-995f8f-kserve-7754f69997-sbtmz\" (UID: \"43404426-b4aa-4af7-9624-d7df9402df36\") " pod="ai-tenant-e2e-shared-a-995f8f/sub-test-model-995f8f-kserve-7754f69997-sbtmz"
Jul 22 03:03:24.642946 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:24.642931    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43404426-b4aa-4af7-9624-d7df9402df36-tls-certs\") pod \"sub-test-model-995f8f-kserve-7754f69997-sbtmz\" (UID: \"43404426-b4aa-4af7-9624-d7df9402df36\") " pod="ai-tenant-e2e-shared-a-995f8f/sub-test-model-995f8f-kserve-7754f69997-sbtmz"
Jul 22 03:03:24.649126 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:24.649102    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4w44h\" (UniqueName: \"kubernetes.io/projected/43404426-b4aa-4af7-9624-d7df9402df36-kube-api-access-4w44h\") pod \"sub-test-model-995f8f-kserve-7754f69997-sbtmz\" (UID: \"43404426-b4aa-4af7-9624-d7df9402df36\") " pod="ai-tenant-e2e-shared-a-995f8f/sub-test-model-995f8f-kserve-7754f69997-sbtmz"
Jul 22 03:03:24.766349 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:24.766311    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-995f8f/sub-test-model-995f8f-kserve-7754f69997-sbtmz"
Jul 22 03:03:24.902974 ip-10-0-138-202 kubenswrapper[2581]: W0722 03:03:24.902937    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod43404426_b4aa_4af7_9624_d7df9402df36.slice/crio-f83a7baaf38833a698302d8da64105f83c3f7ec565bbcea177713250d63d20cd WatchSource:0}: Error finding container f83a7baaf38833a698302d8da64105f83c3f7ec565bbcea177713250d63d20cd: Status 404 returned error can't find the container with id f83a7baaf38833a698302d8da64105f83c3f7ec565bbcea177713250d63d20cd
Jul 22 03:03:24.906248 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:24.906225    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-995f8f/sub-test-model-995f8f-kserve-7754f69997-sbtmz"]
Jul 22 03:03:25.721274 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:25.721232    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-995f8f/sub-test-model-995f8f-kserve-7754f69997-sbtmz" event={"ID":"43404426-b4aa-4af7-9624-d7df9402df36","Type":"ContainerStarted","Data":"87fb080b1fc06b3d378707acdf39ca50d2932683f07a006fa44a9eb6aa7f0db1"}
Jul 22 03:03:25.721660 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:25.721285    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-995f8f/sub-test-model-995f8f-kserve-7754f69997-sbtmz" event={"ID":"43404426-b4aa-4af7-9624-d7df9402df36","Type":"ContainerStarted","Data":"f83a7baaf38833a698302d8da64105f83c3f7ec565bbcea177713250d63d20cd"}
Jul 22 03:03:25.739662 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:25.739613    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-995f8f/sub-test-model-995f8f-kserve-7754f69997-sbtmz" podStartSLOduration=1.739597762 podStartE2EDuration="1.739597762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:03:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:03:25.738940704 +0000 UTC m=+2627.279734331" watchObservedRunningTime="2026-07-22 03:03:25.739597762 +0000 UTC m=+2627.280391388"
Jul 22 03:03:26.575805 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:26.575733    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-995f8f/auth-test-model-995f8f-kserve-bc6d9d84c-28k5n" podUID="c540d47f-4cb7-443b-84d9-efcf7f731b35" containerName="main" containerID="cri-o://6bec6e20c6b1386790c1963feb6fd700a81f8c98aa98f19df1d9723d921dd2c1" gracePeriod=45
Jul 22 03:03:26.728529 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:26.728499    2581 generic.go:356] "Generic (PLEG): container finished" podID="c540d47f-4cb7-443b-84d9-efcf7f731b35" containerID="6bec6e20c6b1386790c1963feb6fd700a81f8c98aa98f19df1d9723d921dd2c1" exitCode=0
Jul 22 03:03:26.728930 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:26.728574    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-995f8f/auth-test-model-995f8f-kserve-bc6d9d84c-28k5n" event={"ID":"c540d47f-4cb7-443b-84d9-efcf7f731b35","Type":"ContainerDied","Data":"6bec6e20c6b1386790c1963feb6fd700a81f8c98aa98f19df1d9723d921dd2c1"}
Jul 22 03:03:26.839682 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:26.839616    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-995f8f/auth-test-model-995f8f-kserve-bc6d9d84c-28k5n"
Jul 22 03:03:26.964850 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:26.964819    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c540d47f-4cb7-443b-84d9-efcf7f731b35-model-cache\") pod \"c540d47f-4cb7-443b-84d9-efcf7f731b35\" (UID: \"c540d47f-4cb7-443b-84d9-efcf7f731b35\") "
Jul 22 03:03:26.965152 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:26.964871    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c540d47f-4cb7-443b-84d9-efcf7f731b35-tls-certs\") pod \"c540d47f-4cb7-443b-84d9-efcf7f731b35\" (UID: \"c540d47f-4cb7-443b-84d9-efcf7f731b35\") "
Jul 22 03:03:26.965152 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:26.964899    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c540d47f-4cb7-443b-84d9-efcf7f731b35-tmp-dir\") pod \"c540d47f-4cb7-443b-84d9-efcf7f731b35\" (UID: \"c540d47f-4cb7-443b-84d9-efcf7f731b35\") "
Jul 22 03:03:26.965152 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:26.964924    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ptcdz\" (UniqueName: \"kubernetes.io/projected/c540d47f-4cb7-443b-84d9-efcf7f731b35-kube-api-access-ptcdz\") pod \"c540d47f-4cb7-443b-84d9-efcf7f731b35\" (UID: \"c540d47f-4cb7-443b-84d9-efcf7f731b35\") "
Jul 22 03:03:26.965152 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:26.964997    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c540d47f-4cb7-443b-84d9-efcf7f731b35-dshm\") pod \"c540d47f-4cb7-443b-84d9-efcf7f731b35\" (UID: \"c540d47f-4cb7-443b-84d9-efcf7f731b35\") "
Jul 22 03:03:26.965152 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:26.965053    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c540d47f-4cb7-443b-84d9-efcf7f731b35-home\") pod \"c540d47f-4cb7-443b-84d9-efcf7f731b35\" (UID: \"c540d47f-4cb7-443b-84d9-efcf7f731b35\") "
Jul 22 03:03:26.965442 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:26.965144    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c540d47f-4cb7-443b-84d9-efcf7f731b35-model-cache" (OuterVolumeSpecName: "model-cache") pod "c540d47f-4cb7-443b-84d9-efcf7f731b35" (UID: "c540d47f-4cb7-443b-84d9-efcf7f731b35"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 03:03:26.965442 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:26.965200    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c540d47f-4cb7-443b-84d9-efcf7f731b35-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c540d47f-4cb7-443b-84d9-efcf7f731b35" (UID: "c540d47f-4cb7-443b-84d9-efcf7f731b35"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 03:03:26.965442 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:26.965360    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c540d47f-4cb7-443b-84d9-efcf7f731b35-home" (OuterVolumeSpecName: "home") pod "c540d47f-4cb7-443b-84d9-efcf7f731b35" (UID: "c540d47f-4cb7-443b-84d9-efcf7f731b35"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 03:03:26.965442 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:26.965422    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c540d47f-4cb7-443b-84d9-efcf7f731b35-home\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:03:26.965442 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:26.965432    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c540d47f-4cb7-443b-84d9-efcf7f731b35-model-cache\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:03:26.965442 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:26.965445    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c540d47f-4cb7-443b-84d9-efcf7f731b35-tmp-dir\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:03:26.967183 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:26.967159    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c540d47f-4cb7-443b-84d9-efcf7f731b35-dshm" (OuterVolumeSpecName: "dshm") pod "c540d47f-4cb7-443b-84d9-efcf7f731b35" (UID: "c540d47f-4cb7-443b-84d9-efcf7f731b35"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 03:03:26.967294 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:26.967211    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c540d47f-4cb7-443b-84d9-efcf7f731b35-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c540d47f-4cb7-443b-84d9-efcf7f731b35" (UID: "c540d47f-4cb7-443b-84d9-efcf7f731b35"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 03:03:26.967294 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:26.967276    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c540d47f-4cb7-443b-84d9-efcf7f731b35-kube-api-access-ptcdz" (OuterVolumeSpecName: "kube-api-access-ptcdz") pod "c540d47f-4cb7-443b-84d9-efcf7f731b35" (UID: "c540d47f-4cb7-443b-84d9-efcf7f731b35"). InnerVolumeSpecName "kube-api-access-ptcdz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 03:03:27.066672 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:27.066632    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c540d47f-4cb7-443b-84d9-efcf7f731b35-tls-certs\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:03:27.066672 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:27.066664    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptcdz\" (UniqueName: \"kubernetes.io/projected/c540d47f-4cb7-443b-84d9-efcf7f731b35-kube-api-access-ptcdz\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:03:27.066672 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:27.066674    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c540d47f-4cb7-443b-84d9-efcf7f731b35-dshm\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:03:27.588749 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:27.588687    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-bb8c9d/auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q" podUID="f6b13fd7-13c4-497c-a1da-21f576f4d6de" containerName="main" containerID="cri-o://c41e2ab03d0985cacc33b7e82f404ebe0a8d003114c3ae1c3f97b723b89d6395" gracePeriod=45
Jul 22 03:03:27.736585 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:27.736543    2581 generic.go:356] "Generic (PLEG): container finished" podID="f6b13fd7-13c4-497c-a1da-21f576f4d6de" containerID="c41e2ab03d0985cacc33b7e82f404ebe0a8d003114c3ae1c3f97b723b89d6395" exitCode=0
Jul 22 03:03:27.737083 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:27.736617    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bb8c9d/auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q" event={"ID":"f6b13fd7-13c4-497c-a1da-21f576f4d6de","Type":"ContainerDied","Data":"c41e2ab03d0985cacc33b7e82f404ebe0a8d003114c3ae1c3f97b723b89d6395"}
Jul 22 03:03:27.738473 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:27.738449    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-995f8f/auth-test-model-995f8f-kserve-bc6d9d84c-28k5n" event={"ID":"c540d47f-4cb7-443b-84d9-efcf7f731b35","Type":"ContainerDied","Data":"f1fa89f2bf3942bb1442c7ca80343309e35b443502a5c27d6aab02a3c8c898b9"}
Jul 22 03:03:27.738596 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:27.738497    2581 scope.go:117] "RemoveContainer" containerID="6bec6e20c6b1386790c1963feb6fd700a81f8c98aa98f19df1d9723d921dd2c1"
Jul 22 03:03:27.738596 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:27.738454    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-995f8f/auth-test-model-995f8f-kserve-bc6d9d84c-28k5n"
Jul 22 03:03:27.760992 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:27.760964    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-995f8f/auth-test-model-995f8f-kserve-bc6d9d84c-28k5n"]
Jul 22 03:03:27.766426 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:27.766400    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-995f8f/auth-test-model-995f8f-kserve-bc6d9d84c-28k5n"]
Jul 22 03:03:27.847579 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:27.847519    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bb8c9d/auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q"
Jul 22 03:03:27.976203 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:27.976150    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6b13fd7-13c4-497c-a1da-21f576f4d6de-home\") pod \"f6b13fd7-13c4-497c-a1da-21f576f4d6de\" (UID: \"f6b13fd7-13c4-497c-a1da-21f576f4d6de\") "
Jul 22 03:03:27.976378 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:27.976219    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wc977\" (UniqueName: \"kubernetes.io/projected/f6b13fd7-13c4-497c-a1da-21f576f4d6de-kube-api-access-wc977\") pod \"f6b13fd7-13c4-497c-a1da-21f576f4d6de\" (UID: \"f6b13fd7-13c4-497c-a1da-21f576f4d6de\") "
Jul 22 03:03:27.976378 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:27.976290    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6b13fd7-13c4-497c-a1da-21f576f4d6de-tls-certs\") pod \"f6b13fd7-13c4-497c-a1da-21f576f4d6de\" (UID: \"f6b13fd7-13c4-497c-a1da-21f576f4d6de\") "
Jul 22 03:03:27.976378 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:27.976329    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6b13fd7-13c4-497c-a1da-21f576f4d6de-tmp-dir\") pod \"f6b13fd7-13c4-497c-a1da-21f576f4d6de\" (UID: \"f6b13fd7-13c4-497c-a1da-21f576f4d6de\") "
Jul 22 03:03:27.976378 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:27.976366    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6b13fd7-13c4-497c-a1da-21f576f4d6de-model-cache\") pod \"f6b13fd7-13c4-497c-a1da-21f576f4d6de\" (UID: \"f6b13fd7-13c4-497c-a1da-21f576f4d6de\") "
Jul 22 03:03:27.976543 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:27.976444    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6b13fd7-13c4-497c-a1da-21f576f4d6de-dshm\") pod \"f6b13fd7-13c4-497c-a1da-21f576f4d6de\" (UID: \"f6b13fd7-13c4-497c-a1da-21f576f4d6de\") "
Jul 22 03:03:27.976543 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:27.976469    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6b13fd7-13c4-497c-a1da-21f576f4d6de-home" (OuterVolumeSpecName: "home") pod "f6b13fd7-13c4-497c-a1da-21f576f4d6de" (UID: "f6b13fd7-13c4-497c-a1da-21f576f4d6de"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 03:03:27.976649 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:27.976579    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6b13fd7-13c4-497c-a1da-21f576f4d6de-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f6b13fd7-13c4-497c-a1da-21f576f4d6de" (UID: "f6b13fd7-13c4-497c-a1da-21f576f4d6de"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 03:03:27.976707 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:27.976647    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6b13fd7-13c4-497c-a1da-21f576f4d6de-model-cache" (OuterVolumeSpecName: "model-cache") pod "f6b13fd7-13c4-497c-a1da-21f576f4d6de" (UID: "f6b13fd7-13c4-497c-a1da-21f576f4d6de"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 03:03:27.976825 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:27.976802    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6b13fd7-13c4-497c-a1da-21f576f4d6de-tmp-dir\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:03:27.976825 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:27.976824    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6b13fd7-13c4-497c-a1da-21f576f4d6de-model-cache\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:03:27.976976 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:27.976835    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6b13fd7-13c4-497c-a1da-21f576f4d6de-home\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:03:27.978426 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:27.978397    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6b13fd7-13c4-497c-a1da-21f576f4d6de-dshm" (OuterVolumeSpecName: "dshm") pod "f6b13fd7-13c4-497c-a1da-21f576f4d6de" (UID: "f6b13fd7-13c4-497c-a1da-21f576f4d6de"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 03:03:27.978426 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:27.978420    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6b13fd7-13c4-497c-a1da-21f576f4d6de-kube-api-access-wc977" (OuterVolumeSpecName: "kube-api-access-wc977") pod "f6b13fd7-13c4-497c-a1da-21f576f4d6de" (UID: "f6b13fd7-13c4-497c-a1da-21f576f4d6de"). InnerVolumeSpecName "kube-api-access-wc977". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 03:03:27.978547 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:27.978491    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6b13fd7-13c4-497c-a1da-21f576f4d6de-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f6b13fd7-13c4-497c-a1da-21f576f4d6de" (UID: "f6b13fd7-13c4-497c-a1da-21f576f4d6de"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 03:03:28.077653 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:28.077610    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wc977\" (UniqueName: \"kubernetes.io/projected/f6b13fd7-13c4-497c-a1da-21f576f4d6de-kube-api-access-wc977\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:03:28.077653 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:28.077643    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6b13fd7-13c4-497c-a1da-21f576f4d6de-tls-certs\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:03:28.077653 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:28.077655    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6b13fd7-13c4-497c-a1da-21f576f4d6de-dshm\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:03:28.745782 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:28.745730    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bb8c9d/auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q" event={"ID":"f6b13fd7-13c4-497c-a1da-21f576f4d6de","Type":"ContainerDied","Data":"ac77ba4bcb903fab6abfb795254634f5147fbaeedb42b6500900b1b63b0552cb"}
Jul 22 03:03:28.746273 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:28.745788    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bb8c9d/auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q"
Jul 22 03:03:28.746273 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:28.745799    2581 scope.go:117] "RemoveContainer" containerID="c41e2ab03d0985cacc33b7e82f404ebe0a8d003114c3ae1c3f97b723b89d6395"
Jul 22 03:03:28.767851 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:28.767826    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-bb8c9d/auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q"]
Jul 22 03:03:28.771800 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:28.771732    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-bb8c9d/auth-test-model-bb8c9d-kserve-97549c6c9-f7j2q"]
Jul 22 03:03:28.903926 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:28.903894    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c540d47f-4cb7-443b-84d9-efcf7f731b35" path="/var/lib/kubelet/pods/c540d47f-4cb7-443b-84d9-efcf7f731b35/volumes"
Jul 22 03:03:28.904310 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:28.904296    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6b13fd7-13c4-497c-a1da-21f576f4d6de" path="/var/lib/kubelet/pods/f6b13fd7-13c4-497c-a1da-21f576f4d6de/volumes"
Jul 22 03:03:29.245093 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:29.245064    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 03:03:34.767090 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:34.767052    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-995f8f/sub-test-model-995f8f-kserve-7754f69997-sbtmz"
Jul 22 03:03:34.767581 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:34.767272    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-995f8f/sub-test-model-995f8f-kserve-7754f69997-sbtmz"
Jul 22 03:03:34.779822 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:34.779793    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-995f8f/sub-test-model-995f8f-kserve-7754f69997-sbtmz"
Jul 22 03:03:35.792690 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:35.792659    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-995f8f/sub-test-model-995f8f-kserve-7754f69997-sbtmz"
Jul 22 03:03:48.945958 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:48.945922    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-bb8c9d/sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g"]
Jul 22 03:03:48.946356 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:48.946336    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c540d47f-4cb7-443b-84d9-efcf7f731b35" containerName="main"
Jul 22 03:03:48.946404 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:48.946360    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c540d47f-4cb7-443b-84d9-efcf7f731b35" containerName="main"
Jul 22 03:03:48.946404 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:48.946390    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6b13fd7-13c4-497c-a1da-21f576f4d6de" containerName="main"
Jul 22 03:03:48.946404 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:48.946399    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6b13fd7-13c4-497c-a1da-21f576f4d6de" containerName="main"
Jul 22 03:03:48.946513 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:48.946502    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6b13fd7-13c4-497c-a1da-21f576f4d6de" containerName="main"
Jul 22 03:03:48.946548 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:48.946517    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c540d47f-4cb7-443b-84d9-efcf7f731b35" containerName="main"
Jul 22 03:03:48.949623 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:48.949607    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bb8c9d/sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g"
Jul 22 03:03:48.953695 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:48.953661    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-bb8c9d\"/\"kube-root-ca.crt\""
Jul 22 03:03:48.953848 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:48.953699    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-bb8c9d\"/\"sub-test-model-bb8c9d-kserve-self-signed-certs\""
Jul 22 03:03:48.953848 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:48.953722    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-bb8c9d\"/\"default-dockercfg-mbfrf\""
Jul 22 03:03:48.953848 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:48.953663    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-bb8c9d\"/\"openshift-service-ca.crt\""
Jul 22 03:03:48.959352 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:48.959329    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-bb8c9d/sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g"]
Jul 22 03:03:48.984038 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:48.984007    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:03:49.064301 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:49.064265    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgx6w\" (UniqueName: \"kubernetes.io/projected/9b51b586-d987-438b-ac8b-9a8a2bd3aee5-kube-api-access-pgx6w\") pod \"sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g\" (UID: \"9b51b586-d987-438b-ac8b-9a8a2bd3aee5\") " pod="ai-tenant-e2e-shared-b-bb8c9d/sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g"
Jul 22 03:03:49.064509 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:49.064307    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b51b586-d987-438b-ac8b-9a8a2bd3aee5-tls-certs\") pod \"sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g\" (UID: \"9b51b586-d987-438b-ac8b-9a8a2bd3aee5\") " pod="ai-tenant-e2e-shared-b-bb8c9d/sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g"
Jul 22 03:03:49.064509 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:49.064329    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b51b586-d987-438b-ac8b-9a8a2bd3aee5-model-cache\") pod \"sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g\" (UID: \"9b51b586-d987-438b-ac8b-9a8a2bd3aee5\") " pod="ai-tenant-e2e-shared-b-bb8c9d/sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g"
Jul 22 03:03:49.064509 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:49.064440    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b51b586-d987-438b-ac8b-9a8a2bd3aee5-tmp-dir\") pod \"sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g\" (UID: \"9b51b586-d987-438b-ac8b-9a8a2bd3aee5\") " pod="ai-tenant-e2e-shared-b-bb8c9d/sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g"
Jul 22 03:03:49.064509 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:49.064479    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b51b586-d987-438b-ac8b-9a8a2bd3aee5-dshm\") pod \"sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g\" (UID: \"9b51b586-d987-438b-ac8b-9a8a2bd3aee5\") " pod="ai-tenant-e2e-shared-b-bb8c9d/sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g"
Jul 22 03:03:49.064686 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:49.064568    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b51b586-d987-438b-ac8b-9a8a2bd3aee5-home\") pod \"sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g\" (UID: \"9b51b586-d987-438b-ac8b-9a8a2bd3aee5\") " pod="ai-tenant-e2e-shared-b-bb8c9d/sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g"
Jul 22 03:03:49.165697 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:49.165656    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgx6w\" (UniqueName: \"kubernetes.io/projected/9b51b586-d987-438b-ac8b-9a8a2bd3aee5-kube-api-access-pgx6w\") pod \"sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g\" (UID: \"9b51b586-d987-438b-ac8b-9a8a2bd3aee5\") " pod="ai-tenant-e2e-shared-b-bb8c9d/sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g"
Jul 22 03:03:49.165697 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:49.165702    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b51b586-d987-438b-ac8b-9a8a2bd3aee5-tls-certs\") pod \"sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g\" (UID: \"9b51b586-d987-438b-ac8b-9a8a2bd3aee5\") " pod="ai-tenant-e2e-shared-b-bb8c9d/sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g"
Jul 22 03:03:49.165993 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:49.165722    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b51b586-d987-438b-ac8b-9a8a2bd3aee5-model-cache\") pod \"sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g\" (UID: \"9b51b586-d987-438b-ac8b-9a8a2bd3aee5\") " pod="ai-tenant-e2e-shared-b-bb8c9d/sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g"
Jul 22 03:03:49.165993 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:49.165779    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b51b586-d987-438b-ac8b-9a8a2bd3aee5-tmp-dir\") pod \"sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g\" (UID: \"9b51b586-d987-438b-ac8b-9a8a2bd3aee5\") " pod="ai-tenant-e2e-shared-b-bb8c9d/sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g"
Jul 22 03:03:49.165993 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:49.165797    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b51b586-d987-438b-ac8b-9a8a2bd3aee5-dshm\") pod \"sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g\" (UID: \"9b51b586-d987-438b-ac8b-9a8a2bd3aee5\") " pod="ai-tenant-e2e-shared-b-bb8c9d/sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g"
Jul 22 03:03:49.165993 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:49.165833    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b51b586-d987-438b-ac8b-9a8a2bd3aee5-home\") pod \"sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g\" (UID: \"9b51b586-d987-438b-ac8b-9a8a2bd3aee5\") " pod="ai-tenant-e2e-shared-b-bb8c9d/sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g"
Jul 22 03:03:49.166222 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:49.166198    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b51b586-d987-438b-ac8b-9a8a2bd3aee5-model-cache\") pod \"sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g\" (UID: \"9b51b586-d987-438b-ac8b-9a8a2bd3aee5\") " pod="ai-tenant-e2e-shared-b-bb8c9d/sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g"
Jul 22 03:03:49.166287 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:49.166241    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b51b586-d987-438b-ac8b-9a8a2bd3aee5-home\") pod \"sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g\" (UID: \"9b51b586-d987-438b-ac8b-9a8a2bd3aee5\") " pod="ai-tenant-e2e-shared-b-bb8c9d/sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g"
Jul 22 03:03:49.166328 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:49.166310    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b51b586-d987-438b-ac8b-9a8a2bd3aee5-tmp-dir\") pod \"sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g\" (UID: \"9b51b586-d987-438b-ac8b-9a8a2bd3aee5\") " pod="ai-tenant-e2e-shared-b-bb8c9d/sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g"
Jul 22 03:03:49.168237 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:49.168209    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b51b586-d987-438b-ac8b-9a8a2bd3aee5-dshm\") pod \"sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g\" (UID: \"9b51b586-d987-438b-ac8b-9a8a2bd3aee5\") " pod="ai-tenant-e2e-shared-b-bb8c9d/sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g"
Jul 22 03:03:49.168437 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:49.168418    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b51b586-d987-438b-ac8b-9a8a2bd3aee5-tls-certs\") pod \"sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g\" (UID: \"9b51b586-d987-438b-ac8b-9a8a2bd3aee5\") " pod="ai-tenant-e2e-shared-b-bb8c9d/sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g"
Jul 22 03:03:49.174351 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:49.174331    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgx6w\" (UniqueName: \"kubernetes.io/projected/9b51b586-d987-438b-ac8b-9a8a2bd3aee5-kube-api-access-pgx6w\") pod \"sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g\" (UID: \"9b51b586-d987-438b-ac8b-9a8a2bd3aee5\") " pod="ai-tenant-e2e-shared-b-bb8c9d/sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g"
Jul 22 03:03:49.259901 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:49.259853    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bb8c9d/sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g"
Jul 22 03:03:49.397365 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:49.397267    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-bb8c9d/sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g"]
Jul 22 03:03:49.399634 ip-10-0-138-202 kubenswrapper[2581]: W0722 03:03:49.399606    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9b51b586_d987_438b_ac8b_9a8a2bd3aee5.slice/crio-60365cb233d2c0115c3446bb25f0924df1d75934346e2bb04af14d49cef7b5b3 WatchSource:0}: Error finding container 60365cb233d2c0115c3446bb25f0924df1d75934346e2bb04af14d49cef7b5b3: Status 404 returned error can't find the container with id 60365cb233d2c0115c3446bb25f0924df1d75934346e2bb04af14d49cef7b5b3
Jul 22 03:03:49.852133 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:49.852089    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bb8c9d/sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g" event={"ID":"9b51b586-d987-438b-ac8b-9a8a2bd3aee5","Type":"ContainerStarted","Data":"1ed0fb555505254c7a4f07c5941d21cdaf86aef6c38413f0854a8815f7bd12ff"}
Jul 22 03:03:49.852133 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:49.852142    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bb8c9d/sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g" event={"ID":"9b51b586-d987-438b-ac8b-9a8a2bd3aee5","Type":"ContainerStarted","Data":"60365cb233d2c0115c3446bb25f0924df1d75934346e2bb04af14d49cef7b5b3"}
Jul 22 03:03:49.870674 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:49.870628    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-bb8c9d/sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g" podStartSLOduration=1.870613673 podStartE2EDuration="1.870613673s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:03:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:03:49.868578488 +0000 UTC m=+2651.409372116" watchObservedRunningTime="2026-07-22 03:03:49.870613673 +0000 UTC m=+2651.411407333"
Jul 22 03:03:52.040186 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:52.040147    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 03:03:59.260317 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:59.260278    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-bb8c9d/sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g"
Jul 22 03:03:59.260952 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:59.260335    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-bb8c9d/sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g"
Jul 22 03:03:59.272604 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:59.272583    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-bb8c9d/sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g"
Jul 22 03:03:59.911520 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:03:59.911494    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-bb8c9d/sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g"
Jul 22 03:04:23.058367 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:04:23.058331    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 03:04:40.844012 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:04:40.843976    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 03:04:47.537045 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:04:47.537006    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-995f8f/sub-test-model-995f8f-kserve-7754f69997-sbtmz"]
Jul 22 03:04:47.563266 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:04:47.563222    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:04:48.354786 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:04:48.354745    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-bb8c9d/sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g"]
Jul 22 03:04:48.369523 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:04:48.369479    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:04:57.455171 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:04:57.455134    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-995f8f/rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc"]
Jul 22 03:04:57.458669 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:04:57.458653    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-995f8f/rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc"
Jul 22 03:04:57.461831 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:04:57.461807    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-995f8f\"/\"rate-test-model-995f8f-kserve-self-signed-certs\""
Jul 22 03:04:57.471359 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:04:57.471337    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-995f8f/rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc"]
Jul 22 03:04:57.483638 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:04:57.483617    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:04:57.594783 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:04:57.594751    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3378a89-1677-437b-98d1-35296313b222-dshm\") pod \"rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc\" (UID: \"d3378a89-1677-437b-98d1-35296313b222\") " pod="ai-tenant-e2e-shared-a-995f8f/rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc"
Jul 22 03:04:57.594978 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:04:57.594803    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3378a89-1677-437b-98d1-35296313b222-tls-certs\") pod \"rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc\" (UID: \"d3378a89-1677-437b-98d1-35296313b222\") " pod="ai-tenant-e2e-shared-a-995f8f/rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc"
Jul 22 03:04:57.594978 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:04:57.594889    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qptcf\" (UniqueName: \"kubernetes.io/projected/d3378a89-1677-437b-98d1-35296313b222-kube-api-access-qptcf\") pod \"rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc\" (UID: \"d3378a89-1677-437b-98d1-35296313b222\") " pod="ai-tenant-e2e-shared-a-995f8f/rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc"
Jul 22 03:04:57.594978 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:04:57.594908    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3378a89-1677-437b-98d1-35296313b222-model-cache\") pod \"rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc\" (UID: \"d3378a89-1677-437b-98d1-35296313b222\") " pod="ai-tenant-e2e-shared-a-995f8f/rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc"
Jul 22 03:04:57.594978 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:04:57.594924    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3378a89-1677-437b-98d1-35296313b222-tmp-dir\") pod \"rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc\" (UID: \"d3378a89-1677-437b-98d1-35296313b222\") " pod="ai-tenant-e2e-shared-a-995f8f/rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc"
Jul 22 03:04:57.595138 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:04:57.594973    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3378a89-1677-437b-98d1-35296313b222-home\") pod \"rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc\" (UID: \"d3378a89-1677-437b-98d1-35296313b222\") " pod="ai-tenant-e2e-shared-a-995f8f/rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc"
Jul 22 03:04:57.695516 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:04:57.695478    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3378a89-1677-437b-98d1-35296313b222-dshm\") pod \"rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc\" (UID: \"d3378a89-1677-437b-98d1-35296313b222\") " pod="ai-tenant-e2e-shared-a-995f8f/rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc"
Jul 22 03:04:57.695687 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:04:57.695533    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3378a89-1677-437b-98d1-35296313b222-tls-certs\") pod \"rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc\" (UID: \"d3378a89-1677-437b-98d1-35296313b222\") " pod="ai-tenant-e2e-shared-a-995f8f/rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc"
Jul 22 03:04:57.695687 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:04:57.695604    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qptcf\" (UniqueName: \"kubernetes.io/projected/d3378a89-1677-437b-98d1-35296313b222-kube-api-access-qptcf\") pod \"rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc\" (UID: \"d3378a89-1677-437b-98d1-35296313b222\") " pod="ai-tenant-e2e-shared-a-995f8f/rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc"
Jul 22 03:04:57.695687 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:04:57.695630    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3378a89-1677-437b-98d1-35296313b222-model-cache\") pod \"rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc\" (UID: \"d3378a89-1677-437b-98d1-35296313b222\") " pod="ai-tenant-e2e-shared-a-995f8f/rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc"
Jul 22 03:04:57.695687 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:04:57.695657    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3378a89-1677-437b-98d1-35296313b222-tmp-dir\") pod \"rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc\" (UID: \"d3378a89-1677-437b-98d1-35296313b222\") " pod="ai-tenant-e2e-shared-a-995f8f/rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc"
Jul 22 03:04:57.695964 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:04:57.695791    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3378a89-1677-437b-98d1-35296313b222-home\") pod \"rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc\" (UID: \"d3378a89-1677-437b-98d1-35296313b222\") " pod="ai-tenant-e2e-shared-a-995f8f/rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc"
Jul 22 03:04:57.696055 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:04:57.696031    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3378a89-1677-437b-98d1-35296313b222-model-cache\") pod \"rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc\" (UID: \"d3378a89-1677-437b-98d1-35296313b222\") " pod="ai-tenant-e2e-shared-a-995f8f/rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc"
Jul 22 03:04:57.696116 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:04:57.696058    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3378a89-1677-437b-98d1-35296313b222-tmp-dir\") pod \"rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc\" (UID: \"d3378a89-1677-437b-98d1-35296313b222\") " pod="ai-tenant-e2e-shared-a-995f8f/rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc"
Jul 22 03:04:57.696156 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:04:57.696142    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3378a89-1677-437b-98d1-35296313b222-home\") pod \"rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc\" (UID: \"d3378a89-1677-437b-98d1-35296313b222\") " pod="ai-tenant-e2e-shared-a-995f8f/rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc"
Jul 22 03:04:57.697838 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:04:57.697819    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3378a89-1677-437b-98d1-35296313b222-dshm\") pod \"rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc\" (UID: \"d3378a89-1677-437b-98d1-35296313b222\") " pod="ai-tenant-e2e-shared-a-995f8f/rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc"
Jul 22 03:04:57.698085 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:04:57.698067    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3378a89-1677-437b-98d1-35296313b222-tls-certs\") pod \"rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc\" (UID: \"d3378a89-1677-437b-98d1-35296313b222\") " pod="ai-tenant-e2e-shared-a-995f8f/rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc"
Jul 22 03:04:57.706546 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:04:57.706492    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qptcf\" (UniqueName: \"kubernetes.io/projected/d3378a89-1677-437b-98d1-35296313b222-kube-api-access-qptcf\") pod \"rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc\" (UID: \"d3378a89-1677-437b-98d1-35296313b222\") " pod="ai-tenant-e2e-shared-a-995f8f/rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc"
Jul 22 03:04:57.767854 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:04:57.767818    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-995f8f/rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc"
Jul 22 03:04:57.909831 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:04:57.909804    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-995f8f/rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc"]
Jul 22 03:04:57.911449 ip-10-0-138-202 kubenswrapper[2581]: W0722 03:04:57.911420    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd3378a89_1677_437b_98d1_35296313b222.slice/crio-854ce614f36a527af70cf1f4cdc51363e0f0c67c3808052e045023dd0cebb2a1 WatchSource:0}: Error finding container 854ce614f36a527af70cf1f4cdc51363e0f0c67c3808052e045023dd0cebb2a1: Status 404 returned error can't find the container with id 854ce614f36a527af70cf1f4cdc51363e0f0c67c3808052e045023dd0cebb2a1
Jul 22 03:04:58.193914 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:04:58.193874    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-995f8f/rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc" event={"ID":"d3378a89-1677-437b-98d1-35296313b222","Type":"ContainerStarted","Data":"d94d43e61d4a8ffc8c59571f1d27f2ea6c02b651077492744340f6a34d8603e6"}
Jul 22 03:04:58.194084 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:04:58.193923    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-995f8f/rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc" event={"ID":"d3378a89-1677-437b-98d1-35296313b222","Type":"ContainerStarted","Data":"854ce614f36a527af70cf1f4cdc51363e0f0c67c3808052e045023dd0cebb2a1"}
Jul 22 03:04:58.218067 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:04:58.217963    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-995f8f/rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc" podStartSLOduration=1.217948143 podStartE2EDuration="1.217948143s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:04:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:04:58.215575252 +0000 UTC m=+2719.756368878" watchObservedRunningTime="2026-07-22 03:04:58.217948143 +0000 UTC m=+2719.758741770"
Jul 22 03:04:59.835197 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:04:59.835163    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 03:05:02.556932 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:02.556893    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-995f8f/sub-test-model-995f8f-kserve-7754f69997-sbtmz" podUID="43404426-b4aa-4af7-9624-d7df9402df36" containerName="main" containerID="cri-o://87fb080b1fc06b3d378707acdf39ca50d2932683f07a006fa44a9eb6aa7f0db1" gracePeriod=45
Jul 22 03:05:02.812377 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:02.812310    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-995f8f/sub-test-model-995f8f-kserve-7754f69997-sbtmz"
Jul 22 03:05:02.946281 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:02.946253    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43404426-b4aa-4af7-9624-d7df9402df36-dshm\") pod \"43404426-b4aa-4af7-9624-d7df9402df36\" (UID: \"43404426-b4aa-4af7-9624-d7df9402df36\") "
Jul 22 03:05:02.946456 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:02.946310    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43404426-b4aa-4af7-9624-d7df9402df36-tls-certs\") pod \"43404426-b4aa-4af7-9624-d7df9402df36\" (UID: \"43404426-b4aa-4af7-9624-d7df9402df36\") "
Jul 22 03:05:02.946456 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:02.946331    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43404426-b4aa-4af7-9624-d7df9402df36-tmp-dir\") pod \"43404426-b4aa-4af7-9624-d7df9402df36\" (UID: \"43404426-b4aa-4af7-9624-d7df9402df36\") "
Jul 22 03:05:02.946456 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:02.946360    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4w44h\" (UniqueName: \"kubernetes.io/projected/43404426-b4aa-4af7-9624-d7df9402df36-kube-api-access-4w44h\") pod \"43404426-b4aa-4af7-9624-d7df9402df36\" (UID: \"43404426-b4aa-4af7-9624-d7df9402df36\") "
Jul 22 03:05:02.946456 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:02.946397    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43404426-b4aa-4af7-9624-d7df9402df36-home\") pod \"43404426-b4aa-4af7-9624-d7df9402df36\" (UID: \"43404426-b4aa-4af7-9624-d7df9402df36\") "
Jul 22 03:05:02.946456 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:02.946445    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43404426-b4aa-4af7-9624-d7df9402df36-model-cache\") pod \"43404426-b4aa-4af7-9624-d7df9402df36\" (UID: \"43404426-b4aa-4af7-9624-d7df9402df36\") "
Jul 22 03:05:02.946729 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:02.946680    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43404426-b4aa-4af7-9624-d7df9402df36-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "43404426-b4aa-4af7-9624-d7df9402df36" (UID: "43404426-b4aa-4af7-9624-d7df9402df36"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 03:05:02.946729 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:02.946671    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43404426-b4aa-4af7-9624-d7df9402df36-home" (OuterVolumeSpecName: "home") pod "43404426-b4aa-4af7-9624-d7df9402df36" (UID: "43404426-b4aa-4af7-9624-d7df9402df36"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 03:05:02.946851 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:02.946763    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43404426-b4aa-4af7-9624-d7df9402df36-model-cache" (OuterVolumeSpecName: "model-cache") pod "43404426-b4aa-4af7-9624-d7df9402df36" (UID: "43404426-b4aa-4af7-9624-d7df9402df36"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 03:05:02.948512 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:02.948486    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43404426-b4aa-4af7-9624-d7df9402df36-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "43404426-b4aa-4af7-9624-d7df9402df36" (UID: "43404426-b4aa-4af7-9624-d7df9402df36"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 03:05:02.948622 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:02.948533    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43404426-b4aa-4af7-9624-d7df9402df36-dshm" (OuterVolumeSpecName: "dshm") pod "43404426-b4aa-4af7-9624-d7df9402df36" (UID: "43404426-b4aa-4af7-9624-d7df9402df36"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 03:05:02.948622 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:02.948553    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43404426-b4aa-4af7-9624-d7df9402df36-kube-api-access-4w44h" (OuterVolumeSpecName: "kube-api-access-4w44h") pod "43404426-b4aa-4af7-9624-d7df9402df36" (UID: "43404426-b4aa-4af7-9624-d7df9402df36"). InnerVolumeSpecName "kube-api-access-4w44h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 03:05:03.047880 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:03.047842    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43404426-b4aa-4af7-9624-d7df9402df36-home\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:05:03.047880 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:03.047872    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43404426-b4aa-4af7-9624-d7df9402df36-model-cache\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:05:03.047880 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:03.047881    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43404426-b4aa-4af7-9624-d7df9402df36-dshm\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:05:03.047880 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:03.047889    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43404426-b4aa-4af7-9624-d7df9402df36-tls-certs\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:05:03.048128 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:03.047896    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43404426-b4aa-4af7-9624-d7df9402df36-tmp-dir\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:05:03.048128 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:03.047907    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4w44h\" (UniqueName: \"kubernetes.io/projected/43404426-b4aa-4af7-9624-d7df9402df36-kube-api-access-4w44h\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:05:03.219803 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:03.219692    2581 generic.go:356] "Generic (PLEG): container finished" podID="43404426-b4aa-4af7-9624-d7df9402df36" containerID="87fb080b1fc06b3d378707acdf39ca50d2932683f07a006fa44a9eb6aa7f0db1" exitCode=0
Jul 22 03:05:03.219803 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:03.219763    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-995f8f/sub-test-model-995f8f-kserve-7754f69997-sbtmz"
Jul 22 03:05:03.219803 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:03.219781    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-995f8f/sub-test-model-995f8f-kserve-7754f69997-sbtmz" event={"ID":"43404426-b4aa-4af7-9624-d7df9402df36","Type":"ContainerDied","Data":"87fb080b1fc06b3d378707acdf39ca50d2932683f07a006fa44a9eb6aa7f0db1"}
Jul 22 03:05:03.220057 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:03.219824    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-995f8f/sub-test-model-995f8f-kserve-7754f69997-sbtmz" event={"ID":"43404426-b4aa-4af7-9624-d7df9402df36","Type":"ContainerDied","Data":"f83a7baaf38833a698302d8da64105f83c3f7ec565bbcea177713250d63d20cd"}
Jul 22 03:05:03.220057 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:03.219841    2581 scope.go:117] "RemoveContainer" containerID="87fb080b1fc06b3d378707acdf39ca50d2932683f07a006fa44a9eb6aa7f0db1"
Jul 22 03:05:03.229261 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:03.229243    2581 scope.go:117] "RemoveContainer" containerID="87fb080b1fc06b3d378707acdf39ca50d2932683f07a006fa44a9eb6aa7f0db1"
Jul 22 03:05:03.229512 ip-10-0-138-202 kubenswrapper[2581]: E0722 03:05:03.229494    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87fb080b1fc06b3d378707acdf39ca50d2932683f07a006fa44a9eb6aa7f0db1\": container with ID starting with 87fb080b1fc06b3d378707acdf39ca50d2932683f07a006fa44a9eb6aa7f0db1 not found: ID does not exist" containerID="87fb080b1fc06b3d378707acdf39ca50d2932683f07a006fa44a9eb6aa7f0db1"
Jul 22 03:05:03.229557 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:03.229521    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87fb080b1fc06b3d378707acdf39ca50d2932683f07a006fa44a9eb6aa7f0db1"} err="failed to get container status \"87fb080b1fc06b3d378707acdf39ca50d2932683f07a006fa44a9eb6aa7f0db1\": rpc error: code = NotFound desc = could not find container \"87fb080b1fc06b3d378707acdf39ca50d2932683f07a006fa44a9eb6aa7f0db1\": container with ID starting with 87fb080b1fc06b3d378707acdf39ca50d2932683f07a006fa44a9eb6aa7f0db1 not found: ID does not exist"
Jul 22 03:05:03.242824 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:03.242801    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-995f8f/sub-test-model-995f8f-kserve-7754f69997-sbtmz"]
Jul 22 03:05:03.246752 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:03.246718    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-995f8f/sub-test-model-995f8f-kserve-7754f69997-sbtmz"]
Jul 22 03:05:03.375267 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:03.375206    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-bb8c9d/sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g" podUID="9b51b586-d987-438b-ac8b-9a8a2bd3aee5" containerName="main" containerID="cri-o://1ed0fb555505254c7a4f07c5941d21cdaf86aef6c38413f0854a8815f7bd12ff" gracePeriod=45
Jul 22 03:05:03.632142 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:03.632115    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bb8c9d/sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g"
Jul 22 03:05:03.753511 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:03.753414    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pgx6w\" (UniqueName: \"kubernetes.io/projected/9b51b586-d987-438b-ac8b-9a8a2bd3aee5-kube-api-access-pgx6w\") pod \"9b51b586-d987-438b-ac8b-9a8a2bd3aee5\" (UID: \"9b51b586-d987-438b-ac8b-9a8a2bd3aee5\") "
Jul 22 03:05:03.753511 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:03.753467    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b51b586-d987-438b-ac8b-9a8a2bd3aee5-dshm\") pod \"9b51b586-d987-438b-ac8b-9a8a2bd3aee5\" (UID: \"9b51b586-d987-438b-ac8b-9a8a2bd3aee5\") "
Jul 22 03:05:03.753511 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:03.753484    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b51b586-d987-438b-ac8b-9a8a2bd3aee5-home\") pod \"9b51b586-d987-438b-ac8b-9a8a2bd3aee5\" (UID: \"9b51b586-d987-438b-ac8b-9a8a2bd3aee5\") "
Jul 22 03:05:03.753840 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:03.753522    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b51b586-d987-438b-ac8b-9a8a2bd3aee5-tmp-dir\") pod \"9b51b586-d987-438b-ac8b-9a8a2bd3aee5\" (UID: \"9b51b586-d987-438b-ac8b-9a8a2bd3aee5\") "
Jul 22 03:05:03.753840 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:03.753544    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b51b586-d987-438b-ac8b-9a8a2bd3aee5-model-cache\") pod \"9b51b586-d987-438b-ac8b-9a8a2bd3aee5\" (UID: \"9b51b586-d987-438b-ac8b-9a8a2bd3aee5\") "
Jul 22 03:05:03.753840 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:03.753609    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b51b586-d987-438b-ac8b-9a8a2bd3aee5-tls-certs\") pod \"9b51b586-d987-438b-ac8b-9a8a2bd3aee5\" (UID: \"9b51b586-d987-438b-ac8b-9a8a2bd3aee5\") "
Jul 22 03:05:03.753840 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:03.753828    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b51b586-d987-438b-ac8b-9a8a2bd3aee5-home" (OuterVolumeSpecName: "home") pod "9b51b586-d987-438b-ac8b-9a8a2bd3aee5" (UID: "9b51b586-d987-438b-ac8b-9a8a2bd3aee5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 03:05:03.754061 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:03.753866    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b51b586-d987-438b-ac8b-9a8a2bd3aee5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9b51b586-d987-438b-ac8b-9a8a2bd3aee5" (UID: "9b51b586-d987-438b-ac8b-9a8a2bd3aee5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 03:05:03.754061 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:03.753882    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b51b586-d987-438b-ac8b-9a8a2bd3aee5-model-cache" (OuterVolumeSpecName: "model-cache") pod "9b51b586-d987-438b-ac8b-9a8a2bd3aee5" (UID: "9b51b586-d987-438b-ac8b-9a8a2bd3aee5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 03:05:03.754061 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:03.754050    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b51b586-d987-438b-ac8b-9a8a2bd3aee5-home\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:05:03.754223 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:03.754071    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b51b586-d987-438b-ac8b-9a8a2bd3aee5-tmp-dir\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:05:03.754223 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:03.754087    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b51b586-d987-438b-ac8b-9a8a2bd3aee5-model-cache\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:05:03.755639 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:03.755610    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b51b586-d987-438b-ac8b-9a8a2bd3aee5-dshm" (OuterVolumeSpecName: "dshm") pod "9b51b586-d987-438b-ac8b-9a8a2bd3aee5" (UID: "9b51b586-d987-438b-ac8b-9a8a2bd3aee5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 03:05:03.755787 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:03.755707    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b51b586-d987-438b-ac8b-9a8a2bd3aee5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9b51b586-d987-438b-ac8b-9a8a2bd3aee5" (UID: "9b51b586-d987-438b-ac8b-9a8a2bd3aee5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 03:05:03.755787 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:03.755721    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b51b586-d987-438b-ac8b-9a8a2bd3aee5-kube-api-access-pgx6w" (OuterVolumeSpecName: "kube-api-access-pgx6w") pod "9b51b586-d987-438b-ac8b-9a8a2bd3aee5" (UID: "9b51b586-d987-438b-ac8b-9a8a2bd3aee5"). InnerVolumeSpecName "kube-api-access-pgx6w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 03:05:03.854925 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:03.854887    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgx6w\" (UniqueName: \"kubernetes.io/projected/9b51b586-d987-438b-ac8b-9a8a2bd3aee5-kube-api-access-pgx6w\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:05:03.854925 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:03.854920    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b51b586-d987-438b-ac8b-9a8a2bd3aee5-dshm\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:05:03.854925 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:03.854932    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b51b586-d987-438b-ac8b-9a8a2bd3aee5-tls-certs\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:05:04.227673 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:04.227577    2581 generic.go:356] "Generic (PLEG): container finished" podID="9b51b586-d987-438b-ac8b-9a8a2bd3aee5" containerID="1ed0fb555505254c7a4f07c5941d21cdaf86aef6c38413f0854a8815f7bd12ff" exitCode=0
Jul 22 03:05:04.227673 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:04.227620    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bb8c9d/sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g"
Jul 22 03:05:04.227673 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:04.227652    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bb8c9d/sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g" event={"ID":"9b51b586-d987-438b-ac8b-9a8a2bd3aee5","Type":"ContainerDied","Data":"1ed0fb555505254c7a4f07c5941d21cdaf86aef6c38413f0854a8815f7bd12ff"}
Jul 22 03:05:04.227974 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:04.227693    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bb8c9d/sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g" event={"ID":"9b51b586-d987-438b-ac8b-9a8a2bd3aee5","Type":"ContainerDied","Data":"60365cb233d2c0115c3446bb25f0924df1d75934346e2bb04af14d49cef7b5b3"}
Jul 22 03:05:04.227974 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:04.227714    2581 scope.go:117] "RemoveContainer" containerID="1ed0fb555505254c7a4f07c5941d21cdaf86aef6c38413f0854a8815f7bd12ff"
Jul 22 03:05:04.237248 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:04.237230    2581 scope.go:117] "RemoveContainer" containerID="1ed0fb555505254c7a4f07c5941d21cdaf86aef6c38413f0854a8815f7bd12ff"
Jul 22 03:05:04.237512 ip-10-0-138-202 kubenswrapper[2581]: E0722 03:05:04.237491    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ed0fb555505254c7a4f07c5941d21cdaf86aef6c38413f0854a8815f7bd12ff\": container with ID starting with 1ed0fb555505254c7a4f07c5941d21cdaf86aef6c38413f0854a8815f7bd12ff not found: ID does not exist" containerID="1ed0fb555505254c7a4f07c5941d21cdaf86aef6c38413f0854a8815f7bd12ff"
Jul 22 03:05:04.237557 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:04.237521    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ed0fb555505254c7a4f07c5941d21cdaf86aef6c38413f0854a8815f7bd12ff"} err="failed to get container status \"1ed0fb555505254c7a4f07c5941d21cdaf86aef6c38413f0854a8815f7bd12ff\": rpc error: code = NotFound desc = could not find container \"1ed0fb555505254c7a4f07c5941d21cdaf86aef6c38413f0854a8815f7bd12ff\": container with ID starting with 1ed0fb555505254c7a4f07c5941d21cdaf86aef6c38413f0854a8815f7bd12ff not found: ID does not exist"
Jul 22 03:05:04.249419 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:04.249397    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-bb8c9d/sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g"]
Jul 22 03:05:04.256096 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:04.256076    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-bb8c9d/sub-test-model-bb8c9d-kserve-57d4d9448c-tx98g"]
Jul 22 03:05:04.904115 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:04.904081    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43404426-b4aa-4af7-9624-d7df9402df36" path="/var/lib/kubelet/pods/43404426-b4aa-4af7-9624-d7df9402df36/volumes"
Jul 22 03:05:04.904506 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:04.904477    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b51b586-d987-438b-ac8b-9a8a2bd3aee5" path="/var/lib/kubelet/pods/9b51b586-d987-438b-ac8b-9a8a2bd3aee5/volumes"
Jul 22 03:05:06.527818 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:06.527773    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:05:06.628526 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:06.628492    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:05:07.671039 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:07.671008    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:05:07.768609 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:07.768561    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-995f8f/rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc"
Jul 22 03:05:07.768841 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:07.768630    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-995f8f/rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc"
Jul 22 03:05:07.780769 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:07.780728    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-995f8f/rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc"
Jul 22 03:05:08.258994 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:08.258958    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-995f8f/rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc"
Jul 22 03:05:25.672406 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:25.672370    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:05:25.769782 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:25.769731    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:05:29.151602 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:29.151568    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-bb8c9d/rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg"]
Jul 22 03:05:29.152069 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:29.151956    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43404426-b4aa-4af7-9624-d7df9402df36" containerName="main"
Jul 22 03:05:29.152069 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:29.151969    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="43404426-b4aa-4af7-9624-d7df9402df36" containerName="main"
Jul 22 03:05:29.152069 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:29.152001    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b51b586-d987-438b-ac8b-9a8a2bd3aee5" containerName="main"
Jul 22 03:05:29.152069 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:29.152008    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b51b586-d987-438b-ac8b-9a8a2bd3aee5" containerName="main"
Jul 22 03:05:29.152308 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:29.152073    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="43404426-b4aa-4af7-9624-d7df9402df36" containerName="main"
Jul 22 03:05:29.152308 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:29.152087    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b51b586-d987-438b-ac8b-9a8a2bd3aee5" containerName="main"
Jul 22 03:05:29.155130 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:29.155113    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bb8c9d/rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg"
Jul 22 03:05:29.159078 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:29.159055    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-bb8c9d\"/\"rate-test-model-bb8c9d-kserve-self-signed-certs\""
Jul 22 03:05:29.159453 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:29.159060    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-bb8c9d\"/\"openshift-service-ca.crt\""
Jul 22 03:05:29.159453 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:29.159159    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-bb8c9d\"/\"kube-root-ca.crt\""
Jul 22 03:05:29.159453 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:29.159216    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-bb8c9d\"/\"default-dockercfg-mbfrf\""
Jul 22 03:05:29.166923 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:29.166896    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-bb8c9d/rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg"]
Jul 22 03:05:29.177041 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:29.177016    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:05:29.277527 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:29.277495    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/775f9c2e-836b-4613-b3ba-fce22b14f176-dshm\") pod \"rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg\" (UID: \"775f9c2e-836b-4613-b3ba-fce22b14f176\") " pod="ai-tenant-e2e-shared-b-bb8c9d/rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg"
Jul 22 03:05:29.277729 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:29.277551    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzk7h\" (UniqueName: \"kubernetes.io/projected/775f9c2e-836b-4613-b3ba-fce22b14f176-kube-api-access-gzk7h\") pod \"rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg\" (UID: \"775f9c2e-836b-4613-b3ba-fce22b14f176\") " pod="ai-tenant-e2e-shared-b-bb8c9d/rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg"
Jul 22 03:05:29.277729 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:29.277579    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/775f9c2e-836b-4613-b3ba-fce22b14f176-tls-certs\") pod \"rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg\" (UID: \"775f9c2e-836b-4613-b3ba-fce22b14f176\") " pod="ai-tenant-e2e-shared-b-bb8c9d/rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg"
Jul 22 03:05:29.277729 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:29.277646    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/775f9c2e-836b-4613-b3ba-fce22b14f176-model-cache\") pod \"rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg\" (UID: \"775f9c2e-836b-4613-b3ba-fce22b14f176\") " pod="ai-tenant-e2e-shared-b-bb8c9d/rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg"
Jul 22 03:05:29.277915 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:29.277729    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/775f9c2e-836b-4613-b3ba-fce22b14f176-home\") pod \"rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg\" (UID: \"775f9c2e-836b-4613-b3ba-fce22b14f176\") " pod="ai-tenant-e2e-shared-b-bb8c9d/rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg"
Jul 22 03:05:29.277915 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:29.277791    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/775f9c2e-836b-4613-b3ba-fce22b14f176-tmp-dir\") pod \"rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg\" (UID: \"775f9c2e-836b-4613-b3ba-fce22b14f176\") " pod="ai-tenant-e2e-shared-b-bb8c9d/rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg"
Jul 22 03:05:29.379266 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:29.379235    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/775f9c2e-836b-4613-b3ba-fce22b14f176-home\") pod \"rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg\" (UID: \"775f9c2e-836b-4613-b3ba-fce22b14f176\") " pod="ai-tenant-e2e-shared-b-bb8c9d/rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg"
Jul 22 03:05:29.379449 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:29.379277    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/775f9c2e-836b-4613-b3ba-fce22b14f176-tmp-dir\") pod \"rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg\" (UID: \"775f9c2e-836b-4613-b3ba-fce22b14f176\") " pod="ai-tenant-e2e-shared-b-bb8c9d/rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg"
Jul 22 03:05:29.379449 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:29.379312    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/775f9c2e-836b-4613-b3ba-fce22b14f176-dshm\") pod \"rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg\" (UID: \"775f9c2e-836b-4613-b3ba-fce22b14f176\") " pod="ai-tenant-e2e-shared-b-bb8c9d/rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg"
Jul 22 03:05:29.379449 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:29.379352    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzk7h\" (UniqueName: \"kubernetes.io/projected/775f9c2e-836b-4613-b3ba-fce22b14f176-kube-api-access-gzk7h\") pod \"rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg\" (UID: \"775f9c2e-836b-4613-b3ba-fce22b14f176\") " pod="ai-tenant-e2e-shared-b-bb8c9d/rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg"
Jul 22 03:05:29.379569 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:29.379472    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/775f9c2e-836b-4613-b3ba-fce22b14f176-tls-certs\") pod \"rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg\" (UID: \"775f9c2e-836b-4613-b3ba-fce22b14f176\") " pod="ai-tenant-e2e-shared-b-bb8c9d/rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg"
Jul 22 03:05:29.379569 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:29.379513    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/775f9c2e-836b-4613-b3ba-fce22b14f176-model-cache\") pod \"rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg\" (UID: \"775f9c2e-836b-4613-b3ba-fce22b14f176\") " pod="ai-tenant-e2e-shared-b-bb8c9d/rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg"
Jul 22 03:05:29.379699 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:29.379677    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/775f9c2e-836b-4613-b3ba-fce22b14f176-home\") pod \"rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg\" (UID: \"775f9c2e-836b-4613-b3ba-fce22b14f176\") " pod="ai-tenant-e2e-shared-b-bb8c9d/rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg"
Jul 22 03:05:29.379796 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:29.379695    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/775f9c2e-836b-4613-b3ba-fce22b14f176-tmp-dir\") pod \"rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg\" (UID: \"775f9c2e-836b-4613-b3ba-fce22b14f176\") " pod="ai-tenant-e2e-shared-b-bb8c9d/rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg"
Jul 22 03:05:29.379879 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:29.379854    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/775f9c2e-836b-4613-b3ba-fce22b14f176-model-cache\") pod \"rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg\" (UID: \"775f9c2e-836b-4613-b3ba-fce22b14f176\") " pod="ai-tenant-e2e-shared-b-bb8c9d/rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg"
Jul 22 03:05:29.381577 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:29.381547    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/775f9c2e-836b-4613-b3ba-fce22b14f176-dshm\") pod \"rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg\" (UID: \"775f9c2e-836b-4613-b3ba-fce22b14f176\") " pod="ai-tenant-e2e-shared-b-bb8c9d/rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg"
Jul 22 03:05:29.382014 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:29.381994    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/775f9c2e-836b-4613-b3ba-fce22b14f176-tls-certs\") pod \"rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg\" (UID: \"775f9c2e-836b-4613-b3ba-fce22b14f176\") " pod="ai-tenant-e2e-shared-b-bb8c9d/rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg"
Jul 22 03:05:29.388829 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:29.388813    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzk7h\" (UniqueName: \"kubernetes.io/projected/775f9c2e-836b-4613-b3ba-fce22b14f176-kube-api-access-gzk7h\") pod \"rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg\" (UID: \"775f9c2e-836b-4613-b3ba-fce22b14f176\") " pod="ai-tenant-e2e-shared-b-bb8c9d/rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg"
Jul 22 03:05:29.466309 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:29.466207    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bb8c9d/rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg"
Jul 22 03:05:29.601242 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:29.601204    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-bb8c9d/rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg"]
Jul 22 03:05:29.604709 ip-10-0-138-202 kubenswrapper[2581]: W0722 03:05:29.604683    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod775f9c2e_836b_4613_b3ba_fce22b14f176.slice/crio-2b07ea1663d0fa32eb9c7446ebfae868e9dc60500d5649b32812b6bdf249a39d WatchSource:0}: Error finding container 2b07ea1663d0fa32eb9c7446ebfae868e9dc60500d5649b32812b6bdf249a39d: Status 404 returned error can't find the container with id 2b07ea1663d0fa32eb9c7446ebfae868e9dc60500d5649b32812b6bdf249a39d
Jul 22 03:05:30.365340 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:30.365298    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bb8c9d/rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg" event={"ID":"775f9c2e-836b-4613-b3ba-fce22b14f176","Type":"ContainerStarted","Data":"423d1682339f869293999bcbba851526b06dcf4bcd9322a2be13b6f91bd5d62c"}
Jul 22 03:05:30.365340 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:30.365346    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bb8c9d/rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg" event={"ID":"775f9c2e-836b-4613-b3ba-fce22b14f176","Type":"ContainerStarted","Data":"2b07ea1663d0fa32eb9c7446ebfae868e9dc60500d5649b32812b6bdf249a39d"}
Jul 22 03:05:30.385635 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:30.385569    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-bb8c9d/rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg" podStartSLOduration=1.385549824 podStartE2EDuration="1.385549824s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:05:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:05:30.384105272 +0000 UTC m=+2751.924898901" watchObservedRunningTime="2026-07-22 03:05:30.385549824 +0000 UTC m=+2751.926343448"
Jul 22 03:05:32.447475 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:32.447438    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 03:05:33.313002 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:33.312972    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:05:33.413951 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:33.413915    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:05:39.137839 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:39.137809    2581 scope.go:117] "RemoveContainer" containerID="da7b4647cce7485a8bbf62569189e8e8302cc0f2352be8f4a33fa1a1573234fb"
Jul 22 03:05:39.149037 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:39.149012    2581 scope.go:117] "RemoveContainer" containerID="640e8e7c69a164ad6b5291e98ee1fe97feb78509c6afa8f68b6fe86ff1e67242"
Jul 22 03:05:39.466592 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:39.466509    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-bb8c9d/rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg"
Jul 22 03:05:39.466592 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:39.466561    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-bb8c9d/rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg"
Jul 22 03:05:39.478862 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:39.478839    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-bb8c9d/rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg"
Jul 22 03:05:40.424118 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:40.424088    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-bb8c9d/rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg"
Jul 22 03:05:46.644637 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:05:46.644600    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 03:06:00.192998 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:00.192967    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:06:00.288773 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:00.288721    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:06:01.858850 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:01.858772    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 03:06:09.709166 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:09.709128    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-995f8f/rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc"]
Jul 22 03:06:09.734816 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:09.734789    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:06:10.528050 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:10.528015    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-bb8c9d/rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg"]
Jul 22 03:06:10.558121 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:10.558085    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:06:11.513201 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:11.513157    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7511-140675580476208"]
Jul 22 03:06:11.518288 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:11.518263    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7511-140675580476208"
Jul 22 03:06:11.522661 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:11.522635    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7511-140675580476208"]
Jul 22 03:06:11.525251 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:11.525228    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-xzbkk\""
Jul 22 03:06:11.590091 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:11.590060    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58sz9\" (UniqueName: \"kubernetes.io/projected/dcde846e-2192-4d75-8fff-84885aa48c8c-kube-api-access-58sz9\") pod \"test-curl-7511-140675580476208\" (UID: \"dcde846e-2192-4d75-8fff-84885aa48c8c\") " pod="opendatahub/test-curl-7511-140675580476208"
Jul 22 03:06:11.690893 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:11.690843    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58sz9\" (UniqueName: \"kubernetes.io/projected/dcde846e-2192-4d75-8fff-84885aa48c8c-kube-api-access-58sz9\") pod \"test-curl-7511-140675580476208\" (UID: \"dcde846e-2192-4d75-8fff-84885aa48c8c\") " pod="opendatahub/test-curl-7511-140675580476208"
Jul 22 03:06:11.700780 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:11.700722    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58sz9\" (UniqueName: \"kubernetes.io/projected/dcde846e-2192-4d75-8fff-84885aa48c8c-kube-api-access-58sz9\") pod \"test-curl-7511-140675580476208\" (UID: \"dcde846e-2192-4d75-8fff-84885aa48c8c\") " pod="opendatahub/test-curl-7511-140675580476208"
Jul 22 03:06:11.829270 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:11.829184    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7511-140675580476208"
Jul 22 03:06:11.967379 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:11.967346    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7511-140675580476208"]
Jul 22 03:06:11.970118 ip-10-0-138-202 kubenswrapper[2581]: W0722 03:06:11.970094    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddcde846e_2192_4d75_8fff_84885aa48c8c.slice/crio-256e4eec3621a8ddc02d976c3f854dc875554b68475f41c8604cf37d7a5c0c7f WatchSource:0}: Error finding container 256e4eec3621a8ddc02d976c3f854dc875554b68475f41c8604cf37d7a5c0c7f: Status 404 returned error can't find the container with id 256e4eec3621a8ddc02d976c3f854dc875554b68475f41c8604cf37d7a5c0c7f
Jul 22 03:06:11.971776 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:11.971759    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 03:06:12.573386 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:12.573352    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7511-140675580476208" event={"ID":"dcde846e-2192-4d75-8fff-84885aa48c8c","Type":"ContainerStarted","Data":"256e4eec3621a8ddc02d976c3f854dc875554b68475f41c8604cf37d7a5c0c7f"}
Jul 22 03:06:13.580956 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:13.580921    2581 generic.go:356] "Generic (PLEG): container finished" podID="dcde846e-2192-4d75-8fff-84885aa48c8c" containerID="7bf15ae59fec1a14244324d4a66cf3409183c89ad749e9d1c8935593e76800b2" exitCode=0
Jul 22 03:06:13.581351 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:13.581007    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7511-140675580476208" event={"ID":"dcde846e-2192-4d75-8fff-84885aa48c8c","Type":"ContainerDied","Data":"7bf15ae59fec1a14244324d4a66cf3409183c89ad749e9d1c8935593e76800b2"}
Jul 22 03:06:14.717428 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:14.717407    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7511-140675580476208"
Jul 22 03:06:14.743626 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:14.743601    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7511-140675580476208_dcde846e-2192-4d75-8fff-84885aa48c8c/test-curl-7511-140675580476208/0.log"
Jul 22 03:06:14.770666 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:14.770641    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7511-140675580476208"]
Jul 22 03:06:14.777296 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:14.777275    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7511-140675580476208"]
Jul 22 03:06:14.820705 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:14.820677    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-58sz9\" (UniqueName: \"kubernetes.io/projected/dcde846e-2192-4d75-8fff-84885aa48c8c-kube-api-access-58sz9\") pod \"dcde846e-2192-4d75-8fff-84885aa48c8c\" (UID: \"dcde846e-2192-4d75-8fff-84885aa48c8c\") "
Jul 22 03:06:14.823019 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:14.822990    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcde846e-2192-4d75-8fff-84885aa48c8c-kube-api-access-58sz9" (OuterVolumeSpecName: "kube-api-access-58sz9") pod "dcde846e-2192-4d75-8fff-84885aa48c8c" (UID: "dcde846e-2192-4d75-8fff-84885aa48c8c"). InnerVolumeSpecName "kube-api-access-58sz9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 03:06:14.904317 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:14.904230    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dcde846e-2192-4d75-8fff-84885aa48c8c" path="/var/lib/kubelet/pods/dcde846e-2192-4d75-8fff-84885aa48c8c/volumes"
Jul 22 03:06:14.910608 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:14.910577    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7511-140675580476336"]
Jul 22 03:06:14.910984 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:14.910968    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcde846e-2192-4d75-8fff-84885aa48c8c" containerName="test-curl-7511-140675580476208"
Jul 22 03:06:14.911030 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:14.910986    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcde846e-2192-4d75-8fff-84885aa48c8c" containerName="test-curl-7511-140675580476208"
Jul 22 03:06:14.911069 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:14.911061    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcde846e-2192-4d75-8fff-84885aa48c8c" containerName="test-curl-7511-140675580476208"
Jul 22 03:06:14.914217 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:14.914200    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7511-140675580476336"
Jul 22 03:06:14.921486 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:14.921466    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7511-140675580476336"]
Jul 22 03:06:14.921651 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:14.921633    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58sz9\" (UniqueName: \"kubernetes.io/projected/dcde846e-2192-4d75-8fff-84885aa48c8c-kube-api-access-58sz9\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:06:15.023089 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:15.023058    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbwbf\" (UniqueName: \"kubernetes.io/projected/faedccd9-f0e6-4fb0-9feb-2f0366986989-kube-api-access-fbwbf\") pod \"test-curl-7511-140675580476336\" (UID: \"faedccd9-f0e6-4fb0-9feb-2f0366986989\") " pod="opendatahub/test-curl-7511-140675580476336"
Jul 22 03:06:15.124367 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:15.124329    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbwbf\" (UniqueName: \"kubernetes.io/projected/faedccd9-f0e6-4fb0-9feb-2f0366986989-kube-api-access-fbwbf\") pod \"test-curl-7511-140675580476336\" (UID: \"faedccd9-f0e6-4fb0-9feb-2f0366986989\") " pod="opendatahub/test-curl-7511-140675580476336"
Jul 22 03:06:15.134272 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:15.134252    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbwbf\" (UniqueName: \"kubernetes.io/projected/faedccd9-f0e6-4fb0-9feb-2f0366986989-kube-api-access-fbwbf\") pod \"test-curl-7511-140675580476336\" (UID: \"faedccd9-f0e6-4fb0-9feb-2f0366986989\") " pod="opendatahub/test-curl-7511-140675580476336"
Jul 22 03:06:15.223262 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:15.223181    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7511-140675580476336"
Jul 22 03:06:15.351762 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:15.351704    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7511-140675580476336"]
Jul 22 03:06:15.354093 ip-10-0-138-202 kubenswrapper[2581]: W0722 03:06:15.354065    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfaedccd9_f0e6_4fb0_9feb_2f0366986989.slice/crio-9dd5e9f073160511216d7174b43a33d5a55b83bf8523712c9b07a3acb6b72003 WatchSource:0}: Error finding container 9dd5e9f073160511216d7174b43a33d5a55b83bf8523712c9b07a3acb6b72003: Status 404 returned error can't find the container with id 9dd5e9f073160511216d7174b43a33d5a55b83bf8523712c9b07a3acb6b72003
Jul 22 03:06:15.591476 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:15.591444    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7511-140675580476208"
Jul 22 03:06:15.591682 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:15.591484    2581 scope.go:117] "RemoveContainer" containerID="7bf15ae59fec1a14244324d4a66cf3409183c89ad749e9d1c8935593e76800b2"
Jul 22 03:06:15.592857 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:15.592684    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7511-140675580476336" event={"ID":"faedccd9-f0e6-4fb0-9feb-2f0366986989","Type":"ContainerStarted","Data":"9dd5e9f073160511216d7174b43a33d5a55b83bf8523712c9b07a3acb6b72003"}
Jul 22 03:06:16.360165 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:16.360130    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 03:06:16.600164 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:16.600125    2581 generic.go:356] "Generic (PLEG): container finished" podID="faedccd9-f0e6-4fb0-9feb-2f0366986989" containerID="e119481e7041e272c11eeeb1e06eb3fb29a96cf315d98b9db6c5f6ed10caaf18" exitCode=0
Jul 22 03:06:16.600315 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:16.600168    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7511-140675580476336" event={"ID":"faedccd9-f0e6-4fb0-9feb-2f0366986989","Type":"ContainerDied","Data":"e119481e7041e272c11eeeb1e06eb3fb29a96cf315d98b9db6c5f6ed10caaf18"}
Jul 22 03:06:17.740322 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:17.740293    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7511-140675580476336"
Jul 22 03:06:17.785457 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:17.785424    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7511-140675580476336_faedccd9-f0e6-4fb0-9feb-2f0366986989/test-curl-7511-140675580476336/0.log"
Jul 22 03:06:17.821580 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:17.821547    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7511-140675580476336"]
Jul 22 03:06:17.824235 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:17.824206    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7511-140675580476336"]
Jul 22 03:06:17.853706 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:17.853679    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fbwbf\" (UniqueName: \"kubernetes.io/projected/faedccd9-f0e6-4fb0-9feb-2f0366986989-kube-api-access-fbwbf\") pod \"faedccd9-f0e6-4fb0-9feb-2f0366986989\" (UID: \"faedccd9-f0e6-4fb0-9feb-2f0366986989\") "
Jul 22 03:06:17.855900 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:17.855830    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/faedccd9-f0e6-4fb0-9feb-2f0366986989-kube-api-access-fbwbf" (OuterVolumeSpecName: "kube-api-access-fbwbf") pod "faedccd9-f0e6-4fb0-9feb-2f0366986989" (UID: "faedccd9-f0e6-4fb0-9feb-2f0366986989"). InnerVolumeSpecName "kube-api-access-fbwbf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 03:06:17.954673 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:17.954632    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fbwbf\" (UniqueName: \"kubernetes.io/projected/faedccd9-f0e6-4fb0-9feb-2f0366986989-kube-api-access-fbwbf\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:06:17.990704 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:17.990670    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7511-140675580388384"]
Jul 22 03:06:17.991115 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:17.991102    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="faedccd9-f0e6-4fb0-9feb-2f0366986989" containerName="test-curl-7511-140675580476336"
Jul 22 03:06:17.991159 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:17.991117    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="faedccd9-f0e6-4fb0-9feb-2f0366986989" containerName="test-curl-7511-140675580476336"
Jul 22 03:06:17.991239 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:17.991197    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="faedccd9-f0e6-4fb0-9feb-2f0366986989" containerName="test-curl-7511-140675580476336"
Jul 22 03:06:17.994221 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:17.994202    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7511-140675580388384"
Jul 22 03:06:18.009061 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:18.009032    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7511-140675580388384"]
Jul 22 03:06:18.156575 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:18.156489    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhgk6\" (UniqueName: \"kubernetes.io/projected/47c6e5d1-8c0f-47ee-b114-f8ea933418a8-kube-api-access-mhgk6\") pod \"test-curl-7511-140675580388384\" (UID: \"47c6e5d1-8c0f-47ee-b114-f8ea933418a8\") " pod="opendatahub/test-curl-7511-140675580388384"
Jul 22 03:06:18.257473 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:18.257437    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhgk6\" (UniqueName: \"kubernetes.io/projected/47c6e5d1-8c0f-47ee-b114-f8ea933418a8-kube-api-access-mhgk6\") pod \"test-curl-7511-140675580388384\" (UID: \"47c6e5d1-8c0f-47ee-b114-f8ea933418a8\") " pod="opendatahub/test-curl-7511-140675580388384"
Jul 22 03:06:18.267865 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:18.267834    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhgk6\" (UniqueName: \"kubernetes.io/projected/47c6e5d1-8c0f-47ee-b114-f8ea933418a8-kube-api-access-mhgk6\") pod \"test-curl-7511-140675580388384\" (UID: \"47c6e5d1-8c0f-47ee-b114-f8ea933418a8\") " pod="opendatahub/test-curl-7511-140675580388384"
Jul 22 03:06:18.303481 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:18.303439    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7511-140675580388384"
Jul 22 03:06:18.433729 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:18.433690    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7511-140675580388384"]
Jul 22 03:06:18.435954 ip-10-0-138-202 kubenswrapper[2581]: W0722 03:06:18.435908    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod47c6e5d1_8c0f_47ee_b114_f8ea933418a8.slice/crio-c2ec6f2259f09d57f96e1a1bbd801ab826ae22f567d43bb73af93ff24ec893d7 WatchSource:0}: Error finding container c2ec6f2259f09d57f96e1a1bbd801ab826ae22f567d43bb73af93ff24ec893d7: Status 404 returned error can't find the container with id c2ec6f2259f09d57f96e1a1bbd801ab826ae22f567d43bb73af93ff24ec893d7
Jul 22 03:06:18.611118 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:18.611085    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9dd5e9f073160511216d7174b43a33d5a55b83bf8523712c9b07a3acb6b72003"
Jul 22 03:06:18.611118 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:18.611102    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7511-140675580476336"
Jul 22 03:06:18.612284 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:18.612256    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7511-140675580388384" event={"ID":"47c6e5d1-8c0f-47ee-b114-f8ea933418a8","Type":"ContainerStarted","Data":"c2ec6f2259f09d57f96e1a1bbd801ab826ae22f567d43bb73af93ff24ec893d7"}
Jul 22 03:06:18.622317 ip-10-0-138-202 kubenswrapper[2581]: E0722 03:06:18.622293    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7511-140675580476336\" is forbidden: User \"system:node:ip-10-0-138-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-202.ec2.internal' and this object" podUID="faedccd9-f0e6-4fb0-9feb-2f0366986989" pod="opendatahub/test-curl-7511-140675580476336"
Jul 22 03:06:18.902439 ip-10-0-138-202 kubenswrapper[2581]: E0722 03:06:18.902405    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7511-140675580476336\" is forbidden: User \"system:node:ip-10-0-138-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-202.ec2.internal' and this object" podUID="faedccd9-f0e6-4fb0-9feb-2f0366986989" pod="opendatahub/test-curl-7511-140675580476336"
Jul 22 03:06:18.903497 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:18.903476    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="faedccd9-f0e6-4fb0-9feb-2f0366986989" path="/var/lib/kubelet/pods/faedccd9-f0e6-4fb0-9feb-2f0366986989/volumes"
Jul 22 03:06:19.618837 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:19.618801    2581 generic.go:356] "Generic (PLEG): container finished" podID="47c6e5d1-8c0f-47ee-b114-f8ea933418a8" containerID="9e603f5b048e7ab3773b5aa4ed64dde6b5b33294391eabc376b01038f2351862" exitCode=0
Jul 22 03:06:19.619022 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:19.618904    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7511-140675580388384" event={"ID":"47c6e5d1-8c0f-47ee-b114-f8ea933418a8","Type":"ContainerDied","Data":"9e603f5b048e7ab3773b5aa4ed64dde6b5b33294391eabc376b01038f2351862"}
Jul 22 03:06:20.758921 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:20.758899    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7511-140675580388384"
Jul 22 03:06:20.783950 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:20.783925    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7511-140675580388384_47c6e5d1-8c0f-47ee-b114-f8ea933418a8/test-curl-7511-140675580388384/0.log"
Jul 22 03:06:20.812348 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:20.812318    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7511-140675580388384"]
Jul 22 03:06:20.816338 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:20.816315    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7511-140675580388384"]
Jul 22 03:06:20.881622 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:20.881538    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mhgk6\" (UniqueName: \"kubernetes.io/projected/47c6e5d1-8c0f-47ee-b114-f8ea933418a8-kube-api-access-mhgk6\") pod \"47c6e5d1-8c0f-47ee-b114-f8ea933418a8\" (UID: \"47c6e5d1-8c0f-47ee-b114-f8ea933418a8\") "
Jul 22 03:06:20.883629 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:20.883600    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47c6e5d1-8c0f-47ee-b114-f8ea933418a8-kube-api-access-mhgk6" (OuterVolumeSpecName: "kube-api-access-mhgk6") pod "47c6e5d1-8c0f-47ee-b114-f8ea933418a8" (UID: "47c6e5d1-8c0f-47ee-b114-f8ea933418a8"). InnerVolumeSpecName "kube-api-access-mhgk6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 03:06:20.903560 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:20.903535    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47c6e5d1-8c0f-47ee-b114-f8ea933418a8" path="/var/lib/kubelet/pods/47c6e5d1-8c0f-47ee-b114-f8ea933418a8/volumes"
Jul 22 03:06:20.948724 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:20.948697    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7511-140675580388384"]
Jul 22 03:06:20.949139 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:20.949112    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47c6e5d1-8c0f-47ee-b114-f8ea933418a8" containerName="test-curl-7511-140675580388384"
Jul 22 03:06:20.949139 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:20.949140    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="47c6e5d1-8c0f-47ee-b114-f8ea933418a8" containerName="test-curl-7511-140675580388384"
Jul 22 03:06:20.949227 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:20.949196    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="47c6e5d1-8c0f-47ee-b114-f8ea933418a8" containerName="test-curl-7511-140675580388384"
Jul 22 03:06:20.952175 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:20.952160    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7511-140675580388384"
Jul 22 03:06:20.959926 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:20.959886    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7511-140675580388384"]
Jul 22 03:06:20.982858 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:20.982835    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhgk6\" (UniqueName: \"kubernetes.io/projected/47c6e5d1-8c0f-47ee-b114-f8ea933418a8-kube-api-access-mhgk6\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:06:21.083816 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:21.083778    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vt45f\" (UniqueName: \"kubernetes.io/projected/a92235c3-3fcc-4885-ba37-506719fa5234-kube-api-access-vt45f\") pod \"test-curl-7511-140675580388384\" (UID: \"a92235c3-3fcc-4885-ba37-506719fa5234\") " pod="opendatahub/test-curl-7511-140675580388384"
Jul 22 03:06:21.184933 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:21.184845    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt45f\" (UniqueName: \"kubernetes.io/projected/a92235c3-3fcc-4885-ba37-506719fa5234-kube-api-access-vt45f\") pod \"test-curl-7511-140675580388384\" (UID: \"a92235c3-3fcc-4885-ba37-506719fa5234\") " pod="opendatahub/test-curl-7511-140675580388384"
Jul 22 03:06:21.207056 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:21.207027    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vt45f\" (UniqueName: \"kubernetes.io/projected/a92235c3-3fcc-4885-ba37-506719fa5234-kube-api-access-vt45f\") pod \"test-curl-7511-140675580388384\" (UID: \"a92235c3-3fcc-4885-ba37-506719fa5234\") " pod="opendatahub/test-curl-7511-140675580388384"
Jul 22 03:06:21.261283 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:21.261250    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7511-140675580388384"
Jul 22 03:06:21.408351 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:21.408255    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7511-140675580388384"]
Jul 22 03:06:21.411169 ip-10-0-138-202 kubenswrapper[2581]: W0722 03:06:21.411137    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda92235c3_3fcc_4885_ba37_506719fa5234.slice/crio-7aa4b5657149a3d0e2bfd92cff1c7618927f8736eb600e0db1111bbaca36124d WatchSource:0}: Error finding container 7aa4b5657149a3d0e2bfd92cff1c7618927f8736eb600e0db1111bbaca36124d: Status 404 returned error can't find the container with id 7aa4b5657149a3d0e2bfd92cff1c7618927f8736eb600e0db1111bbaca36124d
Jul 22 03:06:21.629905 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:21.629876    2581 scope.go:117] "RemoveContainer" containerID="9e603f5b048e7ab3773b5aa4ed64dde6b5b33294391eabc376b01038f2351862"
Jul 22 03:06:21.630071 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:21.629883    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7511-140675580388384"
Jul 22 03:06:21.631143 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:21.630980    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7511-140675580388384" event={"ID":"a92235c3-3fcc-4885-ba37-506719fa5234","Type":"ContainerStarted","Data":"7aa4b5657149a3d0e2bfd92cff1c7618927f8736eb600e0db1111bbaca36124d"}
Jul 22 03:06:22.638245 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:22.638212    2581 generic.go:356] "Generic (PLEG): container finished" podID="a92235c3-3fcc-4885-ba37-506719fa5234" containerID="6333b34d031bf3c35643934d7214884f9b8880403ec6fe2ea73866466fabb9fb" exitCode=0
Jul 22 03:06:22.638701 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:22.638288    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7511-140675580388384" event={"ID":"a92235c3-3fcc-4885-ba37-506719fa5234","Type":"ContainerDied","Data":"6333b34d031bf3c35643934d7214884f9b8880403ec6fe2ea73866466fabb9fb"}
Jul 22 03:06:23.777171 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:23.777148    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7511-140675580388384"
Jul 22 03:06:23.812647 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:23.812623    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7511-140675580388384_a92235c3-3fcc-4885-ba37-506719fa5234/test-curl-7511-140675580388384/0.log"
Jul 22 03:06:23.843960 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:23.843935    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7511-140675580388384"]
Jul 22 03:06:23.847711 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:23.847686    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7511-140675580388384"]
Jul 22 03:06:23.910417 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:23.910333    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vt45f\" (UniqueName: \"kubernetes.io/projected/a92235c3-3fcc-4885-ba37-506719fa5234-kube-api-access-vt45f\") pod \"a92235c3-3fcc-4885-ba37-506719fa5234\" (UID: \"a92235c3-3fcc-4885-ba37-506719fa5234\") "
Jul 22 03:06:23.912391 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:23.912367    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a92235c3-3fcc-4885-ba37-506719fa5234-kube-api-access-vt45f" (OuterVolumeSpecName: "kube-api-access-vt45f") pod "a92235c3-3fcc-4885-ba37-506719fa5234" (UID: "a92235c3-3fcc-4885-ba37-506719fa5234"). InnerVolumeSpecName "kube-api-access-vt45f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 03:06:23.988793 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:23.988759    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7511-140675580389392"]
Jul 22 03:06:23.989204 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:23.989190    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a92235c3-3fcc-4885-ba37-506719fa5234" containerName="test-curl-7511-140675580388384"
Jul 22 03:06:23.989245 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:23.989207    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a92235c3-3fcc-4885-ba37-506719fa5234" containerName="test-curl-7511-140675580388384"
Jul 22 03:06:23.989285 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:23.989275    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a92235c3-3fcc-4885-ba37-506719fa5234" containerName="test-curl-7511-140675580388384"
Jul 22 03:06:23.992398 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:23.992382    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7511-140675580389392"
Jul 22 03:06:24.007379 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:24.007355    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7511-140675580389392"]
Jul 22 03:06:24.011161 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:24.011141    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vt45f\" (UniqueName: \"kubernetes.io/projected/a92235c3-3fcc-4885-ba37-506719fa5234-kube-api-access-vt45f\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:06:24.112338 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:24.112303    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m8qr\" (UniqueName: \"kubernetes.io/projected/47557c2e-244f-407b-a552-9a3d6429b54f-kube-api-access-5m8qr\") pod \"test-curl-7511-140675580389392\" (UID: \"47557c2e-244f-407b-a552-9a3d6429b54f\") " pod="opendatahub/test-curl-7511-140675580389392"
Jul 22 03:06:24.213062 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:24.212976    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m8qr\" (UniqueName: \"kubernetes.io/projected/47557c2e-244f-407b-a552-9a3d6429b54f-kube-api-access-5m8qr\") pod \"test-curl-7511-140675580389392\" (UID: \"47557c2e-244f-407b-a552-9a3d6429b54f\") " pod="opendatahub/test-curl-7511-140675580389392"
Jul 22 03:06:24.222902 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:24.222872    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m8qr\" (UniqueName: \"kubernetes.io/projected/47557c2e-244f-407b-a552-9a3d6429b54f-kube-api-access-5m8qr\") pod \"test-curl-7511-140675580389392\" (UID: \"47557c2e-244f-407b-a552-9a3d6429b54f\") " pod="opendatahub/test-curl-7511-140675580389392"
Jul 22 03:06:24.300918 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:24.300882    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7511-140675580389392"
Jul 22 03:06:24.445093 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:24.445069    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7511-140675580389392"]
Jul 22 03:06:24.447113 ip-10-0-138-202 kubenswrapper[2581]: W0722 03:06:24.447083    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod47557c2e_244f_407b_a552_9a3d6429b54f.slice/crio-84a38ae4fffafa7a5ababea344bce15b4fce46dab210fd1dc6a669f4b290e55c WatchSource:0}: Error finding container 84a38ae4fffafa7a5ababea344bce15b4fce46dab210fd1dc6a669f4b290e55c: Status 404 returned error can't find the container with id 84a38ae4fffafa7a5ababea344bce15b4fce46dab210fd1dc6a669f4b290e55c
Jul 22 03:06:24.649202 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:24.649175    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7aa4b5657149a3d0e2bfd92cff1c7618927f8736eb600e0db1111bbaca36124d"
Jul 22 03:06:24.649202 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:24.649175    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7511-140675580388384"
Jul 22 03:06:24.650314 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:24.650288    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7511-140675580389392" event={"ID":"47557c2e-244f-407b-a552-9a3d6429b54f","Type":"ContainerStarted","Data":"84a38ae4fffafa7a5ababea344bce15b4fce46dab210fd1dc6a669f4b290e55c"}
Jul 22 03:06:24.660592 ip-10-0-138-202 kubenswrapper[2581]: E0722 03:06:24.660566    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7511-140675580388384\" is forbidden: User \"system:node:ip-10-0-138-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-202.ec2.internal' and this object" podUID="a92235c3-3fcc-4885-ba37-506719fa5234" pod="opendatahub/test-curl-7511-140675580388384"
Jul 22 03:06:24.728501 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:24.728473    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-995f8f/rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc" podUID="d3378a89-1677-437b-98d1-35296313b222" containerName="main" containerID="cri-o://d94d43e61d4a8ffc8c59571f1d27f2ea6c02b651077492744340f6a34d8603e6" gracePeriod=45
Jul 22 03:06:24.904957 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:24.904928    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a92235c3-3fcc-4885-ba37-506719fa5234" path="/var/lib/kubelet/pods/a92235c3-3fcc-4885-ba37-506719fa5234/volumes"
Jul 22 03:06:24.997049 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:24.997025    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-995f8f/rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc"
Jul 22 03:06:25.123625 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.123590    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3378a89-1677-437b-98d1-35296313b222-tmp-dir\") pod \"d3378a89-1677-437b-98d1-35296313b222\" (UID: \"d3378a89-1677-437b-98d1-35296313b222\") "
Jul 22 03:06:25.123824 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.123674    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3378a89-1677-437b-98d1-35296313b222-dshm\") pod \"d3378a89-1677-437b-98d1-35296313b222\" (UID: \"d3378a89-1677-437b-98d1-35296313b222\") "
Jul 22 03:06:25.123824 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.123709    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3378a89-1677-437b-98d1-35296313b222-home\") pod \"d3378a89-1677-437b-98d1-35296313b222\" (UID: \"d3378a89-1677-437b-98d1-35296313b222\") "
Jul 22 03:06:25.123824 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.123765    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3378a89-1677-437b-98d1-35296313b222-tls-certs\") pod \"d3378a89-1677-437b-98d1-35296313b222\" (UID: \"d3378a89-1677-437b-98d1-35296313b222\") "
Jul 22 03:06:25.123824 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.123801    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3378a89-1677-437b-98d1-35296313b222-model-cache\") pod \"d3378a89-1677-437b-98d1-35296313b222\" (UID: \"d3378a89-1677-437b-98d1-35296313b222\") "
Jul 22 03:06:25.124099 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.123885    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3378a89-1677-437b-98d1-35296313b222-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d3378a89-1677-437b-98d1-35296313b222" (UID: "d3378a89-1677-437b-98d1-35296313b222"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 03:06:25.124099 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.123938    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qptcf\" (UniqueName: \"kubernetes.io/projected/d3378a89-1677-437b-98d1-35296313b222-kube-api-access-qptcf\") pod \"d3378a89-1677-437b-98d1-35296313b222\" (UID: \"d3378a89-1677-437b-98d1-35296313b222\") "
Jul 22 03:06:25.124099 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.124043    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3378a89-1677-437b-98d1-35296313b222-home" (OuterVolumeSpecName: "home") pod "d3378a89-1677-437b-98d1-35296313b222" (UID: "d3378a89-1677-437b-98d1-35296313b222"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 03:06:25.124099 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.124059    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3378a89-1677-437b-98d1-35296313b222-model-cache" (OuterVolumeSpecName: "model-cache") pod "d3378a89-1677-437b-98d1-35296313b222" (UID: "d3378a89-1677-437b-98d1-35296313b222"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 03:06:25.124324 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.124306    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3378a89-1677-437b-98d1-35296313b222-tmp-dir\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:06:25.124385 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.124329    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3378a89-1677-437b-98d1-35296313b222-home\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:06:25.124385 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.124342    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3378a89-1677-437b-98d1-35296313b222-model-cache\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:06:25.125981 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.125952    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3378a89-1677-437b-98d1-35296313b222-dshm" (OuterVolumeSpecName: "dshm") pod "d3378a89-1677-437b-98d1-35296313b222" (UID: "d3378a89-1677-437b-98d1-35296313b222"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 03:06:25.125981 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.125952    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3378a89-1677-437b-98d1-35296313b222-kube-api-access-qptcf" (OuterVolumeSpecName: "kube-api-access-qptcf") pod "d3378a89-1677-437b-98d1-35296313b222" (UID: "d3378a89-1677-437b-98d1-35296313b222"). InnerVolumeSpecName "kube-api-access-qptcf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 03:06:25.126127 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.126007    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3378a89-1677-437b-98d1-35296313b222-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d3378a89-1677-437b-98d1-35296313b222" (UID: "d3378a89-1677-437b-98d1-35296313b222"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 03:06:25.225786 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.225752    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qptcf\" (UniqueName: \"kubernetes.io/projected/d3378a89-1677-437b-98d1-35296313b222-kube-api-access-qptcf\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:06:25.225786 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.225777    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3378a89-1677-437b-98d1-35296313b222-dshm\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:06:25.225786 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.225788    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3378a89-1677-437b-98d1-35296313b222-tls-certs\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:06:25.547650 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.547580    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-bb8c9d/rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg" podUID="775f9c2e-836b-4613-b3ba-fce22b14f176" containerName="main" containerID="cri-o://423d1682339f869293999bcbba851526b06dcf4bcd9322a2be13b6f91bd5d62c" gracePeriod=45
Jul 22 03:06:25.655965 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.655911    2581 generic.go:356] "Generic (PLEG): container finished" podID="d3378a89-1677-437b-98d1-35296313b222" containerID="d94d43e61d4a8ffc8c59571f1d27f2ea6c02b651077492744340f6a34d8603e6" exitCode=0
Jul 22 03:06:25.656127 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.655973    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-995f8f/rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc"
Jul 22 03:06:25.656127 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.655995    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-995f8f/rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc" event={"ID":"d3378a89-1677-437b-98d1-35296313b222","Type":"ContainerDied","Data":"d94d43e61d4a8ffc8c59571f1d27f2ea6c02b651077492744340f6a34d8603e6"}
Jul 22 03:06:25.656127 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.656046    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-995f8f/rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc" event={"ID":"d3378a89-1677-437b-98d1-35296313b222","Type":"ContainerDied","Data":"854ce614f36a527af70cf1f4cdc51363e0f0c67c3808052e045023dd0cebb2a1"}
Jul 22 03:06:25.656127 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.656072    2581 scope.go:117] "RemoveContainer" containerID="d94d43e61d4a8ffc8c59571f1d27f2ea6c02b651077492744340f6a34d8603e6"
Jul 22 03:06:25.657855 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.657779    2581 generic.go:356] "Generic (PLEG): container finished" podID="47557c2e-244f-407b-a552-9a3d6429b54f" containerID="7a3f3782f57a3698767d35bbeea5596bdcc4dc053870ee6e5da6c0c385c08b6f" exitCode=0
Jul 22 03:06:25.657855 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.657820    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7511-140675580389392" event={"ID":"47557c2e-244f-407b-a552-9a3d6429b54f","Type":"ContainerDied","Data":"7a3f3782f57a3698767d35bbeea5596bdcc4dc053870ee6e5da6c0c385c08b6f"}
Jul 22 03:06:25.665846 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.665822    2581 scope.go:117] "RemoveContainer" containerID="d94d43e61d4a8ffc8c59571f1d27f2ea6c02b651077492744340f6a34d8603e6"
Jul 22 03:06:25.666149 ip-10-0-138-202 kubenswrapper[2581]: E0722 03:06:25.666122    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d94d43e61d4a8ffc8c59571f1d27f2ea6c02b651077492744340f6a34d8603e6\": container with ID starting with d94d43e61d4a8ffc8c59571f1d27f2ea6c02b651077492744340f6a34d8603e6 not found: ID does not exist" containerID="d94d43e61d4a8ffc8c59571f1d27f2ea6c02b651077492744340f6a34d8603e6"
Jul 22 03:06:25.666230 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.666155    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d94d43e61d4a8ffc8c59571f1d27f2ea6c02b651077492744340f6a34d8603e6"} err="failed to get container status \"d94d43e61d4a8ffc8c59571f1d27f2ea6c02b651077492744340f6a34d8603e6\": rpc error: code = NotFound desc = could not find container \"d94d43e61d4a8ffc8c59571f1d27f2ea6c02b651077492744340f6a34d8603e6\": container with ID starting with d94d43e61d4a8ffc8c59571f1d27f2ea6c02b651077492744340f6a34d8603e6 not found: ID does not exist"
Jul 22 03:06:25.693847 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.693820    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-995f8f/rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc"]
Jul 22 03:06:25.699220 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.699193    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-995f8f/rate-test-model-995f8f-kserve-5f7cff6fdc-75mfc"]
Jul 22 03:06:25.806525 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.806473    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bb8c9d/rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg"
Jul 22 03:06:25.932352 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.932314    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/775f9c2e-836b-4613-b3ba-fce22b14f176-model-cache\") pod \"775f9c2e-836b-4613-b3ba-fce22b14f176\" (UID: \"775f9c2e-836b-4613-b3ba-fce22b14f176\") "
Jul 22 03:06:25.932844 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.932366    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/775f9c2e-836b-4613-b3ba-fce22b14f176-dshm\") pod \"775f9c2e-836b-4613-b3ba-fce22b14f176\" (UID: \"775f9c2e-836b-4613-b3ba-fce22b14f176\") "
Jul 22 03:06:25.932844 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.932419    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/775f9c2e-836b-4613-b3ba-fce22b14f176-tmp-dir\") pod \"775f9c2e-836b-4613-b3ba-fce22b14f176\" (UID: \"775f9c2e-836b-4613-b3ba-fce22b14f176\") "
Jul 22 03:06:25.932844 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.932470    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/775f9c2e-836b-4613-b3ba-fce22b14f176-tls-certs\") pod \"775f9c2e-836b-4613-b3ba-fce22b14f176\" (UID: \"775f9c2e-836b-4613-b3ba-fce22b14f176\") "
Jul 22 03:06:25.932844 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.932504    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/775f9c2e-836b-4613-b3ba-fce22b14f176-home\") pod \"775f9c2e-836b-4613-b3ba-fce22b14f176\" (UID: \"775f9c2e-836b-4613-b3ba-fce22b14f176\") "
Jul 22 03:06:25.932844 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.932577    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gzk7h\" (UniqueName: \"kubernetes.io/projected/775f9c2e-836b-4613-b3ba-fce22b14f176-kube-api-access-gzk7h\") pod \"775f9c2e-836b-4613-b3ba-fce22b14f176\" (UID: \"775f9c2e-836b-4613-b3ba-fce22b14f176\") "
Jul 22 03:06:25.932844 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.932639    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/775f9c2e-836b-4613-b3ba-fce22b14f176-model-cache" (OuterVolumeSpecName: "model-cache") pod "775f9c2e-836b-4613-b3ba-fce22b14f176" (UID: "775f9c2e-836b-4613-b3ba-fce22b14f176"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 03:06:25.932844 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.932730    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/775f9c2e-836b-4613-b3ba-fce22b14f176-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "775f9c2e-836b-4613-b3ba-fce22b14f176" (UID: "775f9c2e-836b-4613-b3ba-fce22b14f176"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 03:06:25.932844 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.932790    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/775f9c2e-836b-4613-b3ba-fce22b14f176-home" (OuterVolumeSpecName: "home") pod "775f9c2e-836b-4613-b3ba-fce22b14f176" (UID: "775f9c2e-836b-4613-b3ba-fce22b14f176"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 03:06:25.933186 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.932925    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/775f9c2e-836b-4613-b3ba-fce22b14f176-home\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:06:25.933186 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.932942    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/775f9c2e-836b-4613-b3ba-fce22b14f176-model-cache\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:06:25.933186 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.932953    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/775f9c2e-836b-4613-b3ba-fce22b14f176-tmp-dir\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:06:25.934549 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.934524    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/775f9c2e-836b-4613-b3ba-fce22b14f176-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "775f9c2e-836b-4613-b3ba-fce22b14f176" (UID: "775f9c2e-836b-4613-b3ba-fce22b14f176"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 03:06:25.934634 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.934534    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/775f9c2e-836b-4613-b3ba-fce22b14f176-dshm" (OuterVolumeSpecName: "dshm") pod "775f9c2e-836b-4613-b3ba-fce22b14f176" (UID: "775f9c2e-836b-4613-b3ba-fce22b14f176"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 03:06:25.934634 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:25.934586    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/775f9c2e-836b-4613-b3ba-fce22b14f176-kube-api-access-gzk7h" (OuterVolumeSpecName: "kube-api-access-gzk7h") pod "775f9c2e-836b-4613-b3ba-fce22b14f176" (UID: "775f9c2e-836b-4613-b3ba-fce22b14f176"). InnerVolumeSpecName "kube-api-access-gzk7h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 03:06:26.034318 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:26.034280    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/775f9c2e-836b-4613-b3ba-fce22b14f176-tls-certs\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:06:26.034318 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:26.034315    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzk7h\" (UniqueName: \"kubernetes.io/projected/775f9c2e-836b-4613-b3ba-fce22b14f176-kube-api-access-gzk7h\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:06:26.034318 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:26.034324    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/775f9c2e-836b-4613-b3ba-fce22b14f176-dshm\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:06:26.671082 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:26.671051    2581 generic.go:356] "Generic (PLEG): container finished" podID="775f9c2e-836b-4613-b3ba-fce22b14f176" containerID="423d1682339f869293999bcbba851526b06dcf4bcd9322a2be13b6f91bd5d62c" exitCode=0
Jul 22 03:06:26.671248 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:26.671105    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bb8c9d/rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg"
Jul 22 03:06:26.671248 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:26.671130    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bb8c9d/rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg" event={"ID":"775f9c2e-836b-4613-b3ba-fce22b14f176","Type":"ContainerDied","Data":"423d1682339f869293999bcbba851526b06dcf4bcd9322a2be13b6f91bd5d62c"}
Jul 22 03:06:26.671248 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:26.671168    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bb8c9d/rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg" event={"ID":"775f9c2e-836b-4613-b3ba-fce22b14f176","Type":"ContainerDied","Data":"2b07ea1663d0fa32eb9c7446ebfae868e9dc60500d5649b32812b6bdf249a39d"}
Jul 22 03:06:26.671248 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:26.671184    2581 scope.go:117] "RemoveContainer" containerID="423d1682339f869293999bcbba851526b06dcf4bcd9322a2be13b6f91bd5d62c"
Jul 22 03:06:26.680635 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:26.680597    2581 scope.go:117] "RemoveContainer" containerID="423d1682339f869293999bcbba851526b06dcf4bcd9322a2be13b6f91bd5d62c"
Jul 22 03:06:26.680933 ip-10-0-138-202 kubenswrapper[2581]: E0722 03:06:26.680911    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"423d1682339f869293999bcbba851526b06dcf4bcd9322a2be13b6f91bd5d62c\": container with ID starting with 423d1682339f869293999bcbba851526b06dcf4bcd9322a2be13b6f91bd5d62c not found: ID does not exist" containerID="423d1682339f869293999bcbba851526b06dcf4bcd9322a2be13b6f91bd5d62c"
Jul 22 03:06:26.681000 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:26.680940    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"423d1682339f869293999bcbba851526b06dcf4bcd9322a2be13b6f91bd5d62c"} err="failed to get container status \"423d1682339f869293999bcbba851526b06dcf4bcd9322a2be13b6f91bd5d62c\": rpc error: code = NotFound desc = could not find container \"423d1682339f869293999bcbba851526b06dcf4bcd9322a2be13b6f91bd5d62c\": container with ID starting with 423d1682339f869293999bcbba851526b06dcf4bcd9322a2be13b6f91bd5d62c not found: ID does not exist"
Jul 22 03:06:26.697122 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:26.697073    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-bb8c9d/rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg"]
Jul 22 03:06:26.702530 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:26.702500    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-bb8c9d/rate-test-model-bb8c9d-kserve-55c4cc955f-4tbxg"]
Jul 22 03:06:26.813337 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:26.813311    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7511-140675580389392"
Jul 22 03:06:26.840778 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:26.840731    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7511-140675580389392_47557c2e-244f-407b-a552-9a3d6429b54f/test-curl-7511-140675580389392/0.log"
Jul 22 03:06:26.869417 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:26.869386    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7511-140675580389392"]
Jul 22 03:06:26.873726 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:26.873704    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7511-140675580389392"]
Jul 22 03:06:26.904240 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:26.904215    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="775f9c2e-836b-4613-b3ba-fce22b14f176" path="/var/lib/kubelet/pods/775f9c2e-836b-4613-b3ba-fce22b14f176/volumes"
Jul 22 03:06:26.904607 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:26.904593    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3378a89-1677-437b-98d1-35296313b222" path="/var/lib/kubelet/pods/d3378a89-1677-437b-98d1-35296313b222/volumes"
Jul 22 03:06:26.941678 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:26.941608    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5m8qr\" (UniqueName: \"kubernetes.io/projected/47557c2e-244f-407b-a552-9a3d6429b54f-kube-api-access-5m8qr\") pod \"47557c2e-244f-407b-a552-9a3d6429b54f\" (UID: \"47557c2e-244f-407b-a552-9a3d6429b54f\") "
Jul 22 03:06:26.943566 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:26.943544    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47557c2e-244f-407b-a552-9a3d6429b54f-kube-api-access-5m8qr" (OuterVolumeSpecName: "kube-api-access-5m8qr") pod "47557c2e-244f-407b-a552-9a3d6429b54f" (UID: "47557c2e-244f-407b-a552-9a3d6429b54f"). InnerVolumeSpecName "kube-api-access-5m8qr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 03:06:27.002686 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:27.002650    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7511-140675580389392"]
Jul 22 03:06:27.003060 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:27.003047    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3378a89-1677-437b-98d1-35296313b222" containerName="main"
Jul 22 03:06:27.003101 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:27.003062    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3378a89-1677-437b-98d1-35296313b222" containerName="main"
Jul 22 03:06:27.003101 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:27.003076    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47557c2e-244f-407b-a552-9a3d6429b54f" containerName="test-curl-7511-140675580389392"
Jul 22 03:06:27.003101 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:27.003087    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="47557c2e-244f-407b-a552-9a3d6429b54f" containerName="test-curl-7511-140675580389392"
Jul 22 03:06:27.003191 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:27.003110    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="775f9c2e-836b-4613-b3ba-fce22b14f176" containerName="main"
Jul 22 03:06:27.003191 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:27.003117    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="775f9c2e-836b-4613-b3ba-fce22b14f176" containerName="main"
Jul 22 03:06:27.003191 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:27.003177    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="47557c2e-244f-407b-a552-9a3d6429b54f" containerName="test-curl-7511-140675580389392"
Jul 22 03:06:27.003191 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:27.003187    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3378a89-1677-437b-98d1-35296313b222" containerName="main"
Jul 22 03:06:27.003310 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:27.003195    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="775f9c2e-836b-4613-b3ba-fce22b14f176" containerName="main"
Jul 22 03:06:27.006554 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:27.006538    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7511-140675580389392"
Jul 22 03:06:27.017517 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:27.017493    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7511-140675580389392"]
Jul 22 03:06:27.043220 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:27.043188    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5m8qr\" (UniqueName: \"kubernetes.io/projected/47557c2e-244f-407b-a552-9a3d6429b54f-kube-api-access-5m8qr\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:06:27.144092 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:27.144063    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nf7zj\" (UniqueName: \"kubernetes.io/projected/8dbd5d14-f330-4a11-a961-59ad2520562a-kube-api-access-nf7zj\") pod \"test-curl-7511-140675580389392\" (UID: \"8dbd5d14-f330-4a11-a961-59ad2520562a\") " pod="opendatahub/test-curl-7511-140675580389392"
Jul 22 03:06:27.245019 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:27.244987    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nf7zj\" (UniqueName: \"kubernetes.io/projected/8dbd5d14-f330-4a11-a961-59ad2520562a-kube-api-access-nf7zj\") pod \"test-curl-7511-140675580389392\" (UID: \"8dbd5d14-f330-4a11-a961-59ad2520562a\") " pod="opendatahub/test-curl-7511-140675580389392"
Jul 22 03:06:27.258233 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:27.258201    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nf7zj\" (UniqueName: \"kubernetes.io/projected/8dbd5d14-f330-4a11-a961-59ad2520562a-kube-api-access-nf7zj\") pod \"test-curl-7511-140675580389392\" (UID: \"8dbd5d14-f330-4a11-a961-59ad2520562a\") " pod="opendatahub/test-curl-7511-140675580389392"
Jul 22 03:06:27.315391 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:27.315352    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7511-140675580389392"
Jul 22 03:06:27.445930 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:27.445906    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7511-140675580389392"]
Jul 22 03:06:27.448047 ip-10-0-138-202 kubenswrapper[2581]: W0722 03:06:27.448020    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8dbd5d14_f330_4a11_a961_59ad2520562a.slice/crio-886e2b128c95b4efbf2dd9e4f76b659785b6606cb8a0055787c2172252a9bb03 WatchSource:0}: Error finding container 886e2b128c95b4efbf2dd9e4f76b659785b6606cb8a0055787c2172252a9bb03: Status 404 returned error can't find the container with id 886e2b128c95b4efbf2dd9e4f76b659785b6606cb8a0055787c2172252a9bb03
Jul 22 03:06:27.677847 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:27.677768    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7511-140675580389392"
Jul 22 03:06:27.678021 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:27.677774    2581 scope.go:117] "RemoveContainer" containerID="7a3f3782f57a3698767d35bbeea5596bdcc4dc053870ee6e5da6c0c385c08b6f"
Jul 22 03:06:27.678971 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:27.678941    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7511-140675580389392" event={"ID":"8dbd5d14-f330-4a11-a961-59ad2520562a","Type":"ContainerStarted","Data":"886e2b128c95b4efbf2dd9e4f76b659785b6606cb8a0055787c2172252a9bb03"}
Jul 22 03:06:28.687294 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:28.687257    2581 generic.go:356] "Generic (PLEG): container finished" podID="8dbd5d14-f330-4a11-a961-59ad2520562a" containerID="d3beccbfbc24d65cd4655554e836237a042fe2863436ebb501e6be4237b16613" exitCode=0
Jul 22 03:06:28.687294 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:28.687300    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7511-140675580389392" event={"ID":"8dbd5d14-f330-4a11-a961-59ad2520562a","Type":"ContainerDied","Data":"d3beccbfbc24d65cd4655554e836237a042fe2863436ebb501e6be4237b16613"}
Jul 22 03:06:28.904258 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:28.904228    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47557c2e-244f-407b-a552-9a3d6429b54f" path="/var/lib/kubelet/pods/47557c2e-244f-407b-a552-9a3d6429b54f/volumes"
Jul 22 03:06:29.825105 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:29.825068    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7511-140675580389392"
Jul 22 03:06:29.850375 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:29.850352    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7511-140675580389392_8dbd5d14-f330-4a11-a961-59ad2520562a/test-curl-7511-140675580389392/0.log"
Jul 22 03:06:29.866679 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:29.866655    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nf7zj\" (UniqueName: \"kubernetes.io/projected/8dbd5d14-f330-4a11-a961-59ad2520562a-kube-api-access-nf7zj\") pod \"8dbd5d14-f330-4a11-a961-59ad2520562a\" (UID: \"8dbd5d14-f330-4a11-a961-59ad2520562a\") "
Jul 22 03:06:29.868647 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:29.868620    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dbd5d14-f330-4a11-a961-59ad2520562a-kube-api-access-nf7zj" (OuterVolumeSpecName: "kube-api-access-nf7zj") pod "8dbd5d14-f330-4a11-a961-59ad2520562a" (UID: "8dbd5d14-f330-4a11-a961-59ad2520562a"). InnerVolumeSpecName "kube-api-access-nf7zj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 03:06:29.875677 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:29.875657    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7511-140675580389392"]
Jul 22 03:06:29.881605 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:29.881585    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7511-140675580389392"]
Jul 22 03:06:29.967987 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:29.967885    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nf7zj\" (UniqueName: \"kubernetes.io/projected/8dbd5d14-f330-4a11-a961-59ad2520562a-kube-api-access-nf7zj\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:06:30.093326 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:30.093290    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-995f8f-tr6pw"]
Jul 22 03:06:30.093729 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:30.093714    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dbd5d14-f330-4a11-a961-59ad2520562a" containerName="test-curl-7511-140675580389392"
Jul 22 03:06:30.093794 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:30.093730    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dbd5d14-f330-4a11-a961-59ad2520562a" containerName="test-curl-7511-140675580389392"
Jul 22 03:06:30.093833 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:30.093823    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8dbd5d14-f330-4a11-a961-59ad2520562a" containerName="test-curl-7511-140675580389392"
Jul 22 03:06:30.096809 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:30.096792    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-995f8f-tr6pw"
Jul 22 03:06:30.099685 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:30.099662    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rd6jc\""
Jul 22 03:06:30.099799 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:30.099707    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 03:06:30.112653 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:30.112628    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-995f8f-tr6pw"]
Jul 22 03:06:30.170171 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:30.170137    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f51f21f-a386-4b83-a1db-99537b858f6f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-995f8f-tr6pw\" (UID: \"0f51f21f-a386-4b83-a1db-99537b858f6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-995f8f-tr6pw"
Jul 22 03:06:30.271027 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:30.271000    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f51f21f-a386-4b83-a1db-99537b858f6f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-995f8f-tr6pw\" (UID: \"0f51f21f-a386-4b83-a1db-99537b858f6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-995f8f-tr6pw"
Jul 22 03:06:30.271642 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:30.271618    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f51f21f-a386-4b83-a1db-99537b858f6f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-995f8f-tr6pw\" (UID: \"0f51f21f-a386-4b83-a1db-99537b858f6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-995f8f-tr6pw"
Jul 22 03:06:30.406733 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:30.406702    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-995f8f-tr6pw"
Jul 22 03:06:30.564944 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:30.564918    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-995f8f-tr6pw"]
Jul 22 03:06:30.567470 ip-10-0-138-202 kubenswrapper[2581]: W0722 03:06:30.567444    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f51f21f_a386_4b83_a1db_99537b858f6f.slice/crio-318ccafd2bc50b6a7e1fadc5f3b31871c4236e084e339caff73bb262ca96b0e4 WatchSource:0}: Error finding container 318ccafd2bc50b6a7e1fadc5f3b31871c4236e084e339caff73bb262ca96b0e4: Status 404 returned error can't find the container with id 318ccafd2bc50b6a7e1fadc5f3b31871c4236e084e339caff73bb262ca96b0e4
Jul 22 03:06:30.697976 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:30.697942    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="886e2b128c95b4efbf2dd9e4f76b659785b6606cb8a0055787c2172252a9bb03"
Jul 22 03:06:30.697976 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:30.697953    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7511-140675580389392"
Jul 22 03:06:30.699168 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:30.699140    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-995f8f-tr6pw" event={"ID":"0f51f21f-a386-4b83-a1db-99537b858f6f","Type":"ContainerStarted","Data":"318ccafd2bc50b6a7e1fadc5f3b31871c4236e084e339caff73bb262ca96b0e4"}
Jul 22 03:06:30.709118 ip-10-0-138-202 kubenswrapper[2581]: E0722 03:06:30.709077    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7511-140675580389392\" is forbidden: User \"system:node:ip-10-0-138-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-202.ec2.internal' and this object" podUID="8dbd5d14-f330-4a11-a961-59ad2520562a" pod="opendatahub/test-curl-7511-140675580389392"
Jul 22 03:06:30.742604 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:30.742560    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:06:30.839194 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:30.839163    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:06:30.904411 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:30.904378    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8dbd5d14-f330-4a11-a961-59ad2520562a" path="/var/lib/kubelet/pods/8dbd5d14-f330-4a11-a961-59ad2520562a/volumes"
Jul 22 03:06:31.706229 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:31.706144    2581 generic.go:356] "Generic (PLEG): container finished" podID="0f51f21f-a386-4b83-a1db-99537b858f6f" containerID="52eb701c6f4698a91cb794af1ecced72fa53ed41c60fd0c54f85eb7f683d13e3" exitCode=0
Jul 22 03:06:31.706362 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:31.706223    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-995f8f-tr6pw" event={"ID":"0f51f21f-a386-4b83-a1db-99537b858f6f","Type":"ContainerDied","Data":"52eb701c6f4698a91cb794af1ecced72fa53ed41c60fd0c54f85eb7f683d13e3"}
Jul 22 03:06:32.845326 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:32.845303    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-995f8f-tr6pw"
Jul 22 03:06:32.889723 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:32.889696    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f51f21f-a386-4b83-a1db-99537b858f6f-maas-api-service-ca\") pod \"0f51f21f-a386-4b83-a1db-99537b858f6f\" (UID: \"0f51f21f-a386-4b83-a1db-99537b858f6f\") "
Jul 22 03:06:32.890027 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:32.890007    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f51f21f-a386-4b83-a1db-99537b858f6f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0f51f21f-a386-4b83-a1db-99537b858f6f" (UID: "0f51f21f-a386-4b83-a1db-99537b858f6f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 03:06:32.990643 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:32.990609    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f51f21f-a386-4b83-a1db-99537b858f6f-maas-api-service-ca\") on node \"ip-10-0-138-202.ec2.internal\" DevicePath \"\""
Jul 22 03:06:33.717033 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:33.716996    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-995f8f-tr6pw" event={"ID":"0f51f21f-a386-4b83-a1db-99537b858f6f","Type":"ContainerDied","Data":"318ccafd2bc50b6a7e1fadc5f3b31871c4236e084e339caff73bb262ca96b0e4"}
Jul 22 03:06:33.717033 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:33.717027    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-995f8f-tr6pw"
Jul 22 03:06:33.717243 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:33.717048    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="318ccafd2bc50b6a7e1fadc5f3b31871c4236e084e339caff73bb262ca96b0e4"
Jul 22 03:06:40.665887 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:40.665855    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:06:43.673203 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:43.673156    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 03:06:45.299538 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:45.299502    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:06:45.301833 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:45.301809    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-995f8f-tr6pw"]
Jul 22 03:06:45.318887 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:45.318858    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-995f8f-tr6pw"]
Jul 22 03:06:45.984307 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:45.984270    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:06:46.904116 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:46.904086    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f51f21f-a386-4b83-a1db-99537b858f6f" path="/var/lib/kubelet/pods/0f51f21f-a386-4b83-a1db-99537b858f6f/volumes"
Jul 22 03:06:51.944611 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:51.944577    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:06:52.045437 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:52.045404    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:06:52.268665 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:52.268636    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:06:52.467466 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:52.467436    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:06:53.149177 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:53.149144    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:06:53.255708 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:06:53.255666    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:07:03.074871 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:03.074837    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:07:06.140881 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:06.140843    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t9zk2"]
Jul 22 03:07:07.649557 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:07.649521    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:07:08.142482 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:08.142449    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:07:14.451908 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:14.451862    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:07:14.552620 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:14.552581    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:07:14.769963 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:14.769929    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:07:14.982372 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:14.982345    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:07:21.449671 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:21.449640    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cb9fd785c-fnrth_fca07606-2755-4cd9-af37-bbd12ba712b7/copy-manifests/0.log"
Jul 22 03:07:21.456572 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:21.456551    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cb9fd785c-fnrth_fca07606-2755-4cd9-af37-bbd12ba712b7/manager/0.log"
Jul 22 03:07:24.097634 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:24.097602    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-wknpt_1e9a69b9-4f1c-4673-9fb3-6bc0b7652766/kuadrant-console-plugin/0.log"
Jul 22 03:07:24.492673 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:24.492638    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-t9zk2_061682e8-bf94-4a04-9ff4-e877181535ad/limitador/0.log"
Jul 22 03:07:25.206009 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:25.205976    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65c4bb8965-d2h6v_6fe58e91-0a3d-4643-b65b-fbd1f3832864/kube-auth-proxy/0.log"
Jul 22 03:07:26.613698 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:26.613668    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-cwmsw_7c67de9f-be55-4efa-8fd5-2f6d319e7cd3/main/0.log"
Jul 22 03:07:26.724181 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:26.724153    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-77w66_c70213dc-ed59-4cb2-bcb7-8a62d938f24d/main/0.log"
Jul 22 03:07:32.396238 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:32.396153    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-c69w5/must-gather-cjksf"]
Jul 22 03:07:32.396686 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:32.396652    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f51f21f-a386-4b83-a1db-99537b858f6f" containerName="revoke-keys"
Jul 22 03:07:32.396686 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:32.396672    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f51f21f-a386-4b83-a1db-99537b858f6f" containerName="revoke-keys"
Jul 22 03:07:32.396796 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:32.396789    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f51f21f-a386-4b83-a1db-99537b858f6f" containerName="revoke-keys"
Jul 22 03:07:32.400045 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:32.400029    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c69w5/must-gather-cjksf"
Jul 22 03:07:32.405342 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:32.405318    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-c69w5\"/\"openshift-service-ca.crt\""
Jul 22 03:07:32.405819 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:32.405801    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-c69w5\"/\"kube-root-ca.crt\""
Jul 22 03:07:32.407046 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:32.407029    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-c69w5\"/\"default-dockercfg-hkft4\""
Jul 22 03:07:32.413420 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:32.413396    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-c69w5/must-gather-cjksf"]
Jul 22 03:07:32.414172 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:32.414150    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:07:32.536899 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:32.536870    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:07:32.543365 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:32.543343    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ae0ab099-916d-4f9e-81b0-c540bfb293af-must-gather-output\") pod \"must-gather-cjksf\" (UID: \"ae0ab099-916d-4f9e-81b0-c540bfb293af\") " pod="openshift-must-gather-c69w5/must-gather-cjksf"
Jul 22 03:07:32.543432 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:32.543377    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m88db\" (UniqueName: \"kubernetes.io/projected/ae0ab099-916d-4f9e-81b0-c540bfb293af-kube-api-access-m88db\") pod \"must-gather-cjksf\" (UID: \"ae0ab099-916d-4f9e-81b0-c540bfb293af\") " pod="openshift-must-gather-c69w5/must-gather-cjksf"
Jul 22 03:07:32.644473 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:32.644434    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ae0ab099-916d-4f9e-81b0-c540bfb293af-must-gather-output\") pod \"must-gather-cjksf\" (UID: \"ae0ab099-916d-4f9e-81b0-c540bfb293af\") " pod="openshift-must-gather-c69w5/must-gather-cjksf"
Jul 22 03:07:32.644663 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:32.644482    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m88db\" (UniqueName: \"kubernetes.io/projected/ae0ab099-916d-4f9e-81b0-c540bfb293af-kube-api-access-m88db\") pod \"must-gather-cjksf\" (UID: \"ae0ab099-916d-4f9e-81b0-c540bfb293af\") " pod="openshift-must-gather-c69w5/must-gather-cjksf"
Jul 22 03:07:32.644821 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:32.644800    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ae0ab099-916d-4f9e-81b0-c540bfb293af-must-gather-output\") pod \"must-gather-cjksf\" (UID: \"ae0ab099-916d-4f9e-81b0-c540bfb293af\") " pod="openshift-must-gather-c69w5/must-gather-cjksf"
Jul 22 03:07:32.656079 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:32.656030    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m88db\" (UniqueName: \"kubernetes.io/projected/ae0ab099-916d-4f9e-81b0-c540bfb293af-kube-api-access-m88db\") pod \"must-gather-cjksf\" (UID: \"ae0ab099-916d-4f9e-81b0-c540bfb293af\") " pod="openshift-must-gather-c69w5/must-gather-cjksf"
Jul 22 03:07:32.709264 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:32.709235    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c69w5/must-gather-cjksf"
Jul 22 03:07:32.852199 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:32.852170    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-c69w5/must-gather-cjksf"]
Jul 22 03:07:32.855533 ip-10-0-138-202 kubenswrapper[2581]: W0722 03:07:32.855506    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae0ab099_916d_4f9e_81b0_c540bfb293af.slice/crio-1c10938d1187402097228f0868a63d5dd0ea112c55ee82537ec1f6b21f9d1d28 WatchSource:0}: Error finding container 1c10938d1187402097228f0868a63d5dd0ea112c55ee82537ec1f6b21f9d1d28: Status 404 returned error can't find the container with id 1c10938d1187402097228f0868a63d5dd0ea112c55ee82537ec1f6b21f9d1d28
Jul 22 03:07:33.010029 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:33.009999    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c69w5/must-gather-cjksf" event={"ID":"ae0ab099-916d-4f9e-81b0-c540bfb293af","Type":"ContainerStarted","Data":"1c10938d1187402097228f0868a63d5dd0ea112c55ee82537ec1f6b21f9d1d28"}
Jul 22 03:07:34.019422 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:34.019370    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c69w5/must-gather-cjksf" event={"ID":"ae0ab099-916d-4f9e-81b0-c540bfb293af","Type":"ContainerStarted","Data":"f86f7b03b85b127954a10d575cda15ca7f1023bb9b09ed0e53498f0df3b6fcb7"}
Jul 22 03:07:34.019422 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:34.019428    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c69w5/must-gather-cjksf" event={"ID":"ae0ab099-916d-4f9e-81b0-c540bfb293af","Type":"ContainerStarted","Data":"7796964c6f611db5ed06fbf089558dff33bd6445d2512df5bdefa354b81bf56c"}
Jul 22 03:07:35.265325 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:35.265296    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-gvv6m_adabd7dc-4255-4a59-ab43-a2b2d5aa5a5a/global-pull-secret-syncer/0.log"
Jul 22 03:07:35.360103 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:35.360074    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log"
Jul 22 03:07:35.472351 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:35.472319    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-202.ec2.internal_03f847ed505e198ceec414e6c1b66d63/haproxy/0.log"
Jul 22 03:07:40.244516 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:40.244481    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-wknpt_1e9a69b9-4f1c-4673-9fb3-6bc0b7652766/kuadrant-console-plugin/0.log"
Jul 22 03:07:40.464715 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:40.464653    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-t9zk2_061682e8-bf94-4a04-9ff4-e877181535ad/limitador/0.log"
Jul 22 03:07:42.206909 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:42.206822    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c5203a3b-a172-418e-8953-233fc2d4ae89/alertmanager/0.log"
Jul 22 03:07:42.229782 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:42.229750    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c5203a3b-a172-418e-8953-233fc2d4ae89/config-reloader/0.log"
Jul 22 03:07:42.251400 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:42.251368    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c5203a3b-a172-418e-8953-233fc2d4ae89/kube-rbac-proxy-web/0.log"
Jul 22 03:07:42.273013 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:42.272978    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c5203a3b-a172-418e-8953-233fc2d4ae89/kube-rbac-proxy/0.log"
Jul 22 03:07:42.294825 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:42.294797    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c5203a3b-a172-418e-8953-233fc2d4ae89/kube-rbac-proxy-metric/0.log"
Jul 22 03:07:42.314760 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:42.314707    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c5203a3b-a172-418e-8953-233fc2d4ae89/prom-label-proxy/0.log"
Jul 22 03:07:42.338827 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:42.338753    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c5203a3b-a172-418e-8953-233fc2d4ae89/init-config-reloader/0.log"
Jul 22 03:07:42.412131 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:42.412093    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-f2pg8_49ea4211-194d-4d3d-992d-20f3462d03b3/kube-state-metrics/0.log"
Jul 22 03:07:42.449648 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:42.449615    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-f2pg8_49ea4211-194d-4d3d-992d-20f3462d03b3/kube-rbac-proxy-main/0.log"
Jul 22 03:07:42.493530 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:42.493471    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-f2pg8_49ea4211-194d-4d3d-992d-20f3462d03b3/kube-rbac-proxy-self/0.log"
Jul 22 03:07:42.517999 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:42.517973    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-78d6957c46-pqvxl_46360a1c-c876-4fd3-ac0c-09956174bebd/metrics-server/0.log"
Jul 22 03:07:42.546693 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:42.546665    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-wvc9x_91bd715b-d6dc-4447-85c2-74a058a4478e/monitoring-plugin/0.log"
Jul 22 03:07:42.668456 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:42.668427    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mp9ts_1b8cfc23-4e2b-46cc-9c6d-82e27573d34c/node-exporter/0.log"
Jul 22 03:07:42.693069 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:42.693027    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mp9ts_1b8cfc23-4e2b-46cc-9c6d-82e27573d34c/kube-rbac-proxy/0.log"
Jul 22 03:07:42.713100 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:42.713074    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mp9ts_1b8cfc23-4e2b-46cc-9c6d-82e27573d34c/init-textfile/0.log"
Jul 22 03:07:42.826364 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:42.826319    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-v78rs_d1049839-4275-4715-bea3-230b58d93041/kube-rbac-proxy-main/0.log"
Jul 22 03:07:42.848817 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:42.848768    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-v78rs_d1049839-4275-4715-bea3-230b58d93041/kube-rbac-proxy-self/0.log"
Jul 22 03:07:42.877893 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:42.877864    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-v78rs_d1049839-4275-4715-bea3-230b58d93041/openshift-state-metrics/0.log"
Jul 22 03:07:42.932350 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:42.932325    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dde833f2-68cb-4230-bb48-3a6605c6457f/prometheus/0.log"
Jul 22 03:07:42.953710 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:42.953685    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dde833f2-68cb-4230-bb48-3a6605c6457f/config-reloader/0.log"
Jul 22 03:07:42.975772 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:42.975708    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dde833f2-68cb-4230-bb48-3a6605c6457f/thanos-sidecar/0.log"
Jul 22 03:07:42.999491 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:42.999433    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dde833f2-68cb-4230-bb48-3a6605c6457f/kube-rbac-proxy-web/0.log"
Jul 22 03:07:43.018635 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:43.018611    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dde833f2-68cb-4230-bb48-3a6605c6457f/kube-rbac-proxy/0.log"
Jul 22 03:07:43.040883 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:43.040792    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dde833f2-68cb-4230-bb48-3a6605c6457f/kube-rbac-proxy-thanos/0.log"
Jul 22 03:07:43.059981 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:43.059946    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dde833f2-68cb-4230-bb48-3a6605c6457f/init-config-reloader/0.log"
Jul 22 03:07:43.089524 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:43.089495    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-nvxw2_039210ef-e3d8-497e-8553-8760df40860a/prometheus-operator/0.log"
Jul 22 03:07:43.108320 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:43.108291    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-nvxw2_039210ef-e3d8-497e-8553-8760df40860a/kube-rbac-proxy/0.log"
Jul 22 03:07:43.134021 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:43.133987    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-5kbp2_9edede50-1dd5-4ba2-87af-4b51e52b9ea5/prometheus-operator-admission-webhook/0.log"
Jul 22 03:07:43.162225 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:43.162187    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5ddb9449c-h5zj7_10f71205-4482-4c08-8fc4-e8a17b554b5b/telemeter-client/0.log"
Jul 22 03:07:43.184027 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:43.183998    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5ddb9449c-h5zj7_10f71205-4482-4c08-8fc4-e8a17b554b5b/reload/0.log"
Jul 22 03:07:43.207634 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:43.207604    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5ddb9449c-h5zj7_10f71205-4482-4c08-8fc4-e8a17b554b5b/kube-rbac-proxy/0.log"
Jul 22 03:07:43.238310 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:43.238281    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-776964c7b7-z4wzv_fea2175d-60d6-4fca-b26f-c935babe00c1/thanos-query/0.log"
Jul 22 03:07:43.259480 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:43.259454    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-776964c7b7-z4wzv_fea2175d-60d6-4fca-b26f-c935babe00c1/kube-rbac-proxy-web/0.log"
Jul 22 03:07:43.282063 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:43.282032    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-776964c7b7-z4wzv_fea2175d-60d6-4fca-b26f-c935babe00c1/kube-rbac-proxy/0.log"
Jul 22 03:07:43.300889 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:43.300798    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-776964c7b7-z4wzv_fea2175d-60d6-4fca-b26f-c935babe00c1/prom-label-proxy/0.log"
Jul 22 03:07:43.321182 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:43.321120    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-776964c7b7-z4wzv_fea2175d-60d6-4fca-b26f-c935babe00c1/kube-rbac-proxy-rules/0.log"
Jul 22 03:07:43.340998 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:43.340962    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-776964c7b7-z4wzv_fea2175d-60d6-4fca-b26f-c935babe00c1/kube-rbac-proxy-metrics/0.log"
Jul 22 03:07:45.764802 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:45.764690    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-c69w5/must-gather-cjksf" podStartSLOduration=12.922432691000001 podStartE2EDuration="13.764665168s" totalImagesPullingTime="842.232477ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:07:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:07:34.040925178 +0000 UTC m=+2875.581718807" watchObservedRunningTime="2026-07-22 03:07:45.764665168 +0000 UTC m=+2887.305458796"
Jul 22 03:07:45.766206 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:45.766175    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-c69w5/perf-node-gather-daemonset-hcwxw"]
Jul 22 03:07:45.775107 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:45.775083    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-hcwxw"
Jul 22 03:07:45.778362 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:45.778249    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-c69w5/perf-node-gather-daemonset-hcwxw"]
Jul 22 03:07:45.896019 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:45.895972    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aa9bf074-a631-4aa5-85d0-1ca2084091da-lib-modules\") pod \"perf-node-gather-daemonset-hcwxw\" (UID: \"aa9bf074-a631-4aa5-85d0-1ca2084091da\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-hcwxw"
Jul 22 03:07:45.896214 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:45.896029    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8ppx\" (UniqueName: \"kubernetes.io/projected/aa9bf074-a631-4aa5-85d0-1ca2084091da-kube-api-access-w8ppx\") pod \"perf-node-gather-daemonset-hcwxw\" (UID: \"aa9bf074-a631-4aa5-85d0-1ca2084091da\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-hcwxw"
Jul 22 03:07:45.896214 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:45.896154    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aa9bf074-a631-4aa5-85d0-1ca2084091da-sys\") pod \"perf-node-gather-daemonset-hcwxw\" (UID: \"aa9bf074-a631-4aa5-85d0-1ca2084091da\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-hcwxw"
Jul 22 03:07:45.896332 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:45.896224    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/aa9bf074-a631-4aa5-85d0-1ca2084091da-podres\") pod \"perf-node-gather-daemonset-hcwxw\" (UID: \"aa9bf074-a631-4aa5-85d0-1ca2084091da\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-hcwxw"
Jul 22 03:07:45.896332 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:45.896258    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/aa9bf074-a631-4aa5-85d0-1ca2084091da-proc\") pod \"perf-node-gather-daemonset-hcwxw\" (UID: \"aa9bf074-a631-4aa5-85d0-1ca2084091da\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-hcwxw"
Jul 22 03:07:46.007509 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:46.007464    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aa9bf074-a631-4aa5-85d0-1ca2084091da-lib-modules\") pod \"perf-node-gather-daemonset-hcwxw\" (UID: \"aa9bf074-a631-4aa5-85d0-1ca2084091da\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-hcwxw"
Jul 22 03:07:46.007797 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:46.007673    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8ppx\" (UniqueName: \"kubernetes.io/projected/aa9bf074-a631-4aa5-85d0-1ca2084091da-kube-api-access-w8ppx\") pod \"perf-node-gather-daemonset-hcwxw\" (UID: \"aa9bf074-a631-4aa5-85d0-1ca2084091da\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-hcwxw"
Jul 22 03:07:46.007908 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:46.007824    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aa9bf074-a631-4aa5-85d0-1ca2084091da-sys\") pod \"perf-node-gather-daemonset-hcwxw\" (UID: \"aa9bf074-a631-4aa5-85d0-1ca2084091da\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-hcwxw"
Jul 22 03:07:46.007908 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:46.007601    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aa9bf074-a631-4aa5-85d0-1ca2084091da-lib-modules\") pod \"perf-node-gather-daemonset-hcwxw\" (UID: \"aa9bf074-a631-4aa5-85d0-1ca2084091da\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-hcwxw"
Jul 22 03:07:46.008020 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:46.007944    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/aa9bf074-a631-4aa5-85d0-1ca2084091da-podres\") pod \"perf-node-gather-daemonset-hcwxw\" (UID: \"aa9bf074-a631-4aa5-85d0-1ca2084091da\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-hcwxw"
Jul 22 03:07:46.008076 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:46.008050    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aa9bf074-a631-4aa5-85d0-1ca2084091da-sys\") pod \"perf-node-gather-daemonset-hcwxw\" (UID: \"aa9bf074-a631-4aa5-85d0-1ca2084091da\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-hcwxw"
Jul 22 03:07:46.008280 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:46.008260    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/aa9bf074-a631-4aa5-85d0-1ca2084091da-proc\") pod \"perf-node-gather-daemonset-hcwxw\" (UID: \"aa9bf074-a631-4aa5-85d0-1ca2084091da\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-hcwxw"
Jul 22 03:07:46.008389 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:46.008370    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/aa9bf074-a631-4aa5-85d0-1ca2084091da-podres\") pod \"perf-node-gather-daemonset-hcwxw\" (UID: \"aa9bf074-a631-4aa5-85d0-1ca2084091da\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-hcwxw"
Jul 22 03:07:46.008469 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:46.008424    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/aa9bf074-a631-4aa5-85d0-1ca2084091da-proc\") pod \"perf-node-gather-daemonset-hcwxw\" (UID: \"aa9bf074-a631-4aa5-85d0-1ca2084091da\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-hcwxw"
Jul 22 03:07:46.019188 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:46.019104    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8ppx\" (UniqueName: \"kubernetes.io/projected/aa9bf074-a631-4aa5-85d0-1ca2084091da-kube-api-access-w8ppx\") pod \"perf-node-gather-daemonset-hcwxw\" (UID: \"aa9bf074-a631-4aa5-85d0-1ca2084091da\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-hcwxw"
Jul 22 03:07:46.088799 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:46.088768    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-hcwxw"
Jul 22 03:07:46.154485 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:46.154370    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-dkd8c_121417f5-c17d-457c-a8f1-6ba982980cec/download-server/0.log"
Jul 22 03:07:46.283824 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:46.283599    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-c69w5/perf-node-gather-daemonset-hcwxw"]
Jul 22 03:07:47.117614 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:47.117572    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-hcwxw" event={"ID":"aa9bf074-a631-4aa5-85d0-1ca2084091da","Type":"ContainerStarted","Data":"f9319260a1be5dcfd34fd2e49dc96a4c738957f80e35638c9ae3cf94f8ebf7ca"}
Jul 22 03:07:47.118040 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:47.117626    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-hcwxw" event={"ID":"aa9bf074-a631-4aa5-85d0-1ca2084091da","Type":"ContainerStarted","Data":"fc17a76c85b4d039ae5f9bfca924900282ee0d74d9c958d742fac6681507bdd5"}
Jul 22 03:07:47.118040 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:47.117670    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-hcwxw"
Jul 22 03:07:47.941604 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:47.941574    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jzzqf_62cd0167-5561-4aca-8c78-a1f7c526e34e/dns/0.log"
Jul 22 03:07:47.963565 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:47.963534    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jzzqf_62cd0167-5561-4aca-8c78-a1f7c526e34e/kube-rbac-proxy/0.log"
Jul 22 03:07:48.057269 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:48.057238    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-r5qtx_5fc01079-d37c-4352-b0fd-5d7e7e09a713/dns-node-resolver/0.log"
Jul 22 03:07:48.664830 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:48.664801    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xfq7z_9facab3f-4b58-4168-b73e-ff7535771473/node-ca/0.log"
Jul 22 03:07:49.726713 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:49.726684    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65c4bb8965-d2h6v_6fe58e91-0a3d-4643-b65b-fbd1f3832864/kube-auth-proxy/0.log"
Jul 22 03:07:50.469616 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:50.469583    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wxzz_7fd1ec82-dd3b-4712-87dc-b63028464820/konnectivity-agent/0.log" limit=1024
Jul 22 03:07:50.594415 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:50.594389    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-w5t8r_0311047b-64b1-4a99-94f6-8cd2571c2d4c/serve-healthcheck-canary/0.log"
Jul 22 03:07:51.054573 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:51.054542    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2mvpx_efc56fd3-50cc-4228-8ecf-44eed0f5864e/kube-rbac-proxy/0.log"
Jul 22 03:07:51.075688 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:51.075664    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2mvpx_efc56fd3-50cc-4228-8ecf-44eed0f5864e/exporter/0.log"
Jul 22 03:07:51.097335 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:51.097289    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2mvpx_efc56fd3-50cc-4228-8ecf-44eed0f5864e/extractor/0.log"
Jul 22 03:07:53.136764 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:53.136718    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-hcwxw"
Jul 22 03:07:53.156248 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:53.156195    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-hcwxw" podStartSLOduration=8.156178365 podStartE2EDuration="8.156178365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:07:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:07:47.154664505 +0000 UTC m=+2888.695458134" watchObservedRunningTime="2026-07-22 03:07:53.156178365 +0000 UTC m=+2894.696971992"
Jul 22 03:07:53.720237 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:53.720197    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cb9fd785c-fnrth_fca07606-2755-4cd9-af37-bbd12ba712b7/manager/0.log"
Jul 22 03:07:53.739521 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:53.739492    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cb9fd785c-fnrth_fca07606-2755-4cd9-af37-bbd12ba712b7/copy-manifests/0.log"
Jul 22 03:07:55.264599 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:55.264566    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-645b87676-nt9sw_ebe3ece5-117c-4ae9-adf8-bb7f16e7f328/manager/0.log"
Jul 22 03:07:56.279889 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:56.279854    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65c4bb8965-d2h6v_6fe58e91-0a3d-4643-b65b-fbd1f3832864/kube-auth-proxy/0.log"
Jul 22 03:07:57.745502 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:07:57.745473    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65c4bb8965-d2h6v_6fe58e91-0a3d-4643-b65b-fbd1f3832864/kube-auth-proxy/0.log"
Jul 22 03:08:01.768429 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:08:01.768399    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-24vpj_f332116e-038b-48e4-b0d6-ee8cb1847e7c/kube-multus/0.log"
Jul 22 03:08:01.879140 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:08:01.879105    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5wkbl_7a87ca25-67e4-47cd-a391-ff3f1035448f/kube-multus-additional-cni-plugins/0.log"
Jul 22 03:08:01.911467 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:08:01.911437    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5wkbl_7a87ca25-67e4-47cd-a391-ff3f1035448f/egress-router-binary-copy/0.log"
Jul 22 03:08:01.957935 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:08:01.957904    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5wkbl_7a87ca25-67e4-47cd-a391-ff3f1035448f/cni-plugins/0.log"
Jul 22 03:08:01.988652 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:08:01.988617    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5wkbl_7a87ca25-67e4-47cd-a391-ff3f1035448f/bond-cni-plugin/0.log"
Jul 22 03:08:02.014189 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:08:02.014162    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5wkbl_7a87ca25-67e4-47cd-a391-ff3f1035448f/routeoverride-cni/0.log"
Jul 22 03:08:02.035970 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:08:02.035893    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5wkbl_7a87ca25-67e4-47cd-a391-ff3f1035448f/whereabouts-cni-bincopy/0.log"
Jul 22 03:08:02.057138 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:08:02.057108    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5wkbl_7a87ca25-67e4-47cd-a391-ff3f1035448f/whereabouts-cni/0.log"
Jul 22 03:08:02.539709 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:08:02.539676    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-6bsw8_fa01c319-94d5-4113-9457-6b808c24dee5/network-metrics-daemon/0.log"
Jul 22 03:08:02.572297 ip-10-0-138-202 kubenswrapper[2581]: I0722 03:08:02.572274    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-6bsw8_fa01c319-94d5-4113-9457-6b808c24dee5/kube-rbac-proxy/0.log"