Jul 24 17:10:31.472768 ip-10-0-139-46 systemd[1]: Starting Kubernetes Kubelet...
Jul 24 17:10:31.935079 ip-10-0-139-46 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 17:10:31.935079 ip-10-0-139-46 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 24 17:10:31.935079 ip-10-0-139-46 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 17:10:31.935079 ip-10-0-139-46 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 24 17:10:31.935079 ip-10-0-139-46 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 17:10:31.937560 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.937468    2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 24 17:10:31.943339 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943322    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 17:10:31.943387 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943341    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 17:10:31.943387 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943346    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 17:10:31.943387 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943362    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 17:10:31.943387 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943365    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 17:10:31.943387 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943368    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 17:10:31.943387 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943371    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 17:10:31.943387 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943374    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 17:10:31.943387 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943376    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 17:10:31.943387 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943379    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 17:10:31.943387 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943382    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 17:10:31.943387 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943384    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 17:10:31.943387 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943387    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 17:10:31.943387 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943390    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 17:10:31.943387 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943393    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 17:10:31.943745 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943396    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 17:10:31.943745 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943399    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 17:10:31.943745 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943402    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 17:10:31.943745 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943405    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 17:10:31.943745 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943407    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 17:10:31.943745 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943410    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 17:10:31.943745 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943412    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 17:10:31.943745 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943414    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 17:10:31.943745 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943417    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 17:10:31.943745 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943420    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 17:10:31.943745 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943422    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 17:10:31.943745 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943425    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 17:10:31.943745 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943427    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 17:10:31.943745 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943429    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 17:10:31.943745 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943432    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 17:10:31.943745 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943434    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 17:10:31.943745 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943436    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 17:10:31.943745 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943439    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 17:10:31.943745 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943441    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 17:10:31.943745 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943444    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 17:10:31.944206 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943459    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 17:10:31.944206 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943461    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 17:10:31.944206 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943464    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 17:10:31.944206 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943466    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 17:10:31.944206 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943469    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 17:10:31.944206 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943472    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 17:10:31.944206 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943475    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 17:10:31.944206 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943477    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 17:10:31.944206 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943480    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 17:10:31.944206 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943482    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 17:10:31.944206 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943485    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 17:10:31.944206 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943487    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 17:10:31.944206 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943489    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 17:10:31.944206 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943496    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 17:10:31.944206 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943500    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 17:10:31.944206 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943502    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 17:10:31.944206 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943506    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 17:10:31.944206 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943508    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 17:10:31.944206 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943511    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 17:10:31.944785 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943514    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 17:10:31.944785 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943517    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 17:10:31.944785 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943520    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 17:10:31.944785 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943522    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 17:10:31.944785 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943524    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 17:10:31.944785 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943527    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 17:10:31.944785 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943529    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 17:10:31.944785 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943531    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 17:10:31.944785 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943535    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 17:10:31.944785 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943538    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 17:10:31.944785 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943541    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 17:10:31.944785 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943544    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 17:10:31.944785 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943547    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 17:10:31.944785 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943549    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 17:10:31.944785 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943552    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 17:10:31.944785 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943555    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 17:10:31.944785 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943558    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 17:10:31.944785 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943561    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 17:10:31.944785 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943564    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 17:10:31.945250 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943566    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 17:10:31.945250 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943569    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 17:10:31.945250 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943571    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 17:10:31.945250 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943574    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 17:10:31.945250 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943576    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 17:10:31.945250 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943579    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 17:10:31.945250 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943581    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 17:10:31.945250 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943585    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 17:10:31.945250 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943587    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 17:10:31.945250 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943590    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 17:10:31.945250 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943592    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 17:10:31.945250 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943595    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 17:10:31.945250 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943597    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 17:10:31.945250 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943599    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 17:10:31.945250 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943602    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 17:10:31.945250 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943605    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 17:10:31.945250 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943607    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 17:10:31.945250 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943610    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 17:10:31.945250 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943612    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 17:10:31.945250 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943614    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 17:10:31.945748 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943617    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 17:10:31.945748 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943619    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 17:10:31.945748 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943622    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 17:10:31.945748 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943624    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 17:10:31.945748 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943627    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 17:10:31.945748 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.943629    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 17:10:31.946263 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946250    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 17:10:31.946263 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946263    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 17:10:31.946318 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946266    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 17:10:31.946318 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946269    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 17:10:31.946318 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946272    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 17:10:31.946318 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946276    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 17:10:31.946318 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946280    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 17:10:31.946318 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946284    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 17:10:31.946318 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946288    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 17:10:31.946318 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946291    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 17:10:31.946318 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946294    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 17:10:31.946318 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946297    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 17:10:31.946318 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946300    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 17:10:31.946318 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946302    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 17:10:31.946318 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946305    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 17:10:31.946318 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946307    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 17:10:31.946318 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946310    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 17:10:31.946318 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946313    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 17:10:31.946318 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946315    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 17:10:31.946318 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946319    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 17:10:31.946318 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946322    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 17:10:31.946802 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946325    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 17:10:31.946802 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946328    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 17:10:31.946802 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946331    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 17:10:31.946802 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946335    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 17:10:31.946802 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946338    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 17:10:31.946802 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946340    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 17:10:31.946802 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946342    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 17:10:31.946802 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946345    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 17:10:31.946802 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946347    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 17:10:31.946802 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946350    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 17:10:31.946802 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946352    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 17:10:31.946802 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946354    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 17:10:31.946802 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946357    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 17:10:31.946802 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946360    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 17:10:31.946802 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946362    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 17:10:31.946802 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946365    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 17:10:31.946802 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946368    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 17:10:31.946802 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946370    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 17:10:31.946802 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946373    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 17:10:31.946802 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946375    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 17:10:31.947316 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946378    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 17:10:31.947316 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946380    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 17:10:31.947316 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946382    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 17:10:31.947316 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946385    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 17:10:31.947316 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946387    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 17:10:31.947316 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946389    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 17:10:31.947316 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946392    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 17:10:31.947316 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946394    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 17:10:31.947316 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946396    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 17:10:31.947316 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946399    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 17:10:31.947316 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946401    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 17:10:31.947316 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946403    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 17:10:31.947316 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946405    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 17:10:31.947316 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946410    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 17:10:31.947316 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946412    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 17:10:31.947316 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946415    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 17:10:31.947316 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946417    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 17:10:31.947316 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946420    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 17:10:31.947316 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946422    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 17:10:31.947316 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946425    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 17:10:31.947823 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946427    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 17:10:31.947823 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946430    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 17:10:31.947823 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946434    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 17:10:31.947823 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946436    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 17:10:31.947823 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946438    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 17:10:31.947823 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946441    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 17:10:31.947823 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946443    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 17:10:31.947823 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946459    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 17:10:31.947823 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946462    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 17:10:31.947823 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946464    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 17:10:31.947823 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946468    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 17:10:31.947823 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946471    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 17:10:31.947823 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946474    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 17:10:31.947823 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946476    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 17:10:31.947823 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946479    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 17:10:31.947823 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946482    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 17:10:31.947823 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946484    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 17:10:31.947823 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946487    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 17:10:31.947823 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946489    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 17:10:31.947823 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946492    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 17:10:31.948303 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946495    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 17:10:31.948303 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946498    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 17:10:31.948303 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946500    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 17:10:31.948303 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946503    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 17:10:31.948303 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946506    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 17:10:31.948303 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946508    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 17:10:31.948303 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946511    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 17:10:31.948303 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946513    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 17:10:31.948303 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946516    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 17:10:31.948303 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946518    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 17:10:31.948303 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946521    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 17:10:31.948303 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946523    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 17:10:31.948303 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946525    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 17:10:31.948303 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946528    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 17:10:31.948303 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946530    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 17:10:31.948303 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946532    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 17:10:31.948303 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946536    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 17:10:31.948303 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946539    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 17:10:31.948303 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946923    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 17:10:31.948303 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946928    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 17:10:31.948798 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946931    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 17:10:31.948798 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946934    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 17:10:31.948798 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946938    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 17:10:31.948798 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946941    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 17:10:31.948798 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946943    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 17:10:31.948798 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946946    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 17:10:31.948798 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946949    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 17:10:31.948798 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946952    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 17:10:31.948798 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946954    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 17:10:31.948798 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946957    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 17:10:31.948798 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946960    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 17:10:31.948798 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946962    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 17:10:31.948798 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946965    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 17:10:31.948798 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946967    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 17:10:31.948798 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946970    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 17:10:31.948798 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946972    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 17:10:31.948798 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946975    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 17:10:31.948798 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946977    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 17:10:31.948798 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946979    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 17:10:31.948798 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946982    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 17:10:31.949287 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946984    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 17:10:31.949287 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946987    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 17:10:31.949287 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946989    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 17:10:31.949287 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946992    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 17:10:31.949287 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946994    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 17:10:31.949287 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946997    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 17:10:31.949287 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.946999    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 17:10:31.949287 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947002    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 17:10:31.949287 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947004    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 17:10:31.949287 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947006    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 17:10:31.949287 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947009    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 17:10:31.949287 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947012    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 17:10:31.949287 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947014    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 17:10:31.949287 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947017    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 17:10:31.949287 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947020    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 17:10:31.949287 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947022    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 17:10:31.949287 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947027    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 17:10:31.949287 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947031    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 17:10:31.949287 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947034    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 17:10:31.949796 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947036    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 17:10:31.949796 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947039    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 17:10:31.949796 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947042    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 17:10:31.949796 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947044    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 17:10:31.949796 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947047    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 17:10:31.949796 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947049    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 17:10:31.949796 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947051    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 17:10:31.949796 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947054    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 17:10:31.949796 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947056    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 17:10:31.949796 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947059    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 17:10:31.949796 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947061    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 17:10:31.949796 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947064    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 17:10:31.949796 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947066    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 17:10:31.949796 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947068    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 17:10:31.949796 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947070    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 17:10:31.949796 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947073    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 17:10:31.949796 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947075    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 17:10:31.949796 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947079    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 17:10:31.949796 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947082    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 17:10:31.950313 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947084    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 17:10:31.950313 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947087    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 17:10:31.950313 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947090    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 17:10:31.950313 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947092    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 17:10:31.950313 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947095    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 17:10:31.950313 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947098    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 17:10:31.950313 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947100    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 17:10:31.950313 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947103    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 17:10:31.950313 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947105    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 17:10:31.950313 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947108    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 17:10:31.950313 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947111    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 17:10:31.950313 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947114    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 17:10:31.950313 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947116    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 17:10:31.950313 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947118    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 17:10:31.950313 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947121    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 17:10:31.950313 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947123    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 17:10:31.950313 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947126    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 17:10:31.950313 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947128    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 17:10:31.950313 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947130    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 17:10:31.950313 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947133    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 17:10:31.950821 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947135    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 17:10:31.950821 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947137    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 17:10:31.950821 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947139    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 17:10:31.950821 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947142    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 17:10:31.950821 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947145    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 17:10:31.950821 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947148    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 17:10:31.950821 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947150    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 17:10:31.950821 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947153    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 17:10:31.950821 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947155    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 17:10:31.950821 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947158    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 17:10:31.950821 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947160    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 17:10:31.950821 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947164    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 17:10:31.950821 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947166    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 17:10:31.950821 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947168    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 17:10:31.950821 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947171    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 17:10:31.950821 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947173    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 17:10:31.950821 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947176    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 17:10:31.950821 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947178    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 17:10:31.950821 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947184    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 17:10:31.950821 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947260    2581 flags.go:64] FLAG: --address="0.0.0.0"
Jul 24 17:10:31.951301 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947268    2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 24 17:10:31.951301 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947292    2581 flags.go:64] FLAG: --anonymous-auth="true"
Jul 24 17:10:31.951301 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947298    2581 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 24 17:10:31.951301 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947305    2581 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 24 17:10:31.951301 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947309    2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 24 17:10:31.951301 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947314    2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 24 17:10:31.951301 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947318    2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 24 17:10:31.951301 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947321    2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 24 17:10:31.951301 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947324    2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 24 17:10:31.951301 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947328    2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 24 17:10:31.951301 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947331    2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 24 17:10:31.951301 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947334    2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 24 17:10:31.951301 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947337    2581 flags.go:64] FLAG: --cgroup-root=""
Jul 24 17:10:31.951301 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947340    2581 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 24 17:10:31.951301 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947343    2581 flags.go:64] FLAG: --client-ca-file=""
Jul 24 17:10:31.951301 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947346    2581 flags.go:64] FLAG: --cloud-provider="external"
Jul 24 17:10:31.951301 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947349    2581 flags.go:64] FLAG: --cluster-dns="[]"
Jul 24 17:10:31.951301 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947353    2581 flags.go:64] FLAG: --cluster-domain=""
Jul 24 17:10:31.951301 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947356    2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 24 17:10:31.951301 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947359    2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 24 17:10:31.951301 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947362    2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 24 17:10:31.951301 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947365    2581 flags.go:64] FLAG: --container-log-max-files="5"
Jul 24 17:10:31.951301 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947369    2581 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 24 17:10:31.951301 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947372    2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 24 17:10:31.951890 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947376    2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 24 17:10:31.951890 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947379    2581 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 24 17:10:31.951890 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947382    2581 flags.go:64] FLAG: --contention-profiling="false"
Jul 24 17:10:31.951890 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947385    2581 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 24 17:10:31.951890 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947388    2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 24 17:10:31.951890 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947391    2581 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 24 17:10:31.951890 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947395    2581 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 24 17:10:31.951890 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947400    2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 24 17:10:31.951890 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947403    2581 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 24 17:10:31.951890 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947406    2581 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 24 17:10:31.951890 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947409    2581 flags.go:64] FLAG: --enable-load-reader="false"
Jul 24 17:10:31.951890 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947411    2581 flags.go:64] FLAG: --enable-server="true"
Jul 24 17:10:31.951890 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947415    2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 24 17:10:31.951890 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947420    2581 flags.go:64] FLAG: --event-burst="100"
Jul 24 17:10:31.951890 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947423    2581 flags.go:64] FLAG: --event-qps="50"
Jul 24 17:10:31.951890 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947425    2581 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 24 17:10:31.951890 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947428    2581 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 24 17:10:31.951890 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947431    2581 flags.go:64] FLAG: --eviction-hard=""
Jul 24 17:10:31.951890 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947435    2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 24 17:10:31.951890 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947438    2581 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 24 17:10:31.951890 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947441    2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 24 17:10:31.951890 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947444    2581 flags.go:64] FLAG: --eviction-soft=""
Jul 24 17:10:31.951890 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947461    2581 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 24 17:10:31.951890 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947464    2581 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 24 17:10:31.951890 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947467    2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 24 17:10:31.952533 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947470    2581 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 24 17:10:31.952533 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947473    2581 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 24 17:10:31.952533 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947476    2581 flags.go:64] FLAG: --fail-swap-on="true"
Jul 24 17:10:31.952533 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947478    2581 flags.go:64] FLAG: --feature-gates=""
Jul 24 17:10:31.952533 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947482    2581 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 24 17:10:31.952533 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947485    2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 24 17:10:31.952533 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947488    2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 24 17:10:31.952533 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947491    2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 24 17:10:31.952533 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947494    2581 flags.go:64] FLAG: --healthz-port="10248"
Jul 24 17:10:31.952533 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947497    2581 flags.go:64] FLAG: --help="false"
Jul 24 17:10:31.952533 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947500    2581 flags.go:64] FLAG: --hostname-override="ip-10-0-139-46.ec2.internal"
Jul 24 17:10:31.952533 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947503    2581 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 24 17:10:31.952533 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947506    2581 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 24 17:10:31.952533 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947510    2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 24 17:10:31.952533 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947515    2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 24 17:10:31.952533 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947518    2581 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 24 17:10:31.952533 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947521    2581 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 24 17:10:31.952533 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947524    2581 flags.go:64] FLAG: --image-service-endpoint=""
Jul 24 17:10:31.952533 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947526    2581 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 24 17:10:31.952533 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947530    2581 flags.go:64] FLAG: --kube-api-burst="100"
Jul 24 17:10:31.952533 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947533    2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 24 17:10:31.952533 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947536    2581 flags.go:64] FLAG: --kube-api-qps="50"
Jul 24 17:10:31.952533 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947539    2581 flags.go:64] FLAG: --kube-reserved=""
Jul 24 17:10:31.952533 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947543    2581 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 24 17:10:31.953158 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947545    2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 24 17:10:31.953158 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947548    2581 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 24 17:10:31.953158 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947551    2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 24 17:10:31.953158 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947554    2581 flags.go:64] FLAG: --lock-file=""
Jul 24 17:10:31.953158 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947556    2581 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 24 17:10:31.953158 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947559    2581 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 24 17:10:31.953158 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947562    2581 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 24 17:10:31.953158 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947568    2581 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 24 17:10:31.953158 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947570    2581 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 24 17:10:31.953158 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947573    2581 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 24 17:10:31.953158 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947576    2581 flags.go:64] FLAG: --logging-format="text"
Jul 24 17:10:31.953158 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947578    2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 24 17:10:31.953158 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947582    2581 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 24 17:10:31.953158 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947584    2581 flags.go:64] FLAG: --manifest-url=""
Jul 24 17:10:31.953158 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947587    2581 flags.go:64] FLAG: --manifest-url-header=""
Jul 24 17:10:31.953158 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947591    2581 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 24 17:10:31.953158 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947594    2581 flags.go:64] FLAG: --max-open-files="1000000"
Jul 24 17:10:31.953158 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947598    2581 flags.go:64] FLAG: --max-pods="110"
Jul 24 17:10:31.953158 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947601    2581 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 24 17:10:31.953158 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947604    2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 24 17:10:31.953158 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947606    2581 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 24 17:10:31.953158 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947610    2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 24 17:10:31.953158 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947615    2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 24 17:10:31.953158 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947618    2581 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 24 17:10:31.953158 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947621    2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 24 17:10:31.953776 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947627    2581 flags.go:64] FLAG: --node-status-max-images="50"
Jul 24 17:10:31.953776 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947630    2581 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 24 17:10:31.953776 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947633    2581 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 24 17:10:31.953776 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947636    2581 flags.go:64] FLAG: --pod-cidr=""
Jul 24 17:10:31.953776 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947640    2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 24 17:10:31.953776 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947645    2581 flags.go:64] FLAG: --pod-manifest-path=""
Jul 24 17:10:31.953776 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947648    2581 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 24 17:10:31.953776 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947651    2581 flags.go:64] FLAG: --pods-per-core="0"
Jul 24 17:10:31.953776 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947654    2581 flags.go:64] FLAG: --port="10250"
Jul 24 17:10:31.953776 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947657    2581 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 24 17:10:31.953776 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947660    2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-04ec90d6d6bcf41b0"
Jul 24 17:10:31.953776 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947663    2581 flags.go:64] FLAG: --qos-reserved=""
Jul 24 17:10:31.953776 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947666    2581 flags.go:64] FLAG: --read-only-port="10255"
Jul 24 17:10:31.953776 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947668    2581 flags.go:64] FLAG: --register-node="true"
Jul 24 17:10:31.953776 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947671    2581 flags.go:64] FLAG: --register-with-taints=""
Jul 24 17:10:31.953776 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947674    2581 flags.go:64] FLAG: --registry-burst="10"
Jul 24 17:10:31.953776 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947677    2581 flags.go:64] FLAG: --registry-qps="5"
Jul 24 17:10:31.953776 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947680    2581 flags.go:64] FLAG: --reserved-cpus=""
Jul 24 17:10:31.953776 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947682    2581 flags.go:64] FLAG: --reserved-memory=""
Jul 24 17:10:31.953776 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947686    2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 24 17:10:31.953776 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947688    2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 24 17:10:31.953776 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947691    2581 flags.go:64] FLAG: --rotate-certificates="false"
Jul 24 17:10:31.953776 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947694    2581 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 24 17:10:31.953776 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947697    2581 flags.go:64] FLAG: --runonce="false"
Jul 24 17:10:31.953776 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947699    2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 24 17:10:31.954461 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947702    2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 24 17:10:31.954461 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947705    2581 flags.go:64] FLAG: --seccomp-default="false"
Jul 24 17:10:31.954461 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947708    2581 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 24 17:10:31.954461 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947711    2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 24 17:10:31.954461 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947716    2581 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 24 17:10:31.954461 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947719    2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 24 17:10:31.954461 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947722    2581 flags.go:64] FLAG: --storage-driver-password="root"
Jul 24 17:10:31.954461 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947725    2581 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 24 17:10:31.954461 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947728    2581 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 24 17:10:31.954461 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947731    2581 flags.go:64] FLAG: --storage-driver-user="root"
Jul 24 17:10:31.954461 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947733    2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 24 17:10:31.954461 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947736    2581 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 24 17:10:31.954461 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947739    2581 flags.go:64] FLAG: --system-cgroups=""
Jul 24 17:10:31.954461 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947742    2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 24 17:10:31.954461 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947747    2581 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 24 17:10:31.954461 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947750    2581 flags.go:64] FLAG: --tls-cert-file=""
Jul 24 17:10:31.954461 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947753    2581 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 24 17:10:31.954461 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947757    2581 flags.go:64] FLAG: --tls-min-version=""
Jul 24 17:10:31.954461 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947760    2581 flags.go:64] FLAG: --tls-private-key-file=""
Jul 24 17:10:31.954461 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947763    2581 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 24 17:10:31.954461 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947765    2581 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 24 17:10:31.954461 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947768    2581 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 24 17:10:31.954461 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947771    2581 flags.go:64] FLAG: --v="2"
Jul 24 17:10:31.954461 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947775    2581 flags.go:64] FLAG: --version="false"
Jul 24 17:10:31.954461 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947779    2581 flags.go:64] FLAG: --vmodule=""
Jul 24 17:10:31.955053 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947783    2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 24 17:10:31.955053 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.947786    2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 24 17:10:31.955053 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947898    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 17:10:31.955053 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947902    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 17:10:31.955053 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947905    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 17:10:31.955053 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947908    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 17:10:31.955053 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947911    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 17:10:31.955053 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947914    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 17:10:31.955053 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947916    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 17:10:31.955053 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947919    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 17:10:31.955053 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947922    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 17:10:31.955053 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947927    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 17:10:31.955053 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947930    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 17:10:31.955053 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947932    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 17:10:31.955053 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947935    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 17:10:31.955053 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947938    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 17:10:31.955053 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947940    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 17:10:31.955053 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947943    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 17:10:31.955053 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947945    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 17:10:31.955541 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947948    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 17:10:31.955541 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947950    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 17:10:31.955541 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947953    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 17:10:31.955541 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947955    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 17:10:31.955541 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947958    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 17:10:31.955541 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947960    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 17:10:31.955541 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947963    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 17:10:31.955541 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947965    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 17:10:31.955541 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947967    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 17:10:31.955541 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947970    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 17:10:31.955541 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947972    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 17:10:31.955541 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947974    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 17:10:31.955541 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947977    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 17:10:31.955541 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947979    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 17:10:31.955541 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947981    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 17:10:31.955541 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947984    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 17:10:31.955541 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947986    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 17:10:31.955541 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947988    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 17:10:31.955541 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947990    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 17:10:31.955541 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947993    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 17:10:31.956021 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947996    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 17:10:31.956021 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.947998    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 17:10:31.956021 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948000    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 17:10:31.956021 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948002    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 17:10:31.956021 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948007    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 17:10:31.956021 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948009    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 17:10:31.956021 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948011    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 17:10:31.956021 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948014    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 17:10:31.956021 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948016    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 17:10:31.956021 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948018    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 17:10:31.956021 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948021    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 17:10:31.956021 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948023    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 17:10:31.956021 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948026    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 17:10:31.956021 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948028    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 17:10:31.956021 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948031    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 17:10:31.956021 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948033    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 17:10:31.956021 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948036    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 17:10:31.956021 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948038    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 17:10:31.956021 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948040    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 17:10:31.956021 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948043    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 17:10:31.956517 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948045    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 17:10:31.956517 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948048    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 17:10:31.956517 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948051    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 17:10:31.956517 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948055    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 17:10:31.956517 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948058    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 17:10:31.956517 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948060    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 17:10:31.956517 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948062    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 17:10:31.956517 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948065    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 17:10:31.956517 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948067    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 17:10:31.956517 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948070    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 17:10:31.956517 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948072    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 17:10:31.956517 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948075    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 17:10:31.956517 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948077    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 17:10:31.956517 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948080    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 17:10:31.956517 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948084    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 17:10:31.956517 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948087    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 17:10:31.956517 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948092    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 17:10:31.956517 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948095    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 17:10:31.956517 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948097    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 17:10:31.956517 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948100    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 17:10:31.957011 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948103    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 17:10:31.957011 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948106    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 17:10:31.957011 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948108    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 17:10:31.957011 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948111    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 17:10:31.957011 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948113    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 17:10:31.957011 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948115    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 17:10:31.957011 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948118    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 17:10:31.957011 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948121    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 17:10:31.957011 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948124    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 17:10:31.957011 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948126    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 17:10:31.957011 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948129    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 17:10:31.957011 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948131    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 17:10:31.957011 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948133    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 17:10:31.957011 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948136    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 17:10:31.957011 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948139    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 17:10:31.957011 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948141    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 17:10:31.957011 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948143    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 17:10:31.957011 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948147    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 17:10:31.957011 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948150    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 17:10:31.957011 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948153    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 17:10:31.957509 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948156    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 17:10:31.957509 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.948158    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 17:10:31.957509 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.948164    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 17:10:31.957509 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.955713    2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 24 17:10:31.957509 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.955732    2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 24 17:10:31.957509 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.955751    2581 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 24 17:10:31.957509 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.955757    2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 24 17:10:31.957509 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955819    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 17:10:31.957509 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955824    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 17:10:31.957509 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955828    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 17:10:31.957509 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955831    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 17:10:31.957509 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955834    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 17:10:31.957509 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955837    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 17:10:31.957509 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955840    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 17:10:31.957509 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955842    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 17:10:31.957888 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955845    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 17:10:31.957888 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955847    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 17:10:31.957888 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955852    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 17:10:31.957888 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955855    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 17:10:31.957888 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955858    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 17:10:31.957888 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955861    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 17:10:31.957888 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955864    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 17:10:31.957888 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955866    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 17:10:31.957888 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955868    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 17:10:31.957888 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955871    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 17:10:31.957888 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955877    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 17:10:31.957888 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955885    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 17:10:31.957888 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955888    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 17:10:31.957888 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955890    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 17:10:31.957888 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955892    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 17:10:31.957888 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955895    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 17:10:31.957888 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955897    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 17:10:31.957888 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955900    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 17:10:31.957888 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955902    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 17:10:31.957888 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955904    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 17:10:31.958365 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955907    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 17:10:31.958365 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955917    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 17:10:31.958365 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955920    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 17:10:31.958365 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955922    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 17:10:31.958365 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955924    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 17:10:31.958365 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955927    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 17:10:31.958365 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955931    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 17:10:31.958365 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955937    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 17:10:31.958365 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955940    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 17:10:31.958365 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955943    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 17:10:31.958365 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955946    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 17:10:31.958365 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955948    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 17:10:31.958365 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955950    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 17:10:31.958365 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955953    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 17:10:31.958365 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955955    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 17:10:31.958365 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955957    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 17:10:31.958365 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955959    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 17:10:31.958365 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955962    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 17:10:31.958365 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955964    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 17:10:31.958884 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955967    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 17:10:31.958884 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955970    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 17:10:31.958884 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955974    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 17:10:31.958884 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955976    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 17:10:31.958884 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955979    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 17:10:31.958884 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955981    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 17:10:31.958884 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955983    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 17:10:31.958884 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955986    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 17:10:31.958884 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955988    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 17:10:31.958884 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955990    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 17:10:31.958884 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955993    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 17:10:31.958884 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955995    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 17:10:31.958884 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.955998    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 17:10:31.958884 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956000    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 17:10:31.958884 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956003    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 17:10:31.958884 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956010    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 17:10:31.958884 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956013    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 17:10:31.958884 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956016    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 17:10:31.958884 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956018    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 17:10:31.959346 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956021    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 17:10:31.959346 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956024    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 17:10:31.959346 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956026    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 17:10:31.959346 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956029    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 17:10:31.959346 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956032    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 17:10:31.959346 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956034    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 17:10:31.959346 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956037    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 17:10:31.959346 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956040    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 17:10:31.959346 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956042    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 17:10:31.959346 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956044    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 17:10:31.959346 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956046    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 17:10:31.959346 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956049    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 17:10:31.959346 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956051    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 17:10:31.959346 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956053    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 17:10:31.959346 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956055    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 17:10:31.959346 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956058    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 17:10:31.959346 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956061    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 17:10:31.959346 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956064    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 17:10:31.959346 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956066    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 17:10:31.959346 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956069    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 17:10:31.959861 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956071    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 17:10:31.959861 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956073    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 17:10:31.959861 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956076    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 17:10:31.959861 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956078    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 17:10:31.959861 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956080    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 17:10:31.959861 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956083    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 17:10:31.959861 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956085    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 17:10:31.959861 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956087    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 17:10:31.959861 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956089    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 17:10:31.959861 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956097    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 17:10:31.959861 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956100    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 17:10:31.959861 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956102    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 17:10:31.959861 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956105    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 17:10:31.959861 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.956109    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 17:10:31.959861 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956253    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 17:10:31.960245 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956257    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 17:10:31.960245 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956261    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 17:10:31.960245 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956265    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 17:10:31.960245 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956268    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 17:10:31.960245 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956271    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 17:10:31.960245 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956273    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 17:10:31.960245 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956276    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 17:10:31.960245 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956278    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 17:10:31.960245 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956281    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 17:10:31.960245 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956284    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 17:10:31.960245 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956286    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 17:10:31.960245 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956290    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 17:10:31.960245 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956294    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 17:10:31.960245 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956297    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 17:10:31.960245 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956300    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 17:10:31.960245 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956303    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 17:10:31.960245 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956306    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 17:10:31.960245 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956308    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 17:10:31.960245 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956311    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 17:10:31.960727 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956314    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 17:10:31.960727 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956316    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 17:10:31.960727 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956319    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 17:10:31.960727 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956321    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 17:10:31.960727 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956324    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 17:10:31.960727 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956326    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 17:10:31.960727 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956329    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 17:10:31.960727 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956331    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 17:10:31.960727 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956341    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 17:10:31.960727 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956343    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 17:10:31.960727 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956346    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 17:10:31.960727 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956348    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 17:10:31.960727 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956350    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 17:10:31.960727 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956353    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 17:10:31.960727 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956355    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 17:10:31.960727 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956358    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 17:10:31.960727 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956360    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 17:10:31.960727 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956362    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 17:10:31.960727 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956365    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 17:10:31.960727 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956367    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 17:10:31.961219 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956369    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 17:10:31.961219 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956372    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 17:10:31.961219 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956374    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 17:10:31.961219 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956376    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 17:10:31.961219 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956379    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 17:10:31.961219 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956381    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 17:10:31.961219 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956384    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 17:10:31.961219 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956386    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 17:10:31.961219 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956389    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 17:10:31.961219 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956391    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 17:10:31.961219 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956394    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 17:10:31.961219 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956396    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 17:10:31.961219 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956398    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 17:10:31.961219 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956401    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 17:10:31.961219 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956404    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 17:10:31.961219 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956407    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 17:10:31.961219 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956409    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 17:10:31.961219 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956412    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 17:10:31.961219 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956414    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 17:10:31.961701 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956416    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 17:10:31.961701 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956419    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 17:10:31.961701 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956426    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 17:10:31.961701 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956429    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 17:10:31.961701 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956431    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 17:10:31.961701 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956433    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 17:10:31.961701 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956436    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 17:10:31.961701 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956438    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 17:10:31.961701 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956440    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 17:10:31.961701 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956443    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 17:10:31.961701 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956445    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 17:10:31.961701 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956464    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 17:10:31.961701 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956466    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 17:10:31.961701 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956469    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 17:10:31.961701 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956471    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 17:10:31.961701 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956473    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 17:10:31.961701 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956476    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 17:10:31.961701 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956478    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 17:10:31.961701 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956481    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 17:10:31.961701 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956486    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 17:10:31.962239 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956488    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 17:10:31.962239 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956491    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 17:10:31.962239 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956493    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 17:10:31.962239 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956496    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 17:10:31.962239 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956498    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 17:10:31.962239 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956501    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 17:10:31.962239 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956503    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 17:10:31.962239 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956505    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 17:10:31.962239 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956508    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 17:10:31.962239 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956510    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 17:10:31.962239 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956513    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 17:10:31.962239 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956515    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 17:10:31.962239 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956518    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 17:10:31.962239 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956521    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 17:10:31.962239 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956523    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 17:10:31.962239 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956532    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 17:10:31.962239 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956535    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 17:10:31.962239 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956537    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 17:10:31.962239 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956540    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 17:10:31.962239 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:31.956542    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 17:10:31.962785 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.956548    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 17:10:31.962785 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.956672    2581 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 24 17:10:31.962785 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.959250    2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 24 17:10:31.962785 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.961687    2581 server.go:1019] "Starting client certificate rotation"
Jul 24 17:10:31.962785 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.961790    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 17:10:31.962785 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.961822    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 17:10:31.968161 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.968136    2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 17:10:31.969646 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.969632    2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 17:10:31.984812 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.984794    2581 log.go:25] "Validated CRI v1 runtime API"
Jul 24 17:10:31.987105 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.987089    2581 log.go:25] "Validated CRI v1 image API"
Jul 24 17:10:31.987952 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.987932    2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 24 17:10:31.993842 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.993825    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 17:10:31.993992 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.993972    2581 fs.go:135] Filesystem UUIDs: map[5e5d9829-be1c-4587-9ce4-4801dfa7c529:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 adee8438-58b7-44e0-852c-1fe960abda65:/dev/nvme0n1p4]
Jul 24 17:10:31.994032 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.993994    2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 24 17:10:31.998727 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.998618    2581 manager.go:216] Machine: {Timestamp:2026-07-24 17:10:31.99811488 +0000 UTC m=+0.393752631 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3025335 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2dea5b803ded24f0ddd4a8d271fe6c SystemUUID:ec2dea5b-803d-ed24-f0dd-d4a8d271fe6c BootID:054763be-9f6d-4e03-a2a7-5aa831fbf9df Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:5b:ad:0a:29:6b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:5b:ad:0a:29:6b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6a:5e:dd:41:79:6a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 24 17:10:31.998727 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.998724    2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 24 17:10:31.998849 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:31.998802    2581 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 24 17:10:32.001529 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.001502    2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 24 17:10:32.001672 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.001532    2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-46.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 24 17:10:32.003191 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.003180    2581 topology_manager.go:138] "Creating topology manager with none policy"
Jul 24 17:10:32.003236 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.003194    2581 container_manager_linux.go:309] "Creating device plugin manager"
Jul 24 17:10:32.003236 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.003202    2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 17:10:32.003891 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.003880    2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 17:10:32.004927 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.004917    2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 24 17:10:32.007055 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.007044    2581 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 17:10:32.007177 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.007168    2581 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 24 17:10:32.007215 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.007197    2581 kubelet.go:486] "Attempting to sync node with API server"
Jul 24 17:10:32.007242 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.007214    2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 24 17:10:32.007242 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.007232    2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 24 17:10:32.007242 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.007241    2581 kubelet.go:398] "Adding apiserver pod source"
Jul 24 17:10:32.007317 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.007251    2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 24 17:10:32.007670 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.007659    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 17:10:32.007711 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.007679    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 17:10:32.011903 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.011874    2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 24 17:10:32.014518 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.014502    2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 24 17:10:32.014597 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.014535    2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 24 17:10:32.016175 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.016146    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ssf7f"
Jul 24 17:10:32.017129 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.017113    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 24 17:10:32.017191 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.017134    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 24 17:10:32.017191 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.017141    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 24 17:10:32.017191 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.017146    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 24 17:10:32.017191 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.017154    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 24 17:10:32.017191 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.017160    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 24 17:10:32.017191 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.017167    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 24 17:10:32.017191 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.017172    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 24 17:10:32.017191 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.017179    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 24 17:10:32.017191 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.017184    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 24 17:10:32.017191 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.017193    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 24 17:10:32.017471 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.017198    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 24 17:10:32.017988 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.017958    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 24 17:10:32.018042 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.017992    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 24 17:10:32.018352 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:32.018328    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-46.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 24 17:10:32.018428 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:32.018365    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 24 17:10:32.021315 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.021301    2581 server.go:1268] "Started kubelet"
Jul 24 17:10:32.021485 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.021425    2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 24 17:10:32.021601 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.021543    2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 24 17:10:32.021667 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.021623    2581 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 24 17:10:32.022237 ip-10-0-139-46 systemd[1]: Started Kubernetes Kubelet.
Jul 24 17:10:32.023655 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.023640    2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 24 17:10:32.023979 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.023965    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ssf7f"
Jul 24 17:10:32.024311 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.024297    2581 server.go:310] "Adding debug handlers to kubelet server"
Jul 24 17:10:32.027151 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.027122    2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-46.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 17:10:32.027424 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:32.027398    2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 24 17:10:32.028406 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:32.027144    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-46.ec2.internal.18c548f21c97ea27  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-46.ec2.internal,UID:ip-10-0-139-46.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-46.ec2.internal,},FirstTimestamp:2026-07-24 17:10:32.021273127 +0000 UTC m=+0.416910882,LastTimestamp:2026-07-24 17:10:32.021273127 +0000 UTC m=+0.416910882,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-46.ec2.internal,}"
Jul 24 17:10:32.033936 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.033917    2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 24 17:10:32.033936 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.033924    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 24 17:10:32.034079 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:32.034013    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-46.ec2.internal\" not found"
Jul 24 17:10:32.034754 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.034733    2581 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 24 17:10:32.034828 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.034812    2581 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 24 17:10:32.034828 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.034742    2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 24 17:10:32.034918 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.034870    2581 factory.go:55] Registering systemd factory
Jul 24 17:10:32.034918 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.034882    2581 factory.go:223] Registration of the systemd container factory successfully
Jul 24 17:10:32.035021 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.034920    2581 reconstruct.go:97] "Volume reconstruction finished"
Jul 24 17:10:32.035021 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.034926    2581 reconciler.go:29] "Reconciler: start to sync state"
Jul 24 17:10:32.035143 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.035127    2581 factory.go:153] Registering CRI-O factory
Jul 24 17:10:32.035143 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.035142    2581 factory.go:223] Registration of the crio container factory successfully
Jul 24 17:10:32.035282 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.035194    2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 24 17:10:32.035282 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.035225    2581 factory.go:103] Registering Raw factory
Jul 24 17:10:32.035282 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.035240    2581 manager.go:1196] Started watching for new ooms in manager
Jul 24 17:10:32.037093 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.035865    2581 manager.go:349] Starting recovery of all containers
Jul 24 17:10:32.037938 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.037907    2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 17:10:32.040669 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:32.040649    2581 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-139-46.ec2.internal\" not found" node="ip-10-0-139-46.ec2.internal"
Jul 24 17:10:32.046224 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.046062    2581 manager.go:354] Recovery completed
Jul 24 17:10:32.051347 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.051334    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 17:10:32.053346 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.053330    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-46.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 17:10:32.053405 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.053358    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-46.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 17:10:32.053405 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.053368    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-46.ec2.internal" event="NodeHasSufficientPID"
Jul 24 17:10:32.053912 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.053896    2581 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 24 17:10:32.053912 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.053910    2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 24 17:10:32.053912 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.053904    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 17:10:32.054047 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.053926    2581 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 17:10:32.054539 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.054525    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-46.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 17:10:32.054604 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.054553    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-46.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 17:10:32.054604 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.054564    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-46.ec2.internal" event="NodeHasSufficientPID"
Jul 24 17:10:32.055855 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.055843    2581 policy_none.go:49] "None policy: Start"
Jul 24 17:10:32.055930 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.055859    2581 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 24 17:10:32.055930 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.055869    2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 24 17:10:32.057079 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.057068    2581 policy_none.go:47] "Start"
Jul 24 17:10:32.093048 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.093033    2581 manager.go:337] "Starting Device Plugin manager"
Jul 24 17:10:32.093147 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:32.093068    2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 24 17:10:32.093147 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.093078    2581 server.go:87] "Starting device plugin registration server"
Jul 24 17:10:32.093564 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.093551    2581 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 24 17:10:32.093598 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.093567    2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 24 17:10:32.093691 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.093670    2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 24 17:10:32.093819 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.093774    2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 24 17:10:32.093819 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.093787    2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 24 17:10:32.094265 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:32.094242    2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 24 17:10:32.094332 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:32.094303    2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-46.ec2.internal\" not found"
Jul 24 17:10:32.182009 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.181976    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 24 17:10:32.183621 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.183288    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 24 17:10:32.183621 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.183324    2581 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 24 17:10:32.183621 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.183347    2581 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 24 17:10:32.183621 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:32.183383    2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 24 17:10:32.186321 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.186268    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 17:10:32.194554 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.194540    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 17:10:32.195331 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.195314    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-46.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 17:10:32.195412 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.195344    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-46.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 17:10:32.195412 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.195354    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-46.ec2.internal" event="NodeHasSufficientPID"
Jul 24 17:10:32.195412 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.195378    2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-46.ec2.internal"
Jul 24 17:10:32.204346 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.204332    2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-46.ec2.internal"
Jul 24 17:10:32.204407 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:32.204353    2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-46.ec2.internal\": node \"ip-10-0-139-46.ec2.internal\" not found"
Jul 24 17:10:32.219463 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:32.219429    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-46.ec2.internal\" not found"
Jul 24 17:10:32.284418 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.284382    2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-46.ec2.internal"]
Jul 24 17:10:32.284511 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.284487    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 17:10:32.289023 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.289004    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-46.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 17:10:32.289138 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.289034    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-46.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 17:10:32.289138 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.289044    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-46.ec2.internal" event="NodeHasSufficientPID"
Jul 24 17:10:32.290692 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.290679    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 17:10:32.290834 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.290792    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.ec2.internal"
Jul 24 17:10:32.290885 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.290846    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 17:10:32.292516 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.292500    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-46.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 17:10:32.292516 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.292511    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-46.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 17:10:32.292637 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.292523    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-46.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 17:10:32.292637 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.292533    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-46.ec2.internal" event="NodeHasSufficientPID"
Jul 24 17:10:32.292637 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.292535    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-46.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 17:10:32.292637 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.292548    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-46.ec2.internal" event="NodeHasSufficientPID"
Jul 24 17:10:32.294077 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.294065    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-46.ec2.internal"
Jul 24 17:10:32.294120 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.294087    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 17:10:32.294814 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.294797    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-46.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 17:10:32.294900 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.294830    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-46.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 17:10:32.294900 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.294844    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-46.ec2.internal" event="NodeHasSufficientPID"
Jul 24 17:10:32.320098 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:32.320079    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-46.ec2.internal\" not found"
Jul 24 17:10:32.322069 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:32.322054    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-46.ec2.internal\" not found" node="ip-10-0-139-46.ec2.internal"
Jul 24 17:10:32.326421 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:32.326407    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-46.ec2.internal\" not found" node="ip-10-0-139-46.ec2.internal"
Jul 24 17:10:32.336780 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.336764    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c23660c78fea953db7eac9d97b433c18-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-46.ec2.internal\" (UID: \"c23660c78fea953db7eac9d97b433c18\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.ec2.internal"
Jul 24 17:10:32.336835 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.336789    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c23660c78fea953db7eac9d97b433c18-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-46.ec2.internal\" (UID: \"c23660c78fea953db7eac9d97b433c18\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.ec2.internal"
Jul 24 17:10:32.336835 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.336807    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/39f953389fbb7b355964e9b9fe5538ef-config\") pod \"kube-apiserver-proxy-ip-10-0-139-46.ec2.internal\" (UID: \"39f953389fbb7b355964e9b9fe5538ef\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-46.ec2.internal"
Jul 24 17:10:32.420191 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:32.420148    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-46.ec2.internal\" not found"
Jul 24 17:10:32.437582 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.437523    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/39f953389fbb7b355964e9b9fe5538ef-config\") pod \"kube-apiserver-proxy-ip-10-0-139-46.ec2.internal\" (UID: \"39f953389fbb7b355964e9b9fe5538ef\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-46.ec2.internal"
Jul 24 17:10:32.437582 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.437563    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c23660c78fea953db7eac9d97b433c18-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-46.ec2.internal\" (UID: \"c23660c78fea953db7eac9d97b433c18\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.ec2.internal"
Jul 24 17:10:32.437582 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.437582    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c23660c78fea953db7eac9d97b433c18-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-46.ec2.internal\" (UID: \"c23660c78fea953db7eac9d97b433c18\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.ec2.internal"
Jul 24 17:10:32.437703 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.437624    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c23660c78fea953db7eac9d97b433c18-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-46.ec2.internal\" (UID: \"c23660c78fea953db7eac9d97b433c18\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.ec2.internal"
Jul 24 17:10:32.437703 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.437625    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/39f953389fbb7b355964e9b9fe5538ef-config\") pod \"kube-apiserver-proxy-ip-10-0-139-46.ec2.internal\" (UID: \"39f953389fbb7b355964e9b9fe5538ef\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-46.ec2.internal"
Jul 24 17:10:32.437703 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.437651    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c23660c78fea953db7eac9d97b433c18-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-46.ec2.internal\" (UID: \"c23660c78fea953db7eac9d97b433c18\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.ec2.internal"
Jul 24 17:10:32.521009 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:32.520980    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-46.ec2.internal\" not found"
Jul 24 17:10:32.621513 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:32.621473    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-46.ec2.internal\" not found"
Jul 24 17:10:32.625652 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.625634    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.ec2.internal"
Jul 24 17:10:32.633079 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:32.633062    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-46.ec2.internal"
Jul 24 17:10:32.722620 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:32.722534    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-46.ec2.internal\" not found"
Jul 24 17:10:34.016718 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:34.016693    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-46.ec2.internal\" not found"
Jul 24 17:10:34.117271 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:34.117237    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-46.ec2.internal\" not found"
Jul 24 17:10:34.154775 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:34.154749    2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 24 17:10:34.155317 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:34.154879    2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 24 17:10:34.155317 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:34.154910    2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 24 17:10:34.205785 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:34.205725    2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 17:10:34.218172 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:34.218147    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-46.ec2.internal\" not found"
Jul 24 17:10:34.223428 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:34.223397    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-23 17:05:33 +0000 UTC" deadline="2028-02-13 05:49:39.532423873 +0000 UTC"
Jul 24 17:10:34.223498 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:34.223428    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13644h39m5.308999441s"
Jul 24 17:10:34.318608 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:34.318544    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-46.ec2.internal\" not found"
Jul 24 17:10:34.366483 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:34.366440    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc23660c78fea953db7eac9d97b433c18.slice/crio-238aeb0c02aa7d35c3e8c0ae13f59ef009479961782d8d101e9c6d800ee79231 WatchSource:0}: Error finding container 238aeb0c02aa7d35c3e8c0ae13f59ef009479961782d8d101e9c6d800ee79231: Status 404 returned error can't find the container with id 238aeb0c02aa7d35c3e8c0ae13f59ef009479961782d8d101e9c6d800ee79231
Jul 24 17:10:34.367009 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:34.366988    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39f953389fbb7b355964e9b9fe5538ef.slice/crio-5a5c53e98cd52b9cdddbdc0cff03d9d18f6d90c233ed3457e6cd7bfc9b916a55 WatchSource:0}: Error finding container 5a5c53e98cd52b9cdddbdc0cff03d9d18f6d90c233ed3457e6cd7bfc9b916a55: Status 404 returned error can't find the container with id 5a5c53e98cd52b9cdddbdc0cff03d9d18f6d90c233ed3457e6cd7bfc9b916a55
Jul 24 17:10:34.371691 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:34.371676    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 17:10:34.380188 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:34.380135    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.ec2.internal" event={"ID":"c23660c78fea953db7eac9d97b433c18","Type":"ContainerStarted","Data":"238aeb0c02aa7d35c3e8c0ae13f59ef009479961782d8d101e9c6d800ee79231"}
Jul 24 17:10:34.380970 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:34.380945    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-46.ec2.internal" event={"ID":"39f953389fbb7b355964e9b9fe5538ef","Type":"ContainerStarted","Data":"5a5c53e98cd52b9cdddbdc0cff03d9d18f6d90c233ed3457e6cd7bfc9b916a55"}
Jul 24 17:10:34.419282 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:34.419245    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-46.ec2.internal\" not found"
Jul 24 17:10:34.486697 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:34.486635    2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 17:10:34.528107 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:34.528084    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.ec2.internal"
Jul 24 17:10:34.539910 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:34.539891    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 17:10:34.541077 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:34.541064    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-46.ec2.internal"
Jul 24 17:10:34.550225 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:34.550206    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 17:10:35.202334 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.202051    2581 apiserver.go:52] "Watching apiserver"
Jul 24 17:10:35.212646 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.212621    2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 24 17:10:35.213245 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.213222    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-9lnl2","kube-system/global-pull-secret-syncer-v42pr","kube-system/kube-apiserver-proxy-ip-10-0-139-46.ec2.internal","openshift-image-registry/node-ca-hv6sp","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.ec2.internal","openshift-multus/multus-22jtw","openshift-network-diagnostics/network-check-target-t2znt","openshift-network-operator/iptables-alerter-df5fv","kube-system/konnectivity-agent-4c827","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb9cp","openshift-cluster-node-tuning-operator/tuned-zdv5f","openshift-dns/node-resolver-hpxzm","openshift-multus/multus-additional-cni-plugins-px6r8","openshift-multus/network-metrics-daemon-d42hh"]
Jul 24 17:10:35.215284 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.215261    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:10:35.215383 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:35.215349    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t2znt" podUID="bcf18baa-0b35-4089-9570-df68ef7a456a"
Jul 24 17:10:35.217214 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.216687    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.219524 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.219131    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:10:35.219524 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:35.219198    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d42hh" podUID="5c99646a-9e3e-4e60-a750-a96ab5aab0bf"
Jul 24 17:10:35.219524 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.219318    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 24 17:10:35.219524 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.219350    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 24 17:10:35.219524 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.219455    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-25h47\""
Jul 24 17:10:35.221446 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.221425    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hv6sp"
Jul 24 17:10:35.222291 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.222274    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-df5fv"
Jul 24 17:10:35.224292 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.223884    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.231766 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.225314    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4c827"
Jul 24 17:10:35.231766 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.226925    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-px6r8"
Jul 24 17:10:35.231766 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.228350    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:10:35.231766 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:35.228415    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v42pr" podUID="24d6b216-f67b-4d45-bd22-c9f159255177"
Jul 24 17:10:35.231766 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.228594    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 24 17:10:35.231766 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.229653    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb9cp"
Jul 24 17:10:35.231766 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.231212    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.234869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.233054    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hpxzm"
Jul 24 17:10:35.234869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.233234    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 24 17:10:35.234869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.233359    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 24 17:10:35.234869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.233436    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 24 17:10:35.234869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.233547    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 24 17:10:35.234869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.233612    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-4qgk5\""
Jul 24 17:10:35.234869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.233660    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 24 17:10:35.234869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.233863    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rhhrp\""
Jul 24 17:10:35.234869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.233885    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-zsq7g\""
Jul 24 17:10:35.234869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.233888    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 24 17:10:35.234869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.233977    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 24 17:10:35.234869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.234051    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 24 17:10:35.234869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.234114    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 24 17:10:35.234869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.234155    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 24 17:10:35.234869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.234055    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 24 17:10:35.234869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.234281    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 24 17:10:35.234869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.234348    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-krf2t\""
Jul 24 17:10:35.234869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.234376    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 24 17:10:35.234869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.234455    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 24 17:10:35.234869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.234558    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 24 17:10:35.234869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.234607    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 24 17:10:35.234869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.234699    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 24 17:10:35.234869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.234730    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 24 17:10:35.234869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.234774    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dclm7\""
Jul 24 17:10:35.234869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.234827    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 24 17:10:35.236006 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.234945    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 24 17:10:35.236006 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.235112    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-d9fj6\""
Jul 24 17:10:35.236006 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.235603    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 24 17:10:35.236006 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.235923    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 24 17:10:35.239787 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.236585    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-56wmj\""
Jul 24 17:10:35.239787 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.236702    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-svz4n\""
Jul 24 17:10:35.239787 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.236993    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 24 17:10:35.239787 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.237029    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 24 17:10:35.239787 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.237976    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 24 17:10:35.239994 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.239802    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndkct\" (UniqueName: \"kubernetes.io/projected/2071cf36-c0fa-4f24-bb3c-bd15dd2896d6-kube-api-access-ndkct\") pod \"iptables-alerter-df5fv\" (UID: \"2071cf36-c0fa-4f24-bb3c-bd15dd2896d6\") " pod="openshift-network-operator/iptables-alerter-df5fv"
Jul 24 17:10:35.239994 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.239841    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gvk2\" (UniqueName: \"kubernetes.io/projected/32f3d102-a6d1-41af-8410-42dc28eb6975-kube-api-access-8gvk2\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.239994 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.239873    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/24d6b216-f67b-4d45-bd22-c9f159255177-original-pull-secret\") pod \"global-pull-secret-syncer-v42pr\" (UID: \"24d6b216-f67b-4d45-bd22-c9f159255177\") " pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:10:35.239994 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.239903    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a-socket-dir\") pod \"aws-ebs-csi-driver-node-bb9cp\" (UID: \"30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb9cp"
Jul 24 17:10:35.239994 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.239938    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9417588a-ebcd-4fe0-a050-2345106ba8f3-var-lib-kubelet\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.239994 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.239960    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9417588a-ebcd-4fe0-a050-2345106ba8f3-etc-sysctl-conf\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.239994 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.239987    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9417588a-ebcd-4fe0-a050-2345106ba8f3-etc-tuned\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.240309 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.240016    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-host-run-k8s-cni-cncf-io\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.240309 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.240062    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-host-run-netns\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.240309 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.240090    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70-konnectivity-ca\") pod \"konnectivity-agent-4c827\" (UID: \"fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70\") " pod="kube-system/konnectivity-agent-4c827"
Jul 24 17:10:35.240309 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.240165    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a-device-dir\") pod \"aws-ebs-csi-driver-node-bb9cp\" (UID: \"30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb9cp"
Jul 24 17:10:35.240309 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.240212    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9417588a-ebcd-4fe0-a050-2345106ba8f3-lib-modules\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.240309 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.240246    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2071cf36-c0fa-4f24-bb3c-bd15dd2896d6-host-slash\") pod \"iptables-alerter-df5fv\" (UID: \"2071cf36-c0fa-4f24-bb3c-bd15dd2896d6\") " pod="openshift-network-operator/iptables-alerter-df5fv"
Jul 24 17:10:35.240309 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.240290    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-etc-kubernetes\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.240666 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.240343    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9417588a-ebcd-4fe0-a050-2345106ba8f3-etc-kubernetes\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.240666 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.240386    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9417588a-ebcd-4fe0-a050-2345106ba8f3-etc-systemd\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.240666 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.240414    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9f393f69-5a28-41ab-a160-42147718f3da-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-px6r8\" (UID: \"9f393f69-5a28-41ab-a160-42147718f3da\") " pod="openshift-multus/multus-additional-cni-plugins-px6r8"
Jul 24 17:10:35.240666 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.240445    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdfgk\" (UniqueName: \"kubernetes.io/projected/30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a-kube-api-access-bdfgk\") pod \"aws-ebs-csi-driver-node-bb9cp\" (UID: \"30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb9cp"
Jul 24 17:10:35.240666 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.240473    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4j6q\" (UniqueName: \"kubernetes.io/projected/bcf18baa-0b35-4089-9570-df68ef7a456a-kube-api-access-d4j6q\") pod \"network-check-target-t2znt\" (UID: \"bcf18baa-0b35-4089-9570-df68ef7a456a\") " pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:10:35.240666 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.240500    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c99646a-9e3e-4e60-a750-a96ab5aab0bf-metrics-certs\") pod \"network-metrics-daemon-d42hh\" (UID: \"5c99646a-9e3e-4e60-a750-a96ab5aab0bf\") " pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:10:35.240666 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.240553    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2071cf36-c0fa-4f24-bb3c-bd15dd2896d6-iptables-alerter-script\") pod \"iptables-alerter-df5fv\" (UID: \"2071cf36-c0fa-4f24-bb3c-bd15dd2896d6\") " pod="openshift-network-operator/iptables-alerter-df5fv"
Jul 24 17:10:35.240666 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.240588    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-multus-cni-dir\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.240666 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.240614    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9f393f69-5a28-41ab-a160-42147718f3da-system-cni-dir\") pod \"multus-additional-cni-plugins-px6r8\" (UID: \"9f393f69-5a28-41ab-a160-42147718f3da\") " pod="openshift-multus/multus-additional-cni-plugins-px6r8"
Jul 24 17:10:35.240666 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.240642    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9417588a-ebcd-4fe0-a050-2345106ba8f3-sys\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.241141 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.240672    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kls44\" (UniqueName: \"kubernetes.io/projected/5c99646a-9e3e-4e60-a750-a96ab5aab0bf-kube-api-access-kls44\") pod \"network-metrics-daemon-d42hh\" (UID: \"5c99646a-9e3e-4e60-a750-a96ab5aab0bf\") " pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:10:35.241141 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.240724    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-multus-socket-dir-parent\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.241141 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.240758    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-hostroot\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.241141 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.240798    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/24d6b216-f67b-4d45-bd22-c9f159255177-dbus\") pod \"global-pull-secret-syncer-v42pr\" (UID: \"24d6b216-f67b-4d45-bd22-c9f159255177\") " pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:10:35.241141 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.240828    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9417588a-ebcd-4fe0-a050-2345106ba8f3-run\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.241141 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.240868    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9417588a-ebcd-4fe0-a050-2345106ba8f3-tmp\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.241141 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.240942    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/24d6b216-f67b-4d45-bd22-c9f159255177-kubelet-config\") pod \"global-pull-secret-syncer-v42pr\" (UID: \"24d6b216-f67b-4d45-bd22-c9f159255177\") " pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:10:35.241141 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.240963    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bb9cp\" (UID: \"30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb9cp"
Jul 24 17:10:35.241141 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.240983    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9417588a-ebcd-4fe0-a050-2345106ba8f3-etc-sysctl-d\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.241141 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.241011    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-multus-conf-dir\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.241141 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.241059    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9f393f69-5a28-41ab-a160-42147718f3da-os-release\") pod \"multus-additional-cni-plugins-px6r8\" (UID: \"9f393f69-5a28-41ab-a160-42147718f3da\") " pod="openshift-multus/multus-additional-cni-plugins-px6r8"
Jul 24 17:10:35.241141 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.241096    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9417588a-ebcd-4fe0-a050-2345106ba8f3-etc-modprobe-d\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.241141 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.241135    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/32f3d102-a6d1-41af-8410-42dc28eb6975-cni-binary-copy\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.241767 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.241167    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/32f3d102-a6d1-41af-8410-42dc28eb6975-multus-daemon-config\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.241767 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.241198    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9f393f69-5a28-41ab-a160-42147718f3da-tuning-conf-dir\") pod \"multus-additional-cni-plugins-px6r8\" (UID: \"9f393f69-5a28-41ab-a160-42147718f3da\") " pod="openshift-multus/multus-additional-cni-plugins-px6r8"
Jul 24 17:10:35.241767 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.241224    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/17ed6b09-72e1-4161-9a92-cb481241bfe1-serviceca\") pod \"node-ca-hv6sp\" (UID: \"17ed6b09-72e1-4161-9a92-cb481241bfe1\") " pod="openshift-image-registry/node-ca-hv6sp"
Jul 24 17:10:35.241767 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.241270    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a-registration-dir\") pod \"aws-ebs-csi-driver-node-bb9cp\" (UID: \"30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb9cp"
Jul 24 17:10:35.241767 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.241301    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2jbc\" (UniqueName: \"kubernetes.io/projected/9417588a-ebcd-4fe0-a050-2345106ba8f3-kube-api-access-m2jbc\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.241767 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.241337    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-os-release\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.241767 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.241410    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-host-var-lib-cni-bin\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.241767 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.241470    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9417588a-ebcd-4fe0-a050-2345106ba8f3-etc-sysconfig\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.241767 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.241509    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-host-var-lib-cni-multus\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.241767 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.241545    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70-agent-certs\") pod \"konnectivity-agent-4c827\" (UID: \"fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70\") " pod="kube-system/konnectivity-agent-4c827"
Jul 24 17:10:35.241767 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.241576    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9f393f69-5a28-41ab-a160-42147718f3da-cni-binary-copy\") pod \"multus-additional-cni-plugins-px6r8\" (UID: \"9f393f69-5a28-41ab-a160-42147718f3da\") " pod="openshift-multus/multus-additional-cni-plugins-px6r8"
Jul 24 17:10:35.241767 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.241617    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jr8t8\" (UniqueName: \"kubernetes.io/projected/17ed6b09-72e1-4161-9a92-cb481241bfe1-kube-api-access-jr8t8\") pod \"node-ca-hv6sp\" (UID: \"17ed6b09-72e1-4161-9a92-cb481241bfe1\") " pod="openshift-image-registry/node-ca-hv6sp"
Jul 24 17:10:35.241767 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.241648    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a-etc-selinux\") pod \"aws-ebs-csi-driver-node-bb9cp\" (UID: \"30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb9cp"
Jul 24 17:10:35.241767 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.241750    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-cnibin\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.245515 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.241803    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9f393f69-5a28-41ab-a160-42147718f3da-cnibin\") pod \"multus-additional-cni-plugins-px6r8\" (UID: \"9f393f69-5a28-41ab-a160-42147718f3da\") " pod="openshift-multus/multus-additional-cni-plugins-px6r8"
Jul 24 17:10:35.245515 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.241839    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g97cz\" (UniqueName: \"kubernetes.io/projected/9f393f69-5a28-41ab-a160-42147718f3da-kube-api-access-g97cz\") pod \"multus-additional-cni-plugins-px6r8\" (UID: \"9f393f69-5a28-41ab-a160-42147718f3da\") " pod="openshift-multus/multus-additional-cni-plugins-px6r8"
Jul 24 17:10:35.245515 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.241917    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-host-var-lib-kubelet\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.245515 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.241945    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-host-run-multus-certs\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.245515 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.241968    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9417588a-ebcd-4fe0-a050-2345106ba8f3-host\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.245515 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.241996    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-system-cni-dir\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.245515 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.242107    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9f393f69-5a28-41ab-a160-42147718f3da-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-px6r8\" (UID: \"9f393f69-5a28-41ab-a160-42147718f3da\") " pod="openshift-multus/multus-additional-cni-plugins-px6r8"
Jul 24 17:10:35.245515 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.242137    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17ed6b09-72e1-4161-9a92-cb481241bfe1-host\") pod \"node-ca-hv6sp\" (UID: \"17ed6b09-72e1-4161-9a92-cb481241bfe1\") " pod="openshift-image-registry/node-ca-hv6sp"
Jul 24 17:10:35.245515 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.242166    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a-sys-fs\") pod \"aws-ebs-csi-driver-node-bb9cp\" (UID: \"30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb9cp"
Jul 24 17:10:35.245515 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.244215    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 17:10:35.260019 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.259998    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-8nxj5"
Jul 24 17:10:35.265259 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.265239    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-8nxj5"
Jul 24 17:10:35.329591 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.329562    2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 24 17:10:35.342323 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.342294    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2jbc\" (UniqueName: \"kubernetes.io/projected/9417588a-ebcd-4fe0-a050-2345106ba8f3-kube-api-access-m2jbc\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.342463 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.342333    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-os-release\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.342463 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.342356    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-host-var-lib-cni-bin\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.342463 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.342383    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-host-run-netns\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.342463 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.342445    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-host-var-lib-cni-bin\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.342696 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.342443    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-var-lib-openvswitch\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.342696 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.342502    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9417588a-ebcd-4fe0-a050-2345106ba8f3-etc-sysconfig\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.342696 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.342528    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-host-var-lib-cni-multus\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.342696 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.342555    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-os-release\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.342696 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.342562    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70-agent-certs\") pod \"konnectivity-agent-4c827\" (UID: \"fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70\") " pod="kube-system/konnectivity-agent-4c827"
Jul 24 17:10:35.342696 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.342597    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-run-openvswitch\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.342696 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.342620    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-host-cni-netd\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.342696 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.342623    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9417588a-ebcd-4fe0-a050-2345106ba8f3-etc-sysconfig\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.342696 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.342645    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-ovnkube-config\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.342696 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.342677    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-host-var-lib-cni-multus\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.342696 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.342680    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-ovn-node-metrics-cert\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.343218 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.342707    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9f393f69-5a28-41ab-a160-42147718f3da-cni-binary-copy\") pod \"multus-additional-cni-plugins-px6r8\" (UID: \"9f393f69-5a28-41ab-a160-42147718f3da\") " pod="openshift-multus/multus-additional-cni-plugins-px6r8"
Jul 24 17:10:35.343218 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.342835    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jr8t8\" (UniqueName: \"kubernetes.io/projected/17ed6b09-72e1-4161-9a92-cb481241bfe1-kube-api-access-jr8t8\") pod \"node-ca-hv6sp\" (UID: \"17ed6b09-72e1-4161-9a92-cb481241bfe1\") " pod="openshift-image-registry/node-ca-hv6sp"
Jul 24 17:10:35.343218 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.342875    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a-etc-selinux\") pod \"aws-ebs-csi-driver-node-bb9cp\" (UID: \"30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb9cp"
Jul 24 17:10:35.343218 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.342898    2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 24 17:10:35.343218 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343021    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a-etc-selinux\") pod \"aws-ebs-csi-driver-node-bb9cp\" (UID: \"30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb9cp"
Jul 24 17:10:35.343218 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.342903    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ccfd714f-9f15-40f6-a06c-6054adfb9992-hosts-file\") pod \"node-resolver-hpxzm\" (UID: \"ccfd714f-9f15-40f6-a06c-6054adfb9992\") " pod="openshift-dns/node-resolver-hpxzm"
Jul 24 17:10:35.343218 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343095    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ccfd714f-9f15-40f6-a06c-6054adfb9992-tmp-dir\") pod \"node-resolver-hpxzm\" (UID: \"ccfd714f-9f15-40f6-a06c-6054adfb9992\") " pod="openshift-dns/node-resolver-hpxzm"
Jul 24 17:10:35.343218 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343122    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-cnibin\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.343218 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343163    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9f393f69-5a28-41ab-a160-42147718f3da-cnibin\") pod \"multus-additional-cni-plugins-px6r8\" (UID: \"9f393f69-5a28-41ab-a160-42147718f3da\") " pod="openshift-multus/multus-additional-cni-plugins-px6r8"
Jul 24 17:10:35.343218 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343174    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-cnibin\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.343218 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343189    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g97cz\" (UniqueName: \"kubernetes.io/projected/9f393f69-5a28-41ab-a160-42147718f3da-kube-api-access-g97cz\") pod \"multus-additional-cni-plugins-px6r8\" (UID: \"9f393f69-5a28-41ab-a160-42147718f3da\") " pod="openshift-multus/multus-additional-cni-plugins-px6r8"
Jul 24 17:10:35.343218 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343215    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-systemd-units\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.343218 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343219    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9f393f69-5a28-41ab-a160-42147718f3da-cnibin\") pod \"multus-additional-cni-plugins-px6r8\" (UID: \"9f393f69-5a28-41ab-a160-42147718f3da\") " pod="openshift-multus/multus-additional-cni-plugins-px6r8"
Jul 24 17:10:35.343795 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343237    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-log-socket\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.343795 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343274    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-host-var-lib-kubelet\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.343795 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343300    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-host-run-multus-certs\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.343795 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343323    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9417588a-ebcd-4fe0-a050-2345106ba8f3-host\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.343795 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343346    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-system-cni-dir\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.343795 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343347    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-host-var-lib-kubelet\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.343795 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343371    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9f393f69-5a28-41ab-a160-42147718f3da-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-px6r8\" (UID: \"9f393f69-5a28-41ab-a160-42147718f3da\") " pod="openshift-multus/multus-additional-cni-plugins-px6r8"
Jul 24 17:10:35.343795 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343380    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-host-run-multus-certs\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.343795 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343396    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17ed6b09-72e1-4161-9a92-cb481241bfe1-host\") pod \"node-ca-hv6sp\" (UID: \"17ed6b09-72e1-4161-9a92-cb481241bfe1\") " pod="openshift-image-registry/node-ca-hv6sp"
Jul 24 17:10:35.343795 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343404    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9417588a-ebcd-4fe0-a050-2345106ba8f3-host\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.343795 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343419    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a-sys-fs\") pod \"aws-ebs-csi-driver-node-bb9cp\" (UID: \"30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb9cp"
Jul 24 17:10:35.343795 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343441    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-system-cni-dir\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.343795 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343446    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17ed6b09-72e1-4161-9a92-cb481241bfe1-host\") pod \"node-ca-hv6sp\" (UID: \"17ed6b09-72e1-4161-9a92-cb481241bfe1\") " pod="openshift-image-registry/node-ca-hv6sp"
Jul 24 17:10:35.343795 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343479    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a-sys-fs\") pod \"aws-ebs-csi-driver-node-bb9cp\" (UID: \"30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb9cp"
Jul 24 17:10:35.343795 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343506    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndkct\" (UniqueName: \"kubernetes.io/projected/2071cf36-c0fa-4f24-bb3c-bd15dd2896d6-kube-api-access-ndkct\") pod \"iptables-alerter-df5fv\" (UID: \"2071cf36-c0fa-4f24-bb3c-bd15dd2896d6\") " pod="openshift-network-operator/iptables-alerter-df5fv"
Jul 24 17:10:35.343795 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343548    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gvk2\" (UniqueName: \"kubernetes.io/projected/32f3d102-a6d1-41af-8410-42dc28eb6975-kube-api-access-8gvk2\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.343795 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343575    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/24d6b216-f67b-4d45-bd22-c9f159255177-original-pull-secret\") pod \"global-pull-secret-syncer-v42pr\" (UID: \"24d6b216-f67b-4d45-bd22-c9f159255177\") " pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:10:35.343795 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343597    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a-socket-dir\") pod \"aws-ebs-csi-driver-node-bb9cp\" (UID: \"30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb9cp"
Jul 24 17:10:35.344677 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343626    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-ovnkube-script-lib\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.344677 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343651    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9417588a-ebcd-4fe0-a050-2345106ba8f3-var-lib-kubelet\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.344677 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343675    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-host-kubelet\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.344677 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343692    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9417588a-ebcd-4fe0-a050-2345106ba8f3-etc-sysctl-conf\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.344677 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:35.343690    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 17:10:35.344677 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343706    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9417588a-ebcd-4fe0-a050-2345106ba8f3-etc-tuned\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.344677 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343721    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-host-run-k8s-cni-cncf-io\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.344677 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343756    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-host-run-k8s-cni-cncf-io\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.344677 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343769    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9417588a-ebcd-4fe0-a050-2345106ba8f3-var-lib-kubelet\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.344677 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:35.343783    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24d6b216-f67b-4d45-bd22-c9f159255177-original-pull-secret podName:24d6b216-f67b-4d45-bd22-c9f159255177 nodeName:}" failed. No retries permitted until 2026-07-24 17:10:35.843740521 +0000 UTC m=+3.045789776 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/24d6b216-f67b-4d45-bd22-c9f159255177-original-pull-secret") pod "global-pull-secret-syncer-v42pr" (UID: "24d6b216-f67b-4d45-bd22-c9f159255177") : object "kube-system"/"original-pull-secret" not registered
Jul 24 17:10:35.344677 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343834    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9f393f69-5a28-41ab-a160-42147718f3da-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-px6r8\" (UID: \"9f393f69-5a28-41ab-a160-42147718f3da\") " pod="openshift-multus/multus-additional-cni-plugins-px6r8"
Jul 24 17:10:35.344677 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343847    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9417588a-ebcd-4fe0-a050-2345106ba8f3-etc-sysctl-conf\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.344677 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343897    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-host-run-netns\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.344677 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343918    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70-konnectivity-ca\") pod \"konnectivity-agent-4c827\" (UID: \"fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70\") " pod="kube-system/konnectivity-agent-4c827"
Jul 24 17:10:35.344677 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343935    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a-device-dir\") pod \"aws-ebs-csi-driver-node-bb9cp\" (UID: \"30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb9cp"
Jul 24 17:10:35.344677 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343953    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-env-overrides\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.344677 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343956    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-host-run-netns\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.345555 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343967    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9417588a-ebcd-4fe0-a050-2345106ba8f3-lib-modules\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.345555 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343913    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a-socket-dir\") pod \"aws-ebs-csi-driver-node-bb9cp\" (UID: \"30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb9cp"
Jul 24 17:10:35.345555 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.343983    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2071cf36-c0fa-4f24-bb3c-bd15dd2896d6-host-slash\") pod \"iptables-alerter-df5fv\" (UID: \"2071cf36-c0fa-4f24-bb3c-bd15dd2896d6\") " pod="openshift-network-operator/iptables-alerter-df5fv"
Jul 24 17:10:35.345555 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344018    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a-device-dir\") pod \"aws-ebs-csi-driver-node-bb9cp\" (UID: \"30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb9cp"
Jul 24 17:10:35.345555 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344121    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9417588a-ebcd-4fe0-a050-2345106ba8f3-lib-modules\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.345555 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344121    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-etc-kubernetes\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.345555 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344146    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-host-slash\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.345555 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344165    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-etc-kubernetes\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.345555 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344162    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.345555 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344208    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgmlq\" (UniqueName: \"kubernetes.io/projected/ccfd714f-9f15-40f6-a06c-6054adfb9992-kube-api-access-fgmlq\") pod \"node-resolver-hpxzm\" (UID: \"ccfd714f-9f15-40f6-a06c-6054adfb9992\") " pod="openshift-dns/node-resolver-hpxzm"
Jul 24 17:10:35.345555 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344219    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2071cf36-c0fa-4f24-bb3c-bd15dd2896d6-host-slash\") pod \"iptables-alerter-df5fv\" (UID: \"2071cf36-c0fa-4f24-bb3c-bd15dd2896d6\") " pod="openshift-network-operator/iptables-alerter-df5fv"
Jul 24 17:10:35.345555 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344224    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9417588a-ebcd-4fe0-a050-2345106ba8f3-etc-kubernetes\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.345555 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344248    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9417588a-ebcd-4fe0-a050-2345106ba8f3-etc-kubernetes\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.345555 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344260    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9417588a-ebcd-4fe0-a050-2345106ba8f3-etc-systemd\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.345555 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344288    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9f393f69-5a28-41ab-a160-42147718f3da-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-px6r8\" (UID: \"9f393f69-5a28-41ab-a160-42147718f3da\") " pod="openshift-multus/multus-additional-cni-plugins-px6r8"
Jul 24 17:10:35.345555 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344296    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9417588a-ebcd-4fe0-a050-2345106ba8f3-etc-systemd\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.345555 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344317    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdfgk\" (UniqueName: \"kubernetes.io/projected/30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a-kube-api-access-bdfgk\") pod \"aws-ebs-csi-driver-node-bb9cp\" (UID: \"30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb9cp"
Jul 24 17:10:35.346315 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344343    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-run-ovn\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.346315 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344371    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4j6q\" (UniqueName: \"kubernetes.io/projected/bcf18baa-0b35-4089-9570-df68ef7a456a-kube-api-access-d4j6q\") pod \"network-check-target-t2znt\" (UID: \"bcf18baa-0b35-4089-9570-df68ef7a456a\") " pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:10:35.346315 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344388    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9f393f69-5a28-41ab-a160-42147718f3da-cni-binary-copy\") pod \"multus-additional-cni-plugins-px6r8\" (UID: \"9f393f69-5a28-41ab-a160-42147718f3da\") " pod="openshift-multus/multus-additional-cni-plugins-px6r8"
Jul 24 17:10:35.346315 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344398    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c99646a-9e3e-4e60-a750-a96ab5aab0bf-metrics-certs\") pod \"network-metrics-daemon-d42hh\" (UID: \"5c99646a-9e3e-4e60-a750-a96ab5aab0bf\") " pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:10:35.346315 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344428    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2071cf36-c0fa-4f24-bb3c-bd15dd2896d6-iptables-alerter-script\") pod \"iptables-alerter-df5fv\" (UID: \"2071cf36-c0fa-4f24-bb3c-bd15dd2896d6\") " pod="openshift-network-operator/iptables-alerter-df5fv"
Jul 24 17:10:35.346315 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344453    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-multus-cni-dir\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.346315 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344481    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9f393f69-5a28-41ab-a160-42147718f3da-system-cni-dir\") pod \"multus-additional-cni-plugins-px6r8\" (UID: \"9f393f69-5a28-41ab-a160-42147718f3da\") " pod="openshift-multus/multus-additional-cni-plugins-px6r8"
Jul 24 17:10:35.346315 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344508    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65tc7\" (UniqueName: \"kubernetes.io/projected/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-kube-api-access-65tc7\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.346315 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344523    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70-konnectivity-ca\") pod \"konnectivity-agent-4c827\" (UID: \"fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70\") " pod="kube-system/konnectivity-agent-4c827"
Jul 24 17:10:35.346315 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344546    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9417588a-ebcd-4fe0-a050-2345106ba8f3-sys\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.346315 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344577    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kls44\" (UniqueName: \"kubernetes.io/projected/5c99646a-9e3e-4e60-a750-a96ab5aab0bf-kube-api-access-kls44\") pod \"network-metrics-daemon-d42hh\" (UID: \"5c99646a-9e3e-4e60-a750-a96ab5aab0bf\") " pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:10:35.346315 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:35.344613    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:10:35.346315 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344632    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9f393f69-5a28-41ab-a160-42147718f3da-system-cni-dir\") pod \"multus-additional-cni-plugins-px6r8\" (UID: \"9f393f69-5a28-41ab-a160-42147718f3da\") " pod="openshift-multus/multus-additional-cni-plugins-px6r8"
Jul 24 17:10:35.346315 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:35.344659    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c99646a-9e3e-4e60-a750-a96ab5aab0bf-metrics-certs podName:5c99646a-9e3e-4e60-a750-a96ab5aab0bf nodeName:}" failed. No retries permitted until 2026-07-24 17:10:35.844642137 +0000 UTC m=+3.046691393 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5c99646a-9e3e-4e60-a750-a96ab5aab0bf-metrics-certs") pod "network-metrics-daemon-d42hh" (UID: "5c99646a-9e3e-4e60-a750-a96ab5aab0bf") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:10:35.346315 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344679    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-multus-socket-dir-parent\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.346315 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344707    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-hostroot\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.346315 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344727    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-multus-cni-dir\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.347083 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344731    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/24d6b216-f67b-4d45-bd22-c9f159255177-dbus\") pod \"global-pull-secret-syncer-v42pr\" (UID: \"24d6b216-f67b-4d45-bd22-c9f159255177\") " pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:10:35.347083 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344724    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9f393f69-5a28-41ab-a160-42147718f3da-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-px6r8\" (UID: \"9f393f69-5a28-41ab-a160-42147718f3da\") " pod="openshift-multus/multus-additional-cni-plugins-px6r8"
Jul 24 17:10:35.347083 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344766    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-multus-socket-dir-parent\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.347083 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344777    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-hostroot\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.347083 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344787    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-run-systemd\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.347083 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344808    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-host-run-ovn-kubernetes\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.347083 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344823    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9417588a-ebcd-4fe0-a050-2345106ba8f3-sys\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.347083 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344831    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9417588a-ebcd-4fe0-a050-2345106ba8f3-run\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.347083 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344855    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9417588a-ebcd-4fe0-a050-2345106ba8f3-run\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.347083 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344862    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9417588a-ebcd-4fe0-a050-2345106ba8f3-tmp\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.347083 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344886    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/24d6b216-f67b-4d45-bd22-c9f159255177-dbus\") pod \"global-pull-secret-syncer-v42pr\" (UID: \"24d6b216-f67b-4d45-bd22-c9f159255177\") " pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:10:35.347083 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344889    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/24d6b216-f67b-4d45-bd22-c9f159255177-kubelet-config\") pod \"global-pull-secret-syncer-v42pr\" (UID: \"24d6b216-f67b-4d45-bd22-c9f159255177\") " pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:10:35.347083 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344917    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bb9cp\" (UID: \"30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb9cp"
Jul 24 17:10:35.347083 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344943    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-etc-openvswitch\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.347083 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344948    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/24d6b216-f67b-4d45-bd22-c9f159255177-kubelet-config\") pod \"global-pull-secret-syncer-v42pr\" (UID: \"24d6b216-f67b-4d45-bd22-c9f159255177\") " pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:10:35.347083 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344983    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9417588a-ebcd-4fe0-a050-2345106ba8f3-etc-sysctl-d\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.347083 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.344989    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2071cf36-c0fa-4f24-bb3c-bd15dd2896d6-iptables-alerter-script\") pod \"iptables-alerter-df5fv\" (UID: \"2071cf36-c0fa-4f24-bb3c-bd15dd2896d6\") " pod="openshift-network-operator/iptables-alerter-df5fv"
Jul 24 17:10:35.347820 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.345006    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bb9cp\" (UID: \"30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb9cp"
Jul 24 17:10:35.347820 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.345007    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-multus-conf-dir\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.347820 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.345061    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9f393f69-5a28-41ab-a160-42147718f3da-os-release\") pod \"multus-additional-cni-plugins-px6r8\" (UID: \"9f393f69-5a28-41ab-a160-42147718f3da\") " pod="openshift-multus/multus-additional-cni-plugins-px6r8"
Jul 24 17:10:35.347820 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.345079    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-node-log\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.347820 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.345109    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9417588a-ebcd-4fe0-a050-2345106ba8f3-etc-modprobe-d\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.347820 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.345111    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9417588a-ebcd-4fe0-a050-2345106ba8f3-etc-sysctl-d\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.347820 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.345061    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/32f3d102-a6d1-41af-8410-42dc28eb6975-multus-conf-dir\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.347820 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.345130    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/32f3d102-a6d1-41af-8410-42dc28eb6975-cni-binary-copy\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.347820 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.345155    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/32f3d102-a6d1-41af-8410-42dc28eb6975-multus-daemon-config\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.347820 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.345180    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9f393f69-5a28-41ab-a160-42147718f3da-tuning-conf-dir\") pod \"multus-additional-cni-plugins-px6r8\" (UID: \"9f393f69-5a28-41ab-a160-42147718f3da\") " pod="openshift-multus/multus-additional-cni-plugins-px6r8"
Jul 24 17:10:35.347820 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.345184    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9417588a-ebcd-4fe0-a050-2345106ba8f3-etc-modprobe-d\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.347820 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.345205    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/17ed6b09-72e1-4161-9a92-cb481241bfe1-serviceca\") pod \"node-ca-hv6sp\" (UID: \"17ed6b09-72e1-4161-9a92-cb481241bfe1\") " pod="openshift-image-registry/node-ca-hv6sp"
Jul 24 17:10:35.347820 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.345233    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a-registration-dir\") pod \"aws-ebs-csi-driver-node-bb9cp\" (UID: \"30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb9cp"
Jul 24 17:10:35.347820 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.345260    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-host-cni-bin\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.347820 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.345197    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9f393f69-5a28-41ab-a160-42147718f3da-os-release\") pod \"multus-additional-cni-plugins-px6r8\" (UID: \"9f393f69-5a28-41ab-a160-42147718f3da\") " pod="openshift-multus/multus-additional-cni-plugins-px6r8"
Jul 24 17:10:35.347820 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.345322    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9f393f69-5a28-41ab-a160-42147718f3da-tuning-conf-dir\") pod \"multus-additional-cni-plugins-px6r8\" (UID: \"9f393f69-5a28-41ab-a160-42147718f3da\") " pod="openshift-multus/multus-additional-cni-plugins-px6r8"
Jul 24 17:10:35.347820 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.345346    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a-registration-dir\") pod \"aws-ebs-csi-driver-node-bb9cp\" (UID: \"30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb9cp"
Jul 24 17:10:35.348683 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.345652    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/17ed6b09-72e1-4161-9a92-cb481241bfe1-serviceca\") pod \"node-ca-hv6sp\" (UID: \"17ed6b09-72e1-4161-9a92-cb481241bfe1\") " pod="openshift-image-registry/node-ca-hv6sp"
Jul 24 17:10:35.348683 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.345669    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/32f3d102-a6d1-41af-8410-42dc28eb6975-cni-binary-copy\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.348683 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.345896    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/32f3d102-a6d1-41af-8410-42dc28eb6975-multus-daemon-config\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.348683 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.347361    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70-agent-certs\") pod \"konnectivity-agent-4c827\" (UID: \"fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70\") " pod="kube-system/konnectivity-agent-4c827"
Jul 24 17:10:35.348683 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.348133    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9417588a-ebcd-4fe0-a050-2345106ba8f3-tmp\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.348683 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.348172    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9417588a-ebcd-4fe0-a050-2345106ba8f3-etc-tuned\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.351240 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:35.351169    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 17:10:35.351240 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:35.351190    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 17:10:35.351240 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:35.351202    2581 projected.go:196] Error preparing data for projected volume kube-api-access-d4j6q for pod openshift-network-diagnostics/network-check-target-t2znt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:10:35.351447 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:35.351270    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcf18baa-0b35-4089-9570-df68ef7a456a-kube-api-access-d4j6q podName:bcf18baa-0b35-4089-9570-df68ef7a456a nodeName:}" failed. No retries permitted until 2026-07-24 17:10:35.851254821 +0000 UTC m=+3.053304073 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-d4j6q" (UniqueName: "kubernetes.io/projected/bcf18baa-0b35-4089-9570-df68ef7a456a-kube-api-access-d4j6q") pod "network-check-target-t2znt" (UID: "bcf18baa-0b35-4089-9570-df68ef7a456a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:10:35.353001 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.352980    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2jbc\" (UniqueName: \"kubernetes.io/projected/9417588a-ebcd-4fe0-a050-2345106ba8f3-kube-api-access-m2jbc\") pod \"tuned-zdv5f\" (UID: \"9417588a-ebcd-4fe0-a050-2345106ba8f3\") " pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.355977 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.355955    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdfgk\" (UniqueName: \"kubernetes.io/projected/30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a-kube-api-access-bdfgk\") pod \"aws-ebs-csi-driver-node-bb9cp\" (UID: \"30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb9cp"
Jul 24 17:10:35.356348 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.356329    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndkct\" (UniqueName: \"kubernetes.io/projected/2071cf36-c0fa-4f24-bb3c-bd15dd2896d6-kube-api-access-ndkct\") pod \"iptables-alerter-df5fv\" (UID: \"2071cf36-c0fa-4f24-bb3c-bd15dd2896d6\") " pod="openshift-network-operator/iptables-alerter-df5fv"
Jul 24 17:10:35.356423 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.356378    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g97cz\" (UniqueName: \"kubernetes.io/projected/9f393f69-5a28-41ab-a160-42147718f3da-kube-api-access-g97cz\") pod \"multus-additional-cni-plugins-px6r8\" (UID: \"9f393f69-5a28-41ab-a160-42147718f3da\") " pod="openshift-multus/multus-additional-cni-plugins-px6r8"
Jul 24 17:10:35.356505 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.356480    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jr8t8\" (UniqueName: \"kubernetes.io/projected/17ed6b09-72e1-4161-9a92-cb481241bfe1-kube-api-access-jr8t8\") pod \"node-ca-hv6sp\" (UID: \"17ed6b09-72e1-4161-9a92-cb481241bfe1\") " pod="openshift-image-registry/node-ca-hv6sp"
Jul 24 17:10:35.357199 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.357129    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gvk2\" (UniqueName: \"kubernetes.io/projected/32f3d102-a6d1-41af-8410-42dc28eb6975-kube-api-access-8gvk2\") pod \"multus-22jtw\" (UID: \"32f3d102-a6d1-41af-8410-42dc28eb6975\") " pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.357854 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.357837    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kls44\" (UniqueName: \"kubernetes.io/projected/5c99646a-9e3e-4e60-a750-a96ab5aab0bf-kube-api-access-kls44\") pod \"network-metrics-daemon-d42hh\" (UID: \"5c99646a-9e3e-4e60-a750-a96ab5aab0bf\") " pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:10:35.446869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.445934    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-run-ovn\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.446869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446001    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65tc7\" (UniqueName: \"kubernetes.io/projected/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-kube-api-access-65tc7\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.446869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446031    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-run-systemd\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.446869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446072    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-host-run-ovn-kubernetes\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.446869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446071    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-run-ovn\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.446869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446098    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-etc-openvswitch\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.446869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446124    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-node-log\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.446869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446133    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-run-systemd\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.446869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446151    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-host-cni-bin\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.446869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446174    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-etc-openvswitch\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.446869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446192    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-host-cni-bin\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.446869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446138    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-host-run-ovn-kubernetes\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.446869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446194    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-host-run-netns\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.446869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446212    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-node-log\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.446869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446236    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-host-run-netns\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.446869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446236    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-var-lib-openvswitch\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.446869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446265    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-var-lib-openvswitch\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.447854 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446268    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-run-openvswitch\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.447854 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446293    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-host-cni-netd\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.447854 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446319    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-ovnkube-config\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.447854 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446341    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-ovn-node-metrics-cert\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.447854 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446367    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ccfd714f-9f15-40f6-a06c-6054adfb9992-hosts-file\") pod \"node-resolver-hpxzm\" (UID: \"ccfd714f-9f15-40f6-a06c-6054adfb9992\") " pod="openshift-dns/node-resolver-hpxzm"
Jul 24 17:10:35.447854 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446375    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-host-cni-netd\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.447854 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446388    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-run-openvswitch\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.447854 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446421    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ccfd714f-9f15-40f6-a06c-6054adfb9992-tmp-dir\") pod \"node-resolver-hpxzm\" (UID: \"ccfd714f-9f15-40f6-a06c-6054adfb9992\") " pod="openshift-dns/node-resolver-hpxzm"
Jul 24 17:10:35.447854 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446426    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ccfd714f-9f15-40f6-a06c-6054adfb9992-hosts-file\") pod \"node-resolver-hpxzm\" (UID: \"ccfd714f-9f15-40f6-a06c-6054adfb9992\") " pod="openshift-dns/node-resolver-hpxzm"
Jul 24 17:10:35.447854 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446450    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-systemd-units\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.447854 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446474    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-log-socket\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.447854 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446521    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-ovnkube-script-lib\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.447854 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446555    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-host-kubelet\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.447854 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446568    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-systemd-units\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.447854 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446582    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-env-overrides\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.447854 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446609    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-host-slash\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.447854 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446635    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.448668 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446659    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgmlq\" (UniqueName: \"kubernetes.io/projected/ccfd714f-9f15-40f6-a06c-6054adfb9992-kube-api-access-fgmlq\") pod \"node-resolver-hpxzm\" (UID: \"ccfd714f-9f15-40f6-a06c-6054adfb9992\") " pod="openshift-dns/node-resolver-hpxzm"
Jul 24 17:10:35.448668 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446732    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ccfd714f-9f15-40f6-a06c-6054adfb9992-tmp-dir\") pod \"node-resolver-hpxzm\" (UID: \"ccfd714f-9f15-40f6-a06c-6054adfb9992\") " pod="openshift-dns/node-resolver-hpxzm"
Jul 24 17:10:35.448668 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446794    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-host-slash\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.448668 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446800    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-host-kubelet\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.448668 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446833    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.448668 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446851    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-log-socket\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.448668 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.446929    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-ovnkube-config\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.448668 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.447022    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-ovnkube-script-lib\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.448668 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.447190    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-env-overrides\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.449464 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.449446    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-ovn-node-metrics-cert\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.455265 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.455204    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65tc7\" (UniqueName: \"kubernetes.io/projected/c002a7e5-2e5d-4d1d-bc44-0e5b3491073c-kube-api-access-65tc7\") pod \"ovnkube-node-9lnl2\" (UID: \"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.455357 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.455291    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgmlq\" (UniqueName: \"kubernetes.io/projected/ccfd714f-9f15-40f6-a06c-6054adfb9992-kube-api-access-fgmlq\") pod \"node-resolver-hpxzm\" (UID: \"ccfd714f-9f15-40f6-a06c-6054adfb9992\") " pod="openshift-dns/node-resolver-hpxzm"
Jul 24 17:10:35.540458 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.540428    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zdv5f"
Jul 24 17:10:35.555512 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.555488    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hv6sp"
Jul 24 17:10:35.567448 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.567425    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-df5fv"
Jul 24 17:10:35.572215 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.572198    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4c827"
Jul 24 17:10:35.581090 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.581073    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-px6r8"
Jul 24 17:10:35.587935 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.587915    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb9cp"
Jul 24 17:10:35.595883 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.595865    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:10:35.603808 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.603788    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-22jtw"
Jul 24 17:10:35.609501 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.609483    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hpxzm"
Jul 24 17:10:35.638894 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.638863    2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 17:10:35.683529 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.683495    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 17:10:35.850197 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.850120    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/24d6b216-f67b-4d45-bd22-c9f159255177-original-pull-secret\") pod \"global-pull-secret-syncer-v42pr\" (UID: \"24d6b216-f67b-4d45-bd22-c9f159255177\") " pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:10:35.850197 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.850183    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c99646a-9e3e-4e60-a750-a96ab5aab0bf-metrics-certs\") pod \"network-metrics-daemon-d42hh\" (UID: \"5c99646a-9e3e-4e60-a750-a96ab5aab0bf\") " pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:10:35.850386 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:35.850293    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 17:10:35.850386 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:35.850344    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24d6b216-f67b-4d45-bd22-c9f159255177-original-pull-secret podName:24d6b216-f67b-4d45-bd22-c9f159255177 nodeName:}" failed. No retries permitted until 2026-07-24 17:10:36.850329715 +0000 UTC m=+4.052378966 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/24d6b216-f67b-4d45-bd22-c9f159255177-original-pull-secret") pod "global-pull-secret-syncer-v42pr" (UID: "24d6b216-f67b-4d45-bd22-c9f159255177") : object "kube-system"/"original-pull-secret" not registered
Jul 24 17:10:35.850386 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:35.850293    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:10:35.850501 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:35.850438    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c99646a-9e3e-4e60-a750-a96ab5aab0bf-metrics-certs podName:5c99646a-9e3e-4e60-a750-a96ab5aab0bf nodeName:}" failed. No retries permitted until 2026-07-24 17:10:36.850419385 +0000 UTC m=+4.052468640 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5c99646a-9e3e-4e60-a750-a96ab5aab0bf-metrics-certs") pod "network-metrics-daemon-d42hh" (UID: "5c99646a-9e3e-4e60-a750-a96ab5aab0bf") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:10:35.951268 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:35.951232    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4j6q\" (UniqueName: \"kubernetes.io/projected/bcf18baa-0b35-4089-9570-df68ef7a456a-kube-api-access-d4j6q\") pod \"network-check-target-t2znt\" (UID: \"bcf18baa-0b35-4089-9570-df68ef7a456a\") " pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:10:35.951421 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:35.951385    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 17:10:35.951421 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:35.951406    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 17:10:35.951421 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:35.951419    2581 projected.go:196] Error preparing data for projected volume kube-api-access-d4j6q for pod openshift-network-diagnostics/network-check-target-t2znt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:10:35.951549 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:35.951483    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcf18baa-0b35-4089-9570-df68ef7a456a-kube-api-access-d4j6q podName:bcf18baa-0b35-4089-9570-df68ef7a456a nodeName:}" failed. No retries permitted until 2026-07-24 17:10:36.951462766 +0000 UTC m=+4.153512042 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-d4j6q" (UniqueName: "kubernetes.io/projected/bcf18baa-0b35-4089-9570-df68ef7a456a-kube-api-access-d4j6q") pod "network-check-target-t2znt" (UID: "bcf18baa-0b35-4089-9570-df68ef7a456a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:10:36.267333 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:36.267255    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 17:05:35 +0000 UTC" deadline="2027-12-31 19:24:04.782564756 +0000 UTC"
Jul 24 17:10:36.267333 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:36.267286    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12602h13m28.51528144s"
Jul 24 17:10:36.378013 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:36.377987    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:10:36.378185 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:36.378112    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d42hh" podUID="5c99646a-9e3e-4e60-a750-a96ab5aab0bf"
Jul 24 17:10:36.378185 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:36.378149    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:10:36.378273 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:36.378225    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t2znt" podUID="bcf18baa-0b35-4089-9570-df68ef7a456a"
Jul 24 17:10:36.432776 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:36.432752    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe4d8ffb_2d02_410f_bd2c_7bd1eb961c70.slice/crio-5ebf3ad31fe5ed3024db21a7269a2bbab423d33ca90e496e7539c911bd42557d WatchSource:0}: Error finding container 5ebf3ad31fe5ed3024db21a7269a2bbab423d33ca90e496e7539c911bd42557d: Status 404 returned error can't find the container with id 5ebf3ad31fe5ed3024db21a7269a2bbab423d33ca90e496e7539c911bd42557d
Jul 24 17:10:36.435859 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:36.435826    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9417588a_ebcd_4fe0_a050_2345106ba8f3.slice/crio-7d08ce70fb226bfdaa04746a9a697d525b245b0dec27c7ac47bc1752244ad646 WatchSource:0}: Error finding container 7d08ce70fb226bfdaa04746a9a697d525b245b0dec27c7ac47bc1752244ad646: Status 404 returned error can't find the container with id 7d08ce70fb226bfdaa04746a9a697d525b245b0dec27c7ac47bc1752244ad646
Jul 24 17:10:36.436827 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:36.436627    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30f8f5cd_dbe7_4d1c_a65b_0c472a7b7c6a.slice/crio-0ff189ba180d1b4a75a024d8b7f325eb54199efd5077c00754d1ed22af27ec85 WatchSource:0}: Error finding container 0ff189ba180d1b4a75a024d8b7f325eb54199efd5077c00754d1ed22af27ec85: Status 404 returned error can't find the container with id 0ff189ba180d1b4a75a024d8b7f325eb54199efd5077c00754d1ed22af27ec85
Jul 24 17:10:36.440345 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:36.440312    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc002a7e5_2e5d_4d1d_bc44_0e5b3491073c.slice/crio-d959f9e744e7d3c86d15a9652e39cfc727edd2299fbdbb4fef98eabb0af6d545 WatchSource:0}: Error finding container d959f9e744e7d3c86d15a9652e39cfc727edd2299fbdbb4fef98eabb0af6d545: Status 404 returned error can't find the container with id d959f9e744e7d3c86d15a9652e39cfc727edd2299fbdbb4fef98eabb0af6d545
Jul 24 17:10:36.443188 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:36.442750    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f393f69_5a28_41ab_a160_42147718f3da.slice/crio-34f73fd94a308a1ad4d7521a6825979d28421befd3321b14f71defbbfd18518c WatchSource:0}: Error finding container 34f73fd94a308a1ad4d7521a6825979d28421befd3321b14f71defbbfd18518c: Status 404 returned error can't find the container with id 34f73fd94a308a1ad4d7521a6825979d28421befd3321b14f71defbbfd18518c
Jul 24 17:10:36.443472 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:36.443449    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17ed6b09_72e1_4161_9a92_cb481241bfe1.slice/crio-5b2b0d7127d525b9320da6f71b60d020c746fdee730b4b1cb301414544048e32 WatchSource:0}: Error finding container 5b2b0d7127d525b9320da6f71b60d020c746fdee730b4b1cb301414544048e32: Status 404 returned error can't find the container with id 5b2b0d7127d525b9320da6f71b60d020c746fdee730b4b1cb301414544048e32
Jul 24 17:10:36.444359 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:36.444337    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2071cf36_c0fa_4f24_bb3c_bd15dd2896d6.slice/crio-b2426c800b62eaa2d58455f71504e8d7be2dc1a4779df698c3ffd4522fbc1419 WatchSource:0}: Error finding container b2426c800b62eaa2d58455f71504e8d7be2dc1a4779df698c3ffd4522fbc1419: Status 404 returned error can't find the container with id b2426c800b62eaa2d58455f71504e8d7be2dc1a4779df698c3ffd4522fbc1419
Jul 24 17:10:36.446681 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:36.446658    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccfd714f_9f15_40f6_a06c_6054adfb9992.slice/crio-697ab2f91008edfc6370caa9b809d03a6538db1488be368d516b9dc92e3b7115 WatchSource:0}: Error finding container 697ab2f91008edfc6370caa9b809d03a6538db1488be368d516b9dc92e3b7115: Status 404 returned error can't find the container with id 697ab2f91008edfc6370caa9b809d03a6538db1488be368d516b9dc92e3b7115
Jul 24 17:10:36.448146 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:10:36.448009    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32f3d102_a6d1_41af_8410_42dc28eb6975.slice/crio-b644ed4595440be1482bd8431e3f833d3c17c3e897b1c8fa3b56512c5f6f9ffb WatchSource:0}: Error finding container b644ed4595440be1482bd8431e3f833d3c17c3e897b1c8fa3b56512c5f6f9ffb: Status 404 returned error can't find the container with id b644ed4595440be1482bd8431e3f833d3c17c3e897b1c8fa3b56512c5f6f9ffb
Jul 24 17:10:36.858427 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:36.857794    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c99646a-9e3e-4e60-a750-a96ab5aab0bf-metrics-certs\") pod \"network-metrics-daemon-d42hh\" (UID: \"5c99646a-9e3e-4e60-a750-a96ab5aab0bf\") " pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:10:36.858427 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:36.858093    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/24d6b216-f67b-4d45-bd22-c9f159255177-original-pull-secret\") pod \"global-pull-secret-syncer-v42pr\" (UID: \"24d6b216-f67b-4d45-bd22-c9f159255177\") " pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:10:36.858427 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:36.857965    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:10:36.858427 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:36.858251    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c99646a-9e3e-4e60-a750-a96ab5aab0bf-metrics-certs podName:5c99646a-9e3e-4e60-a750-a96ab5aab0bf nodeName:}" failed. No retries permitted until 2026-07-24 17:10:38.858237433 +0000 UTC m=+6.060286685 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5c99646a-9e3e-4e60-a750-a96ab5aab0bf-metrics-certs") pod "network-metrics-daemon-d42hh" (UID: "5c99646a-9e3e-4e60-a750-a96ab5aab0bf") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:10:36.858427 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:36.858204    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 17:10:36.858427 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:36.858351    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24d6b216-f67b-4d45-bd22-c9f159255177-original-pull-secret podName:24d6b216-f67b-4d45-bd22-c9f159255177 nodeName:}" failed. No retries permitted until 2026-07-24 17:10:38.858330675 +0000 UTC m=+6.060379932 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/24d6b216-f67b-4d45-bd22-c9f159255177-original-pull-secret") pod "global-pull-secret-syncer-v42pr" (UID: "24d6b216-f67b-4d45-bd22-c9f159255177") : object "kube-system"/"original-pull-secret" not registered
Jul 24 17:10:36.959432 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:36.959282    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4j6q\" (UniqueName: \"kubernetes.io/projected/bcf18baa-0b35-4089-9570-df68ef7a456a-kube-api-access-d4j6q\") pod \"network-check-target-t2znt\" (UID: \"bcf18baa-0b35-4089-9570-df68ef7a456a\") " pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:10:36.959600 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:36.959481    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 17:10:36.959600 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:36.959505    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 17:10:36.959600 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:36.959518    2581 projected.go:196] Error preparing data for projected volume kube-api-access-d4j6q for pod openshift-network-diagnostics/network-check-target-t2znt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:10:36.959600 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:36.959579    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcf18baa-0b35-4089-9570-df68ef7a456a-kube-api-access-d4j6q podName:bcf18baa-0b35-4089-9570-df68ef7a456a nodeName:}" failed. No retries permitted until 2026-07-24 17:10:38.959558274 +0000 UTC m=+6.161607531 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-d4j6q" (UniqueName: "kubernetes.io/projected/bcf18baa-0b35-4089-9570-df68ef7a456a-kube-api-access-d4j6q") pod "network-check-target-t2znt" (UID: "bcf18baa-0b35-4089-9570-df68ef7a456a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:10:37.268257 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:37.268119    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 17:05:35 +0000 UTC" deadline="2028-02-04 17:59:27.23130702 +0000 UTC"
Jul 24 17:10:37.268257 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:37.268180    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13440h48m49.963130801s"
Jul 24 17:10:37.381254 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:37.380716    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:10:37.381254 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:37.380846    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v42pr" podUID="24d6b216-f67b-4d45-bd22-c9f159255177"
Jul 24 17:10:37.392001 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:37.391902    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-df5fv" event={"ID":"2071cf36-c0fa-4f24-bb3c-bd15dd2896d6","Type":"ContainerStarted","Data":"b2426c800b62eaa2d58455f71504e8d7be2dc1a4779df698c3ffd4522fbc1419"}
Jul 24 17:10:37.399263 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:37.399192    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hv6sp" event={"ID":"17ed6b09-72e1-4161-9a92-cb481241bfe1","Type":"ContainerStarted","Data":"5b2b0d7127d525b9320da6f71b60d020c746fdee730b4b1cb301414544048e32"}
Jul 24 17:10:37.408240 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:37.408142    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2" event={"ID":"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c","Type":"ContainerStarted","Data":"d959f9e744e7d3c86d15a9652e39cfc727edd2299fbdbb4fef98eabb0af6d545"}
Jul 24 17:10:37.410379 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:37.410349    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zdv5f" event={"ID":"9417588a-ebcd-4fe0-a050-2345106ba8f3","Type":"ContainerStarted","Data":"7d08ce70fb226bfdaa04746a9a697d525b245b0dec27c7ac47bc1752244ad646"}
Jul 24 17:10:37.412825 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:37.412798    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4c827" event={"ID":"fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70","Type":"ContainerStarted","Data":"5ebf3ad31fe5ed3024db21a7269a2bbab423d33ca90e496e7539c911bd42557d"}
Jul 24 17:10:37.420178 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:37.420139    2581 generic.go:356] "Generic (PLEG): container finished" podID="c23660c78fea953db7eac9d97b433c18" containerID="e46f116ebdc9eacec937ab8096053d0452013e8b5a5691168c5a003b7845e441" exitCode=0
Jul 24 17:10:37.420423 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:37.420400    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.ec2.internal" event={"ID":"c23660c78fea953db7eac9d97b433c18","Type":"ContainerDied","Data":"e46f116ebdc9eacec937ab8096053d0452013e8b5a5691168c5a003b7845e441"}
Jul 24 17:10:37.432465 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:37.432439    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-46.ec2.internal" event={"ID":"39f953389fbb7b355964e9b9fe5538ef","Type":"ContainerStarted","Data":"351859c554cbe06fce4f778ae6471e5c5a2f7d873a3f33c14c98a54bcf04e08d"}
Jul 24 17:10:37.450054 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:37.449840    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-22jtw" event={"ID":"32f3d102-a6d1-41af-8410-42dc28eb6975","Type":"ContainerStarted","Data":"b644ed4595440be1482bd8431e3f833d3c17c3e897b1c8fa3b56512c5f6f9ffb"}
Jul 24 17:10:37.450790 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:37.450738    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-46.ec2.internal" podStartSLOduration=3.450717846 podStartE2EDuration="3.450717846s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:10:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:10:37.449870915 +0000 UTC m=+4.651920191" watchObservedRunningTime="2026-07-24 17:10:37.450717846 +0000 UTC m=+4.652767121"
Jul 24 17:10:37.457835 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:37.457808    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hpxzm" event={"ID":"ccfd714f-9f15-40f6-a06c-6054adfb9992","Type":"ContainerStarted","Data":"697ab2f91008edfc6370caa9b809d03a6538db1488be368d516b9dc92e3b7115"}
Jul 24 17:10:37.466932 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:37.466902    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-px6r8" event={"ID":"9f393f69-5a28-41ab-a160-42147718f3da","Type":"ContainerStarted","Data":"34f73fd94a308a1ad4d7521a6825979d28421befd3321b14f71defbbfd18518c"}
Jul 24 17:10:37.491218 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:37.491117    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb9cp" event={"ID":"30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a","Type":"ContainerStarted","Data":"0ff189ba180d1b4a75a024d8b7f325eb54199efd5077c00754d1ed22af27ec85"}
Jul 24 17:10:38.378175 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:38.378141    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:10:38.378733 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:38.378319    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d42hh" podUID="5c99646a-9e3e-4e60-a750-a96ab5aab0bf"
Jul 24 17:10:38.378733 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:38.378400    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:10:38.378733 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:38.378468    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t2znt" podUID="bcf18baa-0b35-4089-9570-df68ef7a456a"
Jul 24 17:10:38.505171 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:38.505094    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.ec2.internal" event={"ID":"c23660c78fea953db7eac9d97b433c18","Type":"ContainerStarted","Data":"658909d9486ac83b4e025e43dbb01a1c8f9424ee7fb9c887a1123989e2bd4f7e"}
Jul 24 17:10:38.523056 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:38.522985    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.ec2.internal" podStartSLOduration=4.522965998 podStartE2EDuration="4.522965998s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:10:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:10:38.522204595 +0000 UTC m=+5.724253871" watchObservedRunningTime="2026-07-24 17:10:38.522965998 +0000 UTC m=+5.725015273"
Jul 24 17:10:38.875301 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:38.875204    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c99646a-9e3e-4e60-a750-a96ab5aab0bf-metrics-certs\") pod \"network-metrics-daemon-d42hh\" (UID: \"5c99646a-9e3e-4e60-a750-a96ab5aab0bf\") " pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:10:38.875468 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:38.875300    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/24d6b216-f67b-4d45-bd22-c9f159255177-original-pull-secret\") pod \"global-pull-secret-syncer-v42pr\" (UID: \"24d6b216-f67b-4d45-bd22-c9f159255177\") " pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:10:38.875468 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:38.875410    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 17:10:38.875468 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:38.875410    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:10:38.875636 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:38.875474    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24d6b216-f67b-4d45-bd22-c9f159255177-original-pull-secret podName:24d6b216-f67b-4d45-bd22-c9f159255177 nodeName:}" failed. No retries permitted until 2026-07-24 17:10:42.875453877 +0000 UTC m=+10.077503147 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/24d6b216-f67b-4d45-bd22-c9f159255177-original-pull-secret") pod "global-pull-secret-syncer-v42pr" (UID: "24d6b216-f67b-4d45-bd22-c9f159255177") : object "kube-system"/"original-pull-secret" not registered
Jul 24 17:10:38.875636 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:38.875495    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c99646a-9e3e-4e60-a750-a96ab5aab0bf-metrics-certs podName:5c99646a-9e3e-4e60-a750-a96ab5aab0bf nodeName:}" failed. No retries permitted until 2026-07-24 17:10:42.875485364 +0000 UTC m=+10.077534622 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5c99646a-9e3e-4e60-a750-a96ab5aab0bf-metrics-certs") pod "network-metrics-daemon-d42hh" (UID: "5c99646a-9e3e-4e60-a750-a96ab5aab0bf") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:10:38.975849 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:38.975811    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4j6q\" (UniqueName: \"kubernetes.io/projected/bcf18baa-0b35-4089-9570-df68ef7a456a-kube-api-access-d4j6q\") pod \"network-check-target-t2znt\" (UID: \"bcf18baa-0b35-4089-9570-df68ef7a456a\") " pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:10:38.976068 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:38.976032    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 17:10:38.976163 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:38.976079    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 17:10:38.976163 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:38.976094    2581 projected.go:196] Error preparing data for projected volume kube-api-access-d4j6q for pod openshift-network-diagnostics/network-check-target-t2znt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:10:38.976163 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:38.976155    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcf18baa-0b35-4089-9570-df68ef7a456a-kube-api-access-d4j6q podName:bcf18baa-0b35-4089-9570-df68ef7a456a nodeName:}" failed. No retries permitted until 2026-07-24 17:10:42.976136553 +0000 UTC m=+10.178185812 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-d4j6q" (UniqueName: "kubernetes.io/projected/bcf18baa-0b35-4089-9570-df68ef7a456a-kube-api-access-d4j6q") pod "network-check-target-t2znt" (UID: "bcf18baa-0b35-4089-9570-df68ef7a456a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:10:39.388176 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:39.388141    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:10:39.388610 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:39.388305    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v42pr" podUID="24d6b216-f67b-4d45-bd22-c9f159255177"
Jul 24 17:10:40.377445 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:40.377412    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:10:40.377654 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:40.377556    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d42hh" podUID="5c99646a-9e3e-4e60-a750-a96ab5aab0bf"
Jul 24 17:10:40.377948 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:40.377928    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:10:40.378063 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:40.378021    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t2znt" podUID="bcf18baa-0b35-4089-9570-df68ef7a456a"
Jul 24 17:10:41.380200 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:41.380122    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:10:41.380629 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:41.380251    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v42pr" podUID="24d6b216-f67b-4d45-bd22-c9f159255177"
Jul 24 17:10:42.378078 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:42.378022    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:10:42.378258 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:42.378189    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d42hh" podUID="5c99646a-9e3e-4e60-a750-a96ab5aab0bf"
Jul 24 17:10:42.378330 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:42.378268    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:10:42.378418 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:42.378379    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t2znt" podUID="bcf18baa-0b35-4089-9570-df68ef7a456a"
Jul 24 17:10:42.921781 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:42.921735    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c99646a-9e3e-4e60-a750-a96ab5aab0bf-metrics-certs\") pod \"network-metrics-daemon-d42hh\" (UID: \"5c99646a-9e3e-4e60-a750-a96ab5aab0bf\") " pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:10:42.922236 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:42.921827    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/24d6b216-f67b-4d45-bd22-c9f159255177-original-pull-secret\") pod \"global-pull-secret-syncer-v42pr\" (UID: \"24d6b216-f67b-4d45-bd22-c9f159255177\") " pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:10:42.922236 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:42.921886    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:10:42.922236 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:42.921953    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c99646a-9e3e-4e60-a750-a96ab5aab0bf-metrics-certs podName:5c99646a-9e3e-4e60-a750-a96ab5aab0bf nodeName:}" failed. No retries permitted until 2026-07-24 17:10:50.921930255 +0000 UTC m=+18.123979512 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5c99646a-9e3e-4e60-a750-a96ab5aab0bf-metrics-certs") pod "network-metrics-daemon-d42hh" (UID: "5c99646a-9e3e-4e60-a750-a96ab5aab0bf") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:10:42.922236 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:42.921955    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 17:10:42.922236 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:42.922010    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24d6b216-f67b-4d45-bd22-c9f159255177-original-pull-secret podName:24d6b216-f67b-4d45-bd22-c9f159255177 nodeName:}" failed. No retries permitted until 2026-07-24 17:10:50.921992207 +0000 UTC m=+18.124041475 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/24d6b216-f67b-4d45-bd22-c9f159255177-original-pull-secret") pod "global-pull-secret-syncer-v42pr" (UID: "24d6b216-f67b-4d45-bd22-c9f159255177") : object "kube-system"/"original-pull-secret" not registered
Jul 24 17:10:43.023459 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:43.022874    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4j6q\" (UniqueName: \"kubernetes.io/projected/bcf18baa-0b35-4089-9570-df68ef7a456a-kube-api-access-d4j6q\") pod \"network-check-target-t2znt\" (UID: \"bcf18baa-0b35-4089-9570-df68ef7a456a\") " pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:10:43.023459 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:43.023022    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 17:10:43.023459 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:43.023054    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 17:10:43.023459 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:43.023066    2581 projected.go:196] Error preparing data for projected volume kube-api-access-d4j6q for pod openshift-network-diagnostics/network-check-target-t2znt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:10:43.023459 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:43.023126    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcf18baa-0b35-4089-9570-df68ef7a456a-kube-api-access-d4j6q podName:bcf18baa-0b35-4089-9570-df68ef7a456a nodeName:}" failed. No retries permitted until 2026-07-24 17:10:51.023104264 +0000 UTC m=+18.225153519 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-d4j6q" (UniqueName: "kubernetes.io/projected/bcf18baa-0b35-4089-9570-df68ef7a456a-kube-api-access-d4j6q") pod "network-check-target-t2znt" (UID: "bcf18baa-0b35-4089-9570-df68ef7a456a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:10:43.380323 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:43.380222    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:10:43.380498 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:43.380468    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v42pr" podUID="24d6b216-f67b-4d45-bd22-c9f159255177"
Jul 24 17:10:44.377393 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:44.377359    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:10:44.377777 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:44.377403    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:10:44.377777 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:44.377507    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d42hh" podUID="5c99646a-9e3e-4e60-a750-a96ab5aab0bf"
Jul 24 17:10:44.377967 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:44.377942    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t2znt" podUID="bcf18baa-0b35-4089-9570-df68ef7a456a"
Jul 24 17:10:45.379664 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:45.379200    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:10:45.379664 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:45.379324    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v42pr" podUID="24d6b216-f67b-4d45-bd22-c9f159255177"
Jul 24 17:10:46.378172 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:46.378144    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:10:46.378324 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:46.378290    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d42hh" podUID="5c99646a-9e3e-4e60-a750-a96ab5aab0bf"
Jul 24 17:10:46.378395 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:46.378368    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:10:46.378499 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:46.378471    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t2znt" podUID="bcf18baa-0b35-4089-9570-df68ef7a456a"
Jul 24 17:10:47.377430 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:47.377391    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:10:47.377793 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:47.377500    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v42pr" podUID="24d6b216-f67b-4d45-bd22-c9f159255177"
Jul 24 17:10:48.377870 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:48.377835    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:10:48.377870 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:48.377883    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:10:48.378353 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:48.377994    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t2znt" podUID="bcf18baa-0b35-4089-9570-df68ef7a456a"
Jul 24 17:10:48.378353 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:48.378145    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d42hh" podUID="5c99646a-9e3e-4e60-a750-a96ab5aab0bf"
Jul 24 17:10:49.377813 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:49.377779    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:10:49.377995 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:49.377884    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v42pr" podUID="24d6b216-f67b-4d45-bd22-c9f159255177"
Jul 24 17:10:50.378218 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:50.378178    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:10:50.378713 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:50.378195    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:10:50.378713 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:50.378317    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d42hh" podUID="5c99646a-9e3e-4e60-a750-a96ab5aab0bf"
Jul 24 17:10:50.378713 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:50.378390    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t2znt" podUID="bcf18baa-0b35-4089-9570-df68ef7a456a"
Jul 24 17:10:50.983721 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:50.983690    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c99646a-9e3e-4e60-a750-a96ab5aab0bf-metrics-certs\") pod \"network-metrics-daemon-d42hh\" (UID: \"5c99646a-9e3e-4e60-a750-a96ab5aab0bf\") " pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:10:50.984009 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:50.983759    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/24d6b216-f67b-4d45-bd22-c9f159255177-original-pull-secret\") pod \"global-pull-secret-syncer-v42pr\" (UID: \"24d6b216-f67b-4d45-bd22-c9f159255177\") " pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:10:50.984009 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:50.983858    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:10:50.984009 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:50.983930    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c99646a-9e3e-4e60-a750-a96ab5aab0bf-metrics-certs podName:5c99646a-9e3e-4e60-a750-a96ab5aab0bf nodeName:}" failed. No retries permitted until 2026-07-24 17:11:06.983909149 +0000 UTC m=+34.185958423 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5c99646a-9e3e-4e60-a750-a96ab5aab0bf-metrics-certs") pod "network-metrics-daemon-d42hh" (UID: "5c99646a-9e3e-4e60-a750-a96ab5aab0bf") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:10:50.984009 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:50.983861    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 17:10:50.984009 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:50.984006    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24d6b216-f67b-4d45-bd22-c9f159255177-original-pull-secret podName:24d6b216-f67b-4d45-bd22-c9f159255177 nodeName:}" failed. No retries permitted until 2026-07-24 17:11:06.983990267 +0000 UTC m=+34.186039522 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/24d6b216-f67b-4d45-bd22-c9f159255177-original-pull-secret") pod "global-pull-secret-syncer-v42pr" (UID: "24d6b216-f67b-4d45-bd22-c9f159255177") : object "kube-system"/"original-pull-secret" not registered
Jul 24 17:10:51.084939 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:51.084896    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4j6q\" (UniqueName: \"kubernetes.io/projected/bcf18baa-0b35-4089-9570-df68ef7a456a-kube-api-access-d4j6q\") pod \"network-check-target-t2znt\" (UID: \"bcf18baa-0b35-4089-9570-df68ef7a456a\") " pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:10:51.085127 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:51.085053    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 17:10:51.085127 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:51.085086    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 17:10:51.085127 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:51.085097    2581 projected.go:196] Error preparing data for projected volume kube-api-access-d4j6q for pod openshift-network-diagnostics/network-check-target-t2znt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:10:51.085245 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:51.085176    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcf18baa-0b35-4089-9570-df68ef7a456a-kube-api-access-d4j6q podName:bcf18baa-0b35-4089-9570-df68ef7a456a nodeName:}" failed. No retries permitted until 2026-07-24 17:11:07.085157011 +0000 UTC m=+34.287206263 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-d4j6q" (UniqueName: "kubernetes.io/projected/bcf18baa-0b35-4089-9570-df68ef7a456a-kube-api-access-d4j6q") pod "network-check-target-t2znt" (UID: "bcf18baa-0b35-4089-9570-df68ef7a456a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:10:51.377942 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:51.377906    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:10:51.378101 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:51.378023    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v42pr" podUID="24d6b216-f67b-4d45-bd22-c9f159255177"
Jul 24 17:10:52.378214 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:52.378176    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:10:52.378680 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:52.378185    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:10:52.378680 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:52.378322    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d42hh" podUID="5c99646a-9e3e-4e60-a750-a96ab5aab0bf"
Jul 24 17:10:52.378680 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:52.378348    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t2znt" podUID="bcf18baa-0b35-4089-9570-df68ef7a456a"
Jul 24 17:10:53.378967 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:53.378928    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:10:53.379471 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:53.379063    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v42pr" podUID="24d6b216-f67b-4d45-bd22-c9f159255177"
Jul 24 17:10:54.377878 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:54.377843    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:10:54.378109 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:54.377883    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:10:54.378109 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:54.377981    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d42hh" podUID="5c99646a-9e3e-4e60-a750-a96ab5aab0bf"
Jul 24 17:10:54.378234 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:54.378133    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t2znt" podUID="bcf18baa-0b35-4089-9570-df68ef7a456a"
Jul 24 17:10:55.377701 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:55.377341    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:10:55.378239 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:55.377815    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v42pr" podUID="24d6b216-f67b-4d45-bd22-c9f159255177"
Jul 24 17:10:55.542330 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:55.542305    2581 generic.go:356] "Generic (PLEG): container finished" podID="9f393f69-5a28-41ab-a160-42147718f3da" containerID="9af68eb345405a2eedcc8110c69e3d57ec1728988d6b43e3b12b78860b3c54e1" exitCode=0
Jul 24 17:10:55.542420 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:55.542373    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-px6r8" event={"ID":"9f393f69-5a28-41ab-a160-42147718f3da","Type":"ContainerDied","Data":"9af68eb345405a2eedcc8110c69e3d57ec1728988d6b43e3b12b78860b3c54e1"}
Jul 24 17:10:55.543848 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:55.543822    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb9cp" event={"ID":"30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a","Type":"ContainerStarted","Data":"2550e343afc5b52d3c692b80912d138b05f73153d2b27099ff0e56fc5fa24846"}
Jul 24 17:10:55.545284 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:55.545263    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hv6sp" event={"ID":"17ed6b09-72e1-4161-9a92-cb481241bfe1","Type":"ContainerStarted","Data":"4f2dc7a5786b863995b350bd03e625fd8bce018779be3b833f11e3f0e125e840"}
Jul 24 17:10:55.547377 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:55.547357    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2" event={"ID":"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c","Type":"ContainerStarted","Data":"6b3b9f8c48b5bf3c42f8b64d46e56e7b1a095f7b35c90dcd29d9a4b72f86e752"}
Jul 24 17:10:55.547454 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:55.547431    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2" event={"ID":"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c","Type":"ContainerStarted","Data":"ece2d59216f02a39d4592f94db5ad2fddda05caeb612a48d508f359dbcbfb369"}
Jul 24 17:10:55.547508 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:55.547485    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2" event={"ID":"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c","Type":"ContainerStarted","Data":"87a99a780bb4eae131bab69c5c92c42069c7bd33c1584c1d3c8025331211c541"}
Jul 24 17:10:55.548860 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:55.548838    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zdv5f" event={"ID":"9417588a-ebcd-4fe0-a050-2345106ba8f3","Type":"ContainerStarted","Data":"8e3ac972b553b80326e5a254cb65a548267da11f1b97afddab732d8858b0f913"}
Jul 24 17:10:55.555377 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:55.555356    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4c827" event={"ID":"fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70","Type":"ContainerStarted","Data":"29a7abcdfb7ca4294764c37be1e8b2bc381a3de505f605f1aa0e2d5e2f44d40d"}
Jul 24 17:10:55.556582 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:55.556564    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-22jtw" event={"ID":"32f3d102-a6d1-41af-8410-42dc28eb6975","Type":"ContainerStarted","Data":"6976b360c1dd7b9ade600efc059b0f9f55c294c979cbd11161b179f2ef779d72"}
Jul 24 17:10:55.557829 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:55.557795    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hpxzm" event={"ID":"ccfd714f-9f15-40f6-a06c-6054adfb9992","Type":"ContainerStarted","Data":"500f7ee67ee9171ac411926ca97b1c1786b6e5956858b260e9d15faeeee5d806"}
Jul 24 17:10:55.578180 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:55.578143    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-zdv5f" podStartSLOduration=3.926332043 podStartE2EDuration="22.578130567s" totalImagesPullingTime="18.651798524s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:10:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:10:55.577979061 +0000 UTC m=+22.780028346" watchObservedRunningTime="2026-07-24 17:10:55.578130567 +0000 UTC m=+22.780179854"
Jul 24 17:10:55.592007 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:55.591968    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-4c827" podStartSLOduration=7.672931735 podStartE2EDuration="22.591955413s" totalImagesPullingTime="14.919023678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:10:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:10:55.591394372 +0000 UTC m=+22.793443658" watchObservedRunningTime="2026-07-24 17:10:55.591955413 +0000 UTC m=+22.794004686"
Jul 24 17:10:55.625146 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:55.625106    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-hv6sp" podStartSLOduration=4.001414217 podStartE2EDuration="22.625091188s" totalImagesPullingTime="18.623676971s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:10:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:10:55.60614336 +0000 UTC m=+22.808192634" watchObservedRunningTime="2026-07-24 17:10:55.625091188 +0000 UTC m=+22.827140462"
Jul 24 17:10:55.625234 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:55.625199    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-22jtw" podStartSLOduration=3.928640451 podStartE2EDuration="22.625195863s" totalImagesPullingTime="18.696555412s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:10:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:10:55.624668359 +0000 UTC m=+22.826717629" watchObservedRunningTime="2026-07-24 17:10:55.625195863 +0000 UTC m=+22.827245136"
Jul 24 17:10:55.647306 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:55.647182    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-hpxzm" podStartSLOduration=4.027351095 podStartE2EDuration="22.647162758s" totalImagesPullingTime="18.619811663s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:10:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:10:55.646076746 +0000 UTC m=+22.848126020" watchObservedRunningTime="2026-07-24 17:10:55.647162758 +0000 UTC m=+22.849212033"
Jul 24 17:10:55.767926 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:55.767902    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hpxzm_ccfd714f-9f15-40f6-a06c-6054adfb9992/dns-node-resolver/0.log"
Jul 24 17:10:55.768220 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:55.768191    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:10:55.847835 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:55.847771    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:10:55.853829 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:55.853740    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-4c827"
Jul 24 17:10:55.854298 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:55.854280    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-4c827"
Jul 24 17:10:56.377629 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:56.377603    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:10:56.377801 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:56.377603    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:10:56.377801 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:56.377736    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d42hh" podUID="5c99646a-9e3e-4e60-a750-a96ab5aab0bf"
Jul 24 17:10:56.377801 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:56.377771    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t2znt" podUID="bcf18baa-0b35-4089-9570-df68ef7a456a"
Jul 24 17:10:56.561989 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:56.561953    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-df5fv" event={"ID":"2071cf36-c0fa-4f24-bb3c-bd15dd2896d6","Type":"ContainerStarted","Data":"832c99a7132611de354620a9bf07a839aa69d53083a81b5806716401c272246b"}
Jul 24 17:10:56.565420 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:56.565386    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2" event={"ID":"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c","Type":"ContainerStarted","Data":"c735bf1c295ecbb898d9c6800a569030456006971f087d779f76673237246230"}
Jul 24 17:10:56.565558 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:56.565468    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2" event={"ID":"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c","Type":"ContainerStarted","Data":"228a95dee63bec5d7696fbb8b8772910a7f6c6aa61461c1c7463b2efd73fa22b"}
Jul 24 17:10:56.565558 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:56.565533    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2" event={"ID":"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c","Type":"ContainerStarted","Data":"b7cbf3bc68ee611b8aa542d7c7e05556cfed4c4a438dc19b44554ef8a1e01356"}
Jul 24 17:10:56.578825 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:56.576240    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-df5fv" podStartSLOduration=4.954985961 podStartE2EDuration="23.576226798s" totalImagesPullingTime="18.621240837s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:10:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:10:56.575936781 +0000 UTC m=+23.777986054" watchObservedRunningTime="2026-07-24 17:10:56.576226798 +0000 UTC m=+23.778276073"
Jul 24 17:10:56.692026 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:56.691993    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:10:56.714352 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:56.714325    2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 24 17:10:56.758453 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:56.758430    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hv6sp_17ed6b09-72e1-4161-9a92-cb481241bfe1/node-ca/0.log"
Jul 24 17:10:56.792856 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:56.792832    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:10:56.975690 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:56.975583    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:10:57.033631 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:57.033592    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:10:57.299836 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:57.299680    2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-24T17:10:56.71434778Z","UUID":"98984bab-aef9-4bd0-aeef-ff8d5cde1a9c","Handler":null,"Name":"","Endpoint":""}
Jul 24 17:10:57.301312 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:57.301289    2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 24 17:10:57.301444 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:57.301358    2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 24 17:10:57.378170 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:57.378141    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:10:57.378530 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:57.378284    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v42pr" podUID="24d6b216-f67b-4d45-bd22-c9f159255177"
Jul 24 17:10:57.569981 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:57.569901    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb9cp" event={"ID":"30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a","Type":"ContainerStarted","Data":"7b1400bf8627fb69a6bae0fa458e687f80660089ef272f5ebb5a743054066105"}
Jul 24 17:10:57.570153 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:57.570006    2581 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 24 17:10:57.964755 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:57.964556    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:10:58.377774 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:58.377737    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:10:58.377967 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:58.377849    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d42hh" podUID="5c99646a-9e3e-4e60-a750-a96ab5aab0bf"
Jul 24 17:10:58.377967 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:58.377911    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:10:58.378106 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:58.378007    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t2znt" podUID="bcf18baa-0b35-4089-9570-df68ef7a456a"
Jul 24 17:10:58.575029 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:58.574991    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb9cp" event={"ID":"30f8f5cd-dbe7-4d1c-a65b-0c472a7b7c6a","Type":"ContainerStarted","Data":"41b485a2bebcfc6cc5279698c5dce79e2dfce263f42e1dc220b6c214a4b25897"}
Jul 24 17:10:58.578985 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:58.578959    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2" event={"ID":"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c","Type":"ContainerStarted","Data":"9170325cf54af79eea55daa3b0ce71e0b22d02fe25a1e8409da508cd7ec2e325"}
Jul 24 17:10:58.596224 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:58.596185    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb9cp" podStartSLOduration=4.647492616 podStartE2EDuration="25.59617354s" totalImagesPullingTime="20.948680924s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:10:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:10:58.596013008 +0000 UTC m=+25.798062282" watchObservedRunningTime="2026-07-24 17:10:58.59617354 +0000 UTC m=+25.798222874"
Jul 24 17:10:59.377826 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:59.377792    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:10:59.378052 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:10:59.377934    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v42pr" podUID="24d6b216-f67b-4d45-bd22-c9f159255177"
Jul 24 17:10:59.826176 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:10:59.826141    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:00.357649 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:00.357623    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:00.377498 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:00.377474    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:11:00.377585 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:00.377500    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:11:00.377585 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:00.377574    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d42hh" podUID="5c99646a-9e3e-4e60-a750-a96ab5aab0bf"
Jul 24 17:11:00.377666 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:00.377629    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t2znt" podUID="bcf18baa-0b35-4089-9570-df68ef7a456a"
Jul 24 17:11:00.587568 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:00.587349    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2" event={"ID":"c002a7e5-2e5d-4d1d-bc44-0e5b3491073c","Type":"ContainerStarted","Data":"051922be2bd135a2c50f8fa177784e85b7c1a0d0815821798955047013e53c07"}
Jul 24 17:11:00.587694 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:00.587635    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:11:00.587694 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:00.587673    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:11:00.587797 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:00.587712    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:11:00.589609 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:00.589582    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-px6r8" event={"ID":"9f393f69-5a28-41ab-a160-42147718f3da","Type":"ContainerStarted","Data":"9e3524f5ce5db76f807165da34be394d3e9578d99441c22157122679a97b33a4"}
Jul 24 17:11:00.605482 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:00.605456    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:11:00.618425 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:00.617318    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2" podStartSLOduration=8.915296154 podStartE2EDuration="27.617297066s" totalImagesPullingTime="18.702000912s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:10:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:11:00.616216187 +0000 UTC m=+27.818265440" watchObservedRunningTime="2026-07-24 17:11:00.617297066 +0000 UTC m=+27.819346342"
Jul 24 17:11:00.620719 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:00.620695    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:11:01.073914 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:01.073877    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:01.306462 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:01.306427    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:01.377654 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:01.377571    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:11:01.377828 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:01.377677    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v42pr" podUID="24d6b216-f67b-4d45-bd22-c9f159255177"
Jul 24 17:11:01.457603 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:01.457569    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:01.569631 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:01.569599    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:01.592728 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:01.592698    2581 generic.go:356] "Generic (PLEG): container finished" podID="9f393f69-5a28-41ab-a160-42147718f3da" containerID="9e3524f5ce5db76f807165da34be394d3e9578d99441c22157122679a97b33a4" exitCode=0
Jul 24 17:11:01.592870 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:01.592727    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-px6r8" event={"ID":"9f393f69-5a28-41ab-a160-42147718f3da","Type":"ContainerDied","Data":"9e3524f5ce5db76f807165da34be394d3e9578d99441c22157122679a97b33a4"}
Jul 24 17:11:01.653777 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:01.653751    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:01.777765 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:01.777731    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:02.377459 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:02.377430    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:11:02.377838 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:02.377431    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:11:02.377838 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:02.377531    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d42hh" podUID="5c99646a-9e3e-4e60-a750-a96ab5aab0bf"
Jul 24 17:11:02.377838 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:02.377602    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t2znt" podUID="bcf18baa-0b35-4089-9570-df68ef7a456a"
Jul 24 17:11:02.773943 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:02.773912    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:02.991657 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:02.991571    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:03.378448 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:03.378380    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:11:03.378808 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:03.378457    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v42pr" podUID="24d6b216-f67b-4d45-bd22-c9f159255177"
Jul 24 17:11:03.380950 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:03.380929    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:03.571930 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:03.571902    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:03.598433 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:03.598407    2581 generic.go:356] "Generic (PLEG): container finished" podID="9f393f69-5a28-41ab-a160-42147718f3da" containerID="0d26df545388f01d37da5d0474252b1864ce686f6adea6a6576178974e97c094" exitCode=0
Jul 24 17:11:03.598575 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:03.598466    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-px6r8" event={"ID":"9f393f69-5a28-41ab-a160-42147718f3da","Type":"ContainerDied","Data":"0d26df545388f01d37da5d0474252b1864ce686f6adea6a6576178974e97c094"}
Jul 24 17:11:03.734388 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:03.734358    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-4c827"
Jul 24 17:11:03.734615 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:03.734474    2581 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 24 17:11:03.735018 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:03.734994    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-4c827"
Jul 24 17:11:04.169782 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:04.169751    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:04.377297 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:04.377265    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:11:04.377447 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:04.377372    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t2znt" podUID="bcf18baa-0b35-4089-9570-df68ef7a456a"
Jul 24 17:11:04.377447 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:04.377432    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:11:04.377554 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:04.377534    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d42hh" podUID="5c99646a-9e3e-4e60-a750-a96ab5aab0bf"
Jul 24 17:11:04.575207 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:04.575173    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:04.773304 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:04.773224    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:04.974257 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:04.974225    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:05.026853 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:05.026793    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:05.377927 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:05.377852    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:11:05.378096 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:05.377962    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v42pr" podUID="24d6b216-f67b-4d45-bd22-c9f159255177"
Jul 24 17:11:05.378769 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:05.378752    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:05.604827 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:05.604794    2581 generic.go:356] "Generic (PLEG): container finished" podID="9f393f69-5a28-41ab-a160-42147718f3da" containerID="4aa589a592f33d80eea8a3e74d889f57681341bfedc1e9a6973791fb2ec13363" exitCode=0
Jul 24 17:11:05.605197 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:05.604853    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-px6r8" event={"ID":"9f393f69-5a28-41ab-a160-42147718f3da","Type":"ContainerDied","Data":"4aa589a592f33d80eea8a3e74d889f57681341bfedc1e9a6973791fb2ec13363"}
Jul 24 17:11:05.777120 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:05.777083    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:05.832944 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:05.832919    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:06.174786 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:06.174754    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:06.377353 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:06.377315    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:11:06.377528 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:06.377324    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:11:06.377528 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:06.377445    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d42hh" podUID="5c99646a-9e3e-4e60-a750-a96ab5aab0bf"
Jul 24 17:11:06.377620 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:06.377560    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t2znt" podUID="bcf18baa-0b35-4089-9570-df68ef7a456a"
Jul 24 17:11:06.574520 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:06.574429    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:06.682838 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:06.682797    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:06.740810 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:06.740767    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:06.981914 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:06.981879    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:07.004313 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:07.004284    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/24d6b216-f67b-4d45-bd22-c9f159255177-original-pull-secret\") pod \"global-pull-secret-syncer-v42pr\" (UID: \"24d6b216-f67b-4d45-bd22-c9f159255177\") " pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:11:07.004462 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:07.004330    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c99646a-9e3e-4e60-a750-a96ab5aab0bf-metrics-certs\") pod \"network-metrics-daemon-d42hh\" (UID: \"5c99646a-9e3e-4e60-a750-a96ab5aab0bf\") " pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:11:07.004462 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:07.004439    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:11:07.004462 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:07.004444    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 17:11:07.004593 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:07.004500    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c99646a-9e3e-4e60-a750-a96ab5aab0bf-metrics-certs podName:5c99646a-9e3e-4e60-a750-a96ab5aab0bf nodeName:}" failed. No retries permitted until 2026-07-24 17:11:39.004482281 +0000 UTC m=+66.206531535 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5c99646a-9e3e-4e60-a750-a96ab5aab0bf-metrics-certs") pod "network-metrics-daemon-d42hh" (UID: "5c99646a-9e3e-4e60-a750-a96ab5aab0bf") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:11:07.004593 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:07.004513    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24d6b216-f67b-4d45-bd22-c9f159255177-original-pull-secret podName:24d6b216-f67b-4d45-bd22-c9f159255177 nodeName:}" failed. No retries permitted until 2026-07-24 17:11:39.004508046 +0000 UTC m=+66.206557298 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/24d6b216-f67b-4d45-bd22-c9f159255177-original-pull-secret") pod "global-pull-secret-syncer-v42pr" (UID: "24d6b216-f67b-4d45-bd22-c9f159255177") : object "kube-system"/"original-pull-secret" not registered
Jul 24 17:11:07.104812 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:07.104785    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4j6q\" (UniqueName: \"kubernetes.io/projected/bcf18baa-0b35-4089-9570-df68ef7a456a-kube-api-access-d4j6q\") pod \"network-check-target-t2znt\" (UID: \"bcf18baa-0b35-4089-9570-df68ef7a456a\") " pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:11:07.104975 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:07.104913    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 17:11:07.104975 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:07.104929    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 17:11:07.104975 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:07.104937    2581 projected.go:196] Error preparing data for projected volume kube-api-access-d4j6q for pod openshift-network-diagnostics/network-check-target-t2znt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:11:07.105115 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:07.104993    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcf18baa-0b35-4089-9570-df68ef7a456a-kube-api-access-d4j6q podName:bcf18baa-0b35-4089-9570-df68ef7a456a nodeName:}" failed. No retries permitted until 2026-07-24 17:11:39.104974132 +0000 UTC m=+66.307023407 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-d4j6q" (UniqueName: "kubernetes.io/projected/bcf18baa-0b35-4089-9570-df68ef7a456a-kube-api-access-d4j6q") pod "network-check-target-t2znt" (UID: "bcf18baa-0b35-4089-9570-df68ef7a456a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:11:07.184369 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:07.184327    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:07.280602 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:07.280508    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:07.377477 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:07.377447    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:11:07.377636 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:07.377564    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v42pr" podUID="24d6b216-f67b-4d45-bd22-c9f159255177"
Jul 24 17:11:07.570691 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:07.570591    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:08.370533 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:08.370500    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:08.377724 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:08.377700    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:11:08.377855 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:08.377734    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:11:08.377855 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:08.377839    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d42hh" podUID="5c99646a-9e3e-4e60-a750-a96ab5aab0bf"
Jul 24 17:11:08.377977 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:08.377953    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t2znt" podUID="bcf18baa-0b35-4089-9570-df68ef7a456a"
Jul 24 17:11:08.571119 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:08.571086    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:08.975158 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:08.975118    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:09.184892 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:09.184853    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:09.377679 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:09.377589    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:11:09.378152 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:09.377732    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v42pr" podUID="24d6b216-f67b-4d45-bd22-c9f159255177"
Jul 24 17:11:09.766108 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:09.766064    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:09.819491 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:09.819462    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:10.176390 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:10.176360    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:10.377280 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:10.377251    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:11:10.377440 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:10.377251    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:11:10.377440 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:10.377355    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t2znt" podUID="bcf18baa-0b35-4089-9570-df68ef7a456a"
Jul 24 17:11:10.377440 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:10.377428    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d42hh" podUID="5c99646a-9e3e-4e60-a750-a96ab5aab0bf"
Jul 24 17:11:10.379363 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:10.379342    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:10.619850 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:10.619767    2581 generic.go:356] "Generic (PLEG): container finished" podID="9f393f69-5a28-41ab-a160-42147718f3da" containerID="30c3a470b0aafec45a57dec74b3f2dca86b61a10b590ec65d065739ac16b5df2" exitCode=0
Jul 24 17:11:10.619850 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:10.619808    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-px6r8" event={"ID":"9f393f69-5a28-41ab-a160-42147718f3da","Type":"ContainerDied","Data":"30c3a470b0aafec45a57dec74b3f2dca86b61a10b590ec65d065739ac16b5df2"}
Jul 24 17:11:10.977844 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:10.977657    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:11.376810 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:11.376723    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:11.377726 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:11.377712    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:11:11.377844 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:11.377824    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v42pr" podUID="24d6b216-f67b-4d45-bd22-c9f159255177"
Jul 24 17:11:11.572253 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:11.572219    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:11.624652 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:11.624622    2581 generic.go:356] "Generic (PLEG): container finished" podID="9f393f69-5a28-41ab-a160-42147718f3da" containerID="4a3cfdb12e9139e8f5604c3d773b3129e8828382c6922695bdfed5312a3b51b5" exitCode=0
Jul 24 17:11:11.624791 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:11.624678    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-px6r8" event={"ID":"9f393f69-5a28-41ab-a160-42147718f3da","Type":"ContainerDied","Data":"4a3cfdb12e9139e8f5604c3d773b3129e8828382c6922695bdfed5312a3b51b5"}
Jul 24 17:11:11.775468 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:11.775423    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:12.377995 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:12.377961    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:11:12.378176 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:12.378083    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d42hh" podUID="5c99646a-9e3e-4e60-a750-a96ab5aab0bf"
Jul 24 17:11:12.378176 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:12.378136    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:11:12.378247 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:12.378220    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t2znt" podUID="bcf18baa-0b35-4089-9570-df68ef7a456a"
Jul 24 17:11:12.629749 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:12.629666    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-px6r8" event={"ID":"9f393f69-5a28-41ab-a160-42147718f3da","Type":"ContainerStarted","Data":"f7f61eebf00de397a296e953ce8f3f56d86b87036a823559d39f5a8cdca21a31"}
Jul 24 17:11:12.654053 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:12.653994    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-px6r8" podStartSLOduration=9.950030782 podStartE2EDuration="39.653978043s" totalImagesPullingTime="29.703947261s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:10:33 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:11:12.653561921 +0000 UTC m=+39.855611194" watchObservedRunningTime="2026-07-24 17:11:12.653978043 +0000 UTC m=+39.856027317"
Jul 24 17:11:13.378146 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:13.378106    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:11:13.378351 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:13.378206    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v42pr" podUID="24d6b216-f67b-4d45-bd22-c9f159255177"
Jul 24 17:11:14.377261 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:14.377220    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:11:14.377668 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:14.377220    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:11:14.377668 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:14.377335    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t2znt" podUID="bcf18baa-0b35-4089-9570-df68ef7a456a"
Jul 24 17:11:14.377668 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:14.377417    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d42hh" podUID="5c99646a-9e3e-4e60-a750-a96ab5aab0bf"
Jul 24 17:11:14.927368 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:14.927332    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:14.985878 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:14.985844    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:15.317777 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:15.317703    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:15.377907 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:15.377872    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:11:15.378278 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:15.377985    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v42pr" podUID="24d6b216-f67b-4d45-bd22-c9f159255177"
Jul 24 17:11:15.397312 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:15.397284    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:15.493134 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:15.493100    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:15.575017 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:15.574943    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:15.646591 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:15.646555    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:16.377432 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:16.377401    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:11:16.377432 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:16.377427    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:11:16.377624 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:16.377552    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d42hh" podUID="5c99646a-9e3e-4e60-a750-a96ab5aab0bf"
Jul 24 17:11:16.377708 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:16.377689    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t2znt" podUID="bcf18baa-0b35-4089-9570-df68ef7a456a"
Jul 24 17:11:16.603218 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:16.603181    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-v42pr"]
Jul 24 17:11:16.604072 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:16.603308    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:11:16.604072 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:16.603422    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v42pr" podUID="24d6b216-f67b-4d45-bd22-c9f159255177"
Jul 24 17:11:16.606272 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:16.606246    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-d42hh"]
Jul 24 17:11:16.606456 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:16.606441    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-t2znt"]
Jul 24 17:11:16.639436 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:16.639410    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:11:16.639583 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:16.639449    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:11:16.639583 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:16.639531    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t2znt" podUID="bcf18baa-0b35-4089-9570-df68ef7a456a"
Jul 24 17:11:16.639711 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:16.639690    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d42hh" podUID="5c99646a-9e3e-4e60-a750-a96ab5aab0bf"
Jul 24 17:11:18.377626 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:18.377593    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:11:18.377626 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:18.377611    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:11:18.377626 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:18.377613    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:11:18.378182 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:18.377718    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v42pr" podUID="24d6b216-f67b-4d45-bd22-c9f159255177"
Jul 24 17:11:18.378182 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:18.377823    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t2znt" podUID="bcf18baa-0b35-4089-9570-df68ef7a456a"
Jul 24 17:11:18.378182 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:18.377892    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d42hh" podUID="5c99646a-9e3e-4e60-a750-a96ab5aab0bf"
Jul 24 17:11:20.377951 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:20.377912    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:11:20.377951 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:20.377941    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:11:20.378383 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:20.378010    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:11:20.378383 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:20.378025    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v42pr" podUID="24d6b216-f67b-4d45-bd22-c9f159255177"
Jul 24 17:11:20.378383 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:20.378113    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d42hh" podUID="5c99646a-9e3e-4e60-a750-a96ab5aab0bf"
Jul 24 17:11:20.378383 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:20.378170    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t2znt" podUID="bcf18baa-0b35-4089-9570-df68ef7a456a"
Jul 24 17:11:21.114138 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.113946    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-46.ec2.internal" event="NodeReady"
Jul 24 17:11:21.114310 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.114221    2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 24 17:11:21.165886 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.165849    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-tt7pb"]
Jul 24 17:11:21.178963 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.178930    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-sg66r"]
Jul 24 17:11:21.179125 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.179081    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tt7pb"
Jul 24 17:11:21.182234 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.182212    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 24 17:11:21.182828 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.182804    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cpjrs\""
Jul 24 17:11:21.183187 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.183137    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 24 17:11:21.191557 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.191537    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sg66r"]
Jul 24 17:11:21.191658 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.191570    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tt7pb"]
Jul 24 17:11:21.191658 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.191604    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-8x79m"]
Jul 24 17:11:21.191762 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.191712    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sg66r"
Jul 24 17:11:21.194301 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.194285    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 24 17:11:21.194408 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.194392    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-g5d6l\""
Jul 24 17:11:21.194819 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.194807    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 24 17:11:21.195212 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.195196    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 24 17:11:21.196660 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.196641    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6glt\" (UniqueName: \"kubernetes.io/projected/c483050c-0fa4-4739-865c-d4aa9fda7cf1-kube-api-access-s6glt\") pod \"dns-default-tt7pb\" (UID: \"c483050c-0fa4-4739-865c-d4aa9fda7cf1\") " pod="openshift-dns/dns-default-tt7pb"
Jul 24 17:11:21.196785 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.196689    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/339766e7-0509-456d-8cc7-790156dfb1be-cert\") pod \"ingress-canary-sg66r\" (UID: \"339766e7-0509-456d-8cc7-790156dfb1be\") " pod="openshift-ingress-canary/ingress-canary-sg66r"
Jul 24 17:11:21.196785 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.196714    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c483050c-0fa4-4739-865c-d4aa9fda7cf1-metrics-tls\") pod \"dns-default-tt7pb\" (UID: \"c483050c-0fa4-4739-865c-d4aa9fda7cf1\") " pod="openshift-dns/dns-default-tt7pb"
Jul 24 17:11:21.196785 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.196735    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c483050c-0fa4-4739-865c-d4aa9fda7cf1-tmp-dir\") pod \"dns-default-tt7pb\" (UID: \"c483050c-0fa4-4739-865c-d4aa9fda7cf1\") " pod="openshift-dns/dns-default-tt7pb"
Jul 24 17:11:21.196785 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.196760    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c483050c-0fa4-4739-865c-d4aa9fda7cf1-config-volume\") pod \"dns-default-tt7pb\" (UID: \"c483050c-0fa4-4739-865c-d4aa9fda7cf1\") " pod="openshift-dns/dns-default-tt7pb"
Jul 24 17:11:21.197006 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.196828    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4d2tw\" (UniqueName: \"kubernetes.io/projected/339766e7-0509-456d-8cc7-790156dfb1be-kube-api-access-4d2tw\") pod \"ingress-canary-sg66r\" (UID: \"339766e7-0509-456d-8cc7-790156dfb1be\") " pod="openshift-ingress-canary/ingress-canary-sg66r"
Jul 24 17:11:21.205895 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.205876    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8x79m"]
Jul 24 17:11:21.206003 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.205979    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8x79m"
Jul 24 17:11:21.208819 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.208801    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 24 17:11:21.208819 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.208809    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 24 17:11:21.208967 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.208809    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 24 17:11:21.208967 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.208812    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 24 17:11:21.208967 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.208804    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bwrlh\""
Jul 24 17:11:21.287675 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.287641    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:21.297719 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.297697    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c483050c-0fa4-4739-865c-d4aa9fda7cf1-metrics-tls\") pod \"dns-default-tt7pb\" (UID: \"c483050c-0fa4-4739-865c-d4aa9fda7cf1\") " pod="openshift-dns/dns-default-tt7pb"
Jul 24 17:11:21.297802 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.297722    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c483050c-0fa4-4739-865c-d4aa9fda7cf1-tmp-dir\") pod \"dns-default-tt7pb\" (UID: \"c483050c-0fa4-4739-865c-d4aa9fda7cf1\") " pod="openshift-dns/dns-default-tt7pb"
Jul 24 17:11:21.297802 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.297743    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/44c6ba96-c4bc-42c6-b993-43a864544dff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8x79m\" (UID: \"44c6ba96-c4bc-42c6-b993-43a864544dff\") " pod="openshift-insights/insights-runtime-extractor-8x79m"
Jul 24 17:11:21.297802 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.297763    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c483050c-0fa4-4739-865c-d4aa9fda7cf1-config-volume\") pod \"dns-default-tt7pb\" (UID: \"c483050c-0fa4-4739-865c-d4aa9fda7cf1\") " pod="openshift-dns/dns-default-tt7pb"
Jul 24 17:11:21.297802 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.297781    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4d2tw\" (UniqueName: \"kubernetes.io/projected/339766e7-0509-456d-8cc7-790156dfb1be-kube-api-access-4d2tw\") pod \"ingress-canary-sg66r\" (UID: \"339766e7-0509-456d-8cc7-790156dfb1be\") " pod="openshift-ingress-canary/ingress-canary-sg66r"
Jul 24 17:11:21.298083 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.297919    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/44c6ba96-c4bc-42c6-b993-43a864544dff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8x79m\" (UID: \"44c6ba96-c4bc-42c6-b993-43a864544dff\") " pod="openshift-insights/insights-runtime-extractor-8x79m"
Jul 24 17:11:21.298083 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.297948    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/44c6ba96-c4bc-42c6-b993-43a864544dff-crio-socket\") pod \"insights-runtime-extractor-8x79m\" (UID: \"44c6ba96-c4bc-42c6-b993-43a864544dff\") " pod="openshift-insights/insights-runtime-extractor-8x79m"
Jul 24 17:11:21.298083 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.297989    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6glt\" (UniqueName: \"kubernetes.io/projected/c483050c-0fa4-4739-865c-d4aa9fda7cf1-kube-api-access-s6glt\") pod \"dns-default-tt7pb\" (UID: \"c483050c-0fa4-4739-865c-d4aa9fda7cf1\") " pod="openshift-dns/dns-default-tt7pb"
Jul 24 17:11:21.298083 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.298011    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/44c6ba96-c4bc-42c6-b993-43a864544dff-data-volume\") pod \"insights-runtime-extractor-8x79m\" (UID: \"44c6ba96-c4bc-42c6-b993-43a864544dff\") " pod="openshift-insights/insights-runtime-extractor-8x79m"
Jul 24 17:11:21.298293 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.298114    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/339766e7-0509-456d-8cc7-790156dfb1be-cert\") pod \"ingress-canary-sg66r\" (UID: \"339766e7-0509-456d-8cc7-790156dfb1be\") " pod="openshift-ingress-canary/ingress-canary-sg66r"
Jul 24 17:11:21.298293 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.298156    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c483050c-0fa4-4739-865c-d4aa9fda7cf1-tmp-dir\") pod \"dns-default-tt7pb\" (UID: \"c483050c-0fa4-4739-865c-d4aa9fda7cf1\") " pod="openshift-dns/dns-default-tt7pb"
Jul 24 17:11:21.298293 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.298173    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nh5lg\" (UniqueName: \"kubernetes.io/projected/44c6ba96-c4bc-42c6-b993-43a864544dff-kube-api-access-nh5lg\") pod \"insights-runtime-extractor-8x79m\" (UID: \"44c6ba96-c4bc-42c6-b993-43a864544dff\") " pod="openshift-insights/insights-runtime-extractor-8x79m"
Jul 24 17:11:21.298446 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.298428    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c483050c-0fa4-4739-865c-d4aa9fda7cf1-config-volume\") pod \"dns-default-tt7pb\" (UID: \"c483050c-0fa4-4739-865c-d4aa9fda7cf1\") " pod="openshift-dns/dns-default-tt7pb"
Jul 24 17:11:21.302675 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.302655    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/339766e7-0509-456d-8cc7-790156dfb1be-cert\") pod \"ingress-canary-sg66r\" (UID: \"339766e7-0509-456d-8cc7-790156dfb1be\") " pod="openshift-ingress-canary/ingress-canary-sg66r"
Jul 24 17:11:21.304894 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.304868    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c483050c-0fa4-4739-865c-d4aa9fda7cf1-metrics-tls\") pod \"dns-default-tt7pb\" (UID: \"c483050c-0fa4-4739-865c-d4aa9fda7cf1\") " pod="openshift-dns/dns-default-tt7pb"
Jul 24 17:11:21.307058 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.307010    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4d2tw\" (UniqueName: \"kubernetes.io/projected/339766e7-0509-456d-8cc7-790156dfb1be-kube-api-access-4d2tw\") pod \"ingress-canary-sg66r\" (UID: \"339766e7-0509-456d-8cc7-790156dfb1be\") " pod="openshift-ingress-canary/ingress-canary-sg66r"
Jul 24 17:11:21.307257 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.307237    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6glt\" (UniqueName: \"kubernetes.io/projected/c483050c-0fa4-4739-865c-d4aa9fda7cf1-kube-api-access-s6glt\") pod \"dns-default-tt7pb\" (UID: \"c483050c-0fa4-4739-865c-d4aa9fda7cf1\") " pod="openshift-dns/dns-default-tt7pb"
Jul 24 17:11:21.343062 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.343017    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:21.399475 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.399450    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/44c6ba96-c4bc-42c6-b993-43a864544dff-data-volume\") pod \"insights-runtime-extractor-8x79m\" (UID: \"44c6ba96-c4bc-42c6-b993-43a864544dff\") " pod="openshift-insights/insights-runtime-extractor-8x79m"
Jul 24 17:11:21.399883 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.399504    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nh5lg\" (UniqueName: \"kubernetes.io/projected/44c6ba96-c4bc-42c6-b993-43a864544dff-kube-api-access-nh5lg\") pod \"insights-runtime-extractor-8x79m\" (UID: \"44c6ba96-c4bc-42c6-b993-43a864544dff\") " pod="openshift-insights/insights-runtime-extractor-8x79m"
Jul 24 17:11:21.399883 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.399527    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/44c6ba96-c4bc-42c6-b993-43a864544dff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8x79m\" (UID: \"44c6ba96-c4bc-42c6-b993-43a864544dff\") " pod="openshift-insights/insights-runtime-extractor-8x79m"
Jul 24 17:11:21.399883 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.399564    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/44c6ba96-c4bc-42c6-b993-43a864544dff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8x79m\" (UID: \"44c6ba96-c4bc-42c6-b993-43a864544dff\") " pod="openshift-insights/insights-runtime-extractor-8x79m"
Jul 24 17:11:21.399883 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.399581    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/44c6ba96-c4bc-42c6-b993-43a864544dff-crio-socket\") pod \"insights-runtime-extractor-8x79m\" (UID: \"44c6ba96-c4bc-42c6-b993-43a864544dff\") " pod="openshift-insights/insights-runtime-extractor-8x79m"
Jul 24 17:11:21.399883 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.399768    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/44c6ba96-c4bc-42c6-b993-43a864544dff-crio-socket\") pod \"insights-runtime-extractor-8x79m\" (UID: \"44c6ba96-c4bc-42c6-b993-43a864544dff\") " pod="openshift-insights/insights-runtime-extractor-8x79m"
Jul 24 17:11:21.399883 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.399825    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/44c6ba96-c4bc-42c6-b993-43a864544dff-data-volume\") pod \"insights-runtime-extractor-8x79m\" (UID: \"44c6ba96-c4bc-42c6-b993-43a864544dff\") " pod="openshift-insights/insights-runtime-extractor-8x79m"
Jul 24 17:11:21.400238 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.400220    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/44c6ba96-c4bc-42c6-b993-43a864544dff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8x79m\" (UID: \"44c6ba96-c4bc-42c6-b993-43a864544dff\") " pod="openshift-insights/insights-runtime-extractor-8x79m"
Jul 24 17:11:21.401898 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.401882    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/44c6ba96-c4bc-42c6-b993-43a864544dff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8x79m\" (UID: \"44c6ba96-c4bc-42c6-b993-43a864544dff\") " pod="openshift-insights/insights-runtime-extractor-8x79m"
Jul 24 17:11:21.409241 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.409183    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nh5lg\" (UniqueName: \"kubernetes.io/projected/44c6ba96-c4bc-42c6-b993-43a864544dff-kube-api-access-nh5lg\") pod \"insights-runtime-extractor-8x79m\" (UID: \"44c6ba96-c4bc-42c6-b993-43a864544dff\") " pod="openshift-insights/insights-runtime-extractor-8x79m"
Jul 24 17:11:21.492230 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.492194    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tt7pb"
Jul 24 17:11:21.499434 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.499412    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sg66r"
Jul 24 17:11:21.514676 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.514658    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8x79m"
Jul 24 17:11:21.704965 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.704934    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sg66r"]
Jul 24 17:11:21.708385 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:11:21.708360    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod339766e7_0509_456d_8cc7_790156dfb1be.slice/crio-7f12ef09d286ddb8a82da2567e639f26f4db01728c86fbedb90c6be7e4f2e2e3 WatchSource:0}: Error finding container 7f12ef09d286ddb8a82da2567e639f26f4db01728c86fbedb90c6be7e4f2e2e3: Status 404 returned error can't find the container with id 7f12ef09d286ddb8a82da2567e639f26f4db01728c86fbedb90c6be7e4f2e2e3
Jul 24 17:11:21.711074 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.711050    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8x79m"]
Jul 24 17:11:21.713212 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:11:21.713187    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44c6ba96_c4bc_42c6_b993_43a864544dff.slice/crio-a9dce08c550baae70938504fd30b92b86e606ef5868271b5341be731e5ed7cda WatchSource:0}: Error finding container a9dce08c550baae70938504fd30b92b86e606ef5868271b5341be731e5ed7cda: Status 404 returned error can't find the container with id a9dce08c550baae70938504fd30b92b86e606ef5868271b5341be731e5ed7cda
Jul 24 17:11:21.716298 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:21.716276    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tt7pb"]
Jul 24 17:11:21.719185 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:11:21.719165    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc483050c_0fa4_4739_865c_d4aa9fda7cf1.slice/crio-4934b104be2880fa68d8ea53e67370140490b736cb44f8693c92584d5644ef2e WatchSource:0}: Error finding container 4934b104be2880fa68d8ea53e67370140490b736cb44f8693c92584d5644ef2e: Status 404 returned error can't find the container with id 4934b104be2880fa68d8ea53e67370140490b736cb44f8693c92584d5644ef2e
Jul 24 17:11:22.377278 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:22.377245    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:11:22.377457 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:22.377253    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:11:22.377457 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:22.377253    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:11:22.381719 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:22.381696    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 24 17:11:22.381719 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:22.381719    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 17:11:22.381903 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:22.381755    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 24 17:11:22.381969 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:22.381949    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-56frs\""
Jul 24 17:11:22.382154 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:22.382136    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 17:11:22.382264 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:22.382249    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gkgpl\""
Jul 24 17:11:22.656765 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:22.656686    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sg66r" event={"ID":"339766e7-0509-456d-8cc7-790156dfb1be","Type":"ContainerStarted","Data":"7f12ef09d286ddb8a82da2567e639f26f4db01728c86fbedb90c6be7e4f2e2e3"}
Jul 24 17:11:22.659482 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:22.659450    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8x79m" event={"ID":"44c6ba96-c4bc-42c6-b993-43a864544dff","Type":"ContainerStarted","Data":"a9dce08c550baae70938504fd30b92b86e606ef5868271b5341be731e5ed7cda"}
Jul 24 17:11:22.661004 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:22.660977    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tt7pb" event={"ID":"c483050c-0fa4-4739-865c-d4aa9fda7cf1","Type":"ContainerStarted","Data":"4934b104be2880fa68d8ea53e67370140490b736cb44f8693c92584d5644ef2e"}
Jul 24 17:11:23.217946 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:23.217904    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:23.320772 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:23.320736    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:23.382607 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:23.382573    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:23.383411 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:23.383391    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6p8zw"]
Jul 24 17:11:23.399573 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:23.399547    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6p8zw"]
Jul 24 17:11:23.399778 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:23.399758    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6p8zw"
Jul 24 17:11:23.403284 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:23.403261    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-52vxk\""
Jul 24 17:11:23.403404 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:23.403261    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 24 17:11:23.414129 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:23.414099    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7eba8703-9720-4dcf-9883-e47e9a46ce76-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-6p8zw\" (UID: \"7eba8703-9720-4dcf-9883-e47e9a46ce76\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6p8zw"
Jul 24 17:11:23.445389 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:23.445351    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:23.514917 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:23.514832    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7eba8703-9720-4dcf-9883-e47e9a46ce76-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-6p8zw\" (UID: \"7eba8703-9720-4dcf-9883-e47e9a46ce76\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6p8zw"
Jul 24 17:11:23.519032 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:23.519004    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7eba8703-9720-4dcf-9883-e47e9a46ce76-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-6p8zw\" (UID: \"7eba8703-9720-4dcf-9883-e47e9a46ce76\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6p8zw"
Jul 24 17:11:23.676180 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:23.676125    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:23.712966 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:23.712926    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6p8zw"
Jul 24 17:11:23.731082 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:23.731017    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:23.788894 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:23.788825    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:23.848297 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:23.848263    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:23.913399 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:23.913329    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:26.084220 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:26.084011    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:26.141336 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:26.141303    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:28.538259 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:28.537978    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-wf48c"]
Jul 24 17:11:28.542939 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:28.542242    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-wf48c"
Jul 24 17:11:28.542939 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:28.542732    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:28.546762 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:28.546733    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-bjcld\""
Jul 24 17:11:28.547359 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:28.546998    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 24 17:11:28.547452 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:28.547380    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 24 17:11:28.557195 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:28.556910    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6p8zw"]
Jul 24 17:11:28.557300 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:28.557256    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-wf48c"]
Jul 24 17:11:28.557655 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:11:28.557543    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7eba8703_9720_4dcf_9883_e47e9a46ce76.slice/crio-62f6b54cc83473816a27df853f1139f14ade9c3296a7578255e06c1850c46de4 WatchSource:0}: Error finding container 62f6b54cc83473816a27df853f1139f14ade9c3296a7578255e06c1850c46de4: Status 404 returned error can't find the container with id 62f6b54cc83473816a27df853f1139f14ade9c3296a7578255e06c1850c46de4
Jul 24 17:11:28.644917 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:28.644884    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:28.657755 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:28.657656    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxfr9\" (UniqueName: \"kubernetes.io/projected/d625ce61-acef-427e-9d65-75720151fcd7-kube-api-access-pxfr9\") pod \"downloads-7c97dc4555-wf48c\" (UID: \"d625ce61-acef-427e-9d65-75720151fcd7\") " pod="openshift-console/downloads-7c97dc4555-wf48c"
Jul 24 17:11:28.679247 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:28.679216    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sg66r" event={"ID":"339766e7-0509-456d-8cc7-790156dfb1be","Type":"ContainerStarted","Data":"18f6ced6a6d9377a30f8d68e7ff0ffba02298f07ec078160d91255f09bf7f6a2"}
Jul 24 17:11:28.680716 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:28.680693    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8x79m" event={"ID":"44c6ba96-c4bc-42c6-b993-43a864544dff","Type":"ContainerStarted","Data":"187209dc0e18eb83543376ce87c90dbe28591fba395529243cd1495ef0fa207e"}
Jul 24 17:11:28.681846 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:28.681828    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6p8zw" event={"ID":"7eba8703-9720-4dcf-9883-e47e9a46ce76","Type":"ContainerStarted","Data":"62f6b54cc83473816a27df853f1139f14ade9c3296a7578255e06c1850c46de4"}
Jul 24 17:11:28.683646 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:28.683625    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tt7pb" event={"ID":"c483050c-0fa4-4739-865c-d4aa9fda7cf1","Type":"ContainerStarted","Data":"a71d497abb77154aba5841230b3297e1664b3180e1327597e80c1a98e0e4af09"}
Jul 24 17:11:28.683725 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:28.683667    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tt7pb" event={"ID":"c483050c-0fa4-4739-865c-d4aa9fda7cf1","Type":"ContainerStarted","Data":"d3ce1115651ac7827f3660750a0091982a17bd29fc1723bc136a853b8ebd7b01"}
Jul 24 17:11:28.683796 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:28.683781    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-tt7pb"
Jul 24 17:11:28.695850 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:28.695812    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-sg66r" podStartSLOduration=1.006735443 podStartE2EDuration="7.695799805s" totalImagesPullingTime="6.689064362s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:11:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:11:28.694948614 +0000 UTC m=+55.896997888" watchObservedRunningTime="2026-07-24 17:11:28.695799805 +0000 UTC m=+55.897849079"
Jul 24 17:11:28.699064 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:28.699032    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:28.711826 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:28.711787    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-tt7pb" podStartSLOduration=1.031861214 podStartE2EDuration="7.711772134s" totalImagesPullingTime="6.67991092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:11:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:11:28.711097822 +0000 UTC m=+55.913147097" watchObservedRunningTime="2026-07-24 17:11:28.711772134 +0000 UTC m=+55.913821407"
Jul 24 17:11:28.758467 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:28.758434    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxfr9\" (UniqueName: \"kubernetes.io/projected/d625ce61-acef-427e-9d65-75720151fcd7-kube-api-access-pxfr9\") pod \"downloads-7c97dc4555-wf48c\" (UID: \"d625ce61-acef-427e-9d65-75720151fcd7\") " pod="openshift-console/downloads-7c97dc4555-wf48c"
Jul 24 17:11:28.768788 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:28.768762    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxfr9\" (UniqueName: \"kubernetes.io/projected/d625ce61-acef-427e-9d65-75720151fcd7-kube-api-access-pxfr9\") pod \"downloads-7c97dc4555-wf48c\" (UID: \"d625ce61-acef-427e-9d65-75720151fcd7\") " pod="openshift-console/downloads-7c97dc4555-wf48c"
Jul 24 17:11:28.842907 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:28.842876    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:28.860937 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:28.860907    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-wf48c"
Jul 24 17:11:29.033986 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:29.033952    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-wf48c"]
Jul 24 17:11:29.043743 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:11:29.043714    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd625ce61_acef_427e_9d65_75720151fcd7.slice/crio-4ba732b7e381f80ba5f73bb531a84ecb51aa77ce49aabf32b109c211953391b1 WatchSource:0}: Error finding container 4ba732b7e381f80ba5f73bb531a84ecb51aa77ce49aabf32b109c211953391b1: Status 404 returned error can't find the container with id 4ba732b7e381f80ba5f73bb531a84ecb51aa77ce49aabf32b109c211953391b1
Jul 24 17:11:29.209697 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:29.209610    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:29.308925 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:29.308892    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:29.442967 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:29.442933    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:29.500316 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:29.500203    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:29.640077 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:29.640026    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:29.687563 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:29.687534    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-wf48c" event={"ID":"d625ce61-acef-427e-9d65-75720151fcd7","Type":"ContainerStarted","Data":"4ba732b7e381f80ba5f73bb531a84ecb51aa77ce49aabf32b109c211953391b1"}
Jul 24 17:11:29.691221 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:29.691191    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:29.923567 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:29.923311    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:30.024274 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:30.024234    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:30.152721 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:30.152655    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:30.265106 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:30.265007    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:30.421864 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:30.421832    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:30.625153 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:30.625072    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:30.692750 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:30.692714    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6p8zw" event={"ID":"7eba8703-9720-4dcf-9883-e47e9a46ce76","Type":"ContainerStarted","Data":"3697f742de8867163a85914a3adc0270564d361829eaa44018049978bba77a92"}
Jul 24 17:11:30.693315 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:30.693258    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6p8zw"
Jul 24 17:11:30.699919 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:30.699894    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6p8zw"
Jul 24 17:11:30.713489 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:30.713442    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6p8zw" podStartSLOduration=6.586160919 podStartE2EDuration="7.713427231s" totalImagesPullingTime="1.127266312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:11:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:11:30.712537968 +0000 UTC m=+57.914587238" watchObservedRunningTime="2026-07-24 17:11:30.713427231 +0000 UTC m=+57.915476483"
Jul 24 17:11:30.738252 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:30.738219    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:31.413161 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:31.413126    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:31.443440 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:31.443407    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-h88gg"]
Jul 24 17:11:31.455463 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:31.455435    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-h88gg"
Jul 24 17:11:31.457411 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:31.457388    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-h88gg"]
Jul 24 17:11:31.458742 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:31.458485    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 24 17:11:31.458742 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:31.458517    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 24 17:11:31.458742 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:31.458539    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-dw5zr\""
Jul 24 17:11:31.459846 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:31.459826    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 24 17:11:31.460415 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:31.460099    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 24 17:11:31.460415 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:31.460281    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 24 17:11:31.477686 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:31.477657    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:31.485645 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:31.485616    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6kmh\" (UniqueName: \"kubernetes.io/projected/546de716-d864-4169-ac8a-c9bae8467697-kube-api-access-n6kmh\") pod \"prometheus-operator-84d69f794d-h88gg\" (UID: \"546de716-d864-4169-ac8a-c9bae8467697\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h88gg"
Jul 24 17:11:31.485789 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:31.485668    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/546de716-d864-4169-ac8a-c9bae8467697-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-h88gg\" (UID: \"546de716-d864-4169-ac8a-c9bae8467697\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h88gg"
Jul 24 17:11:31.485789 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:31.485708    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/546de716-d864-4169-ac8a-c9bae8467697-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-h88gg\" (UID: \"546de716-d864-4169-ac8a-c9bae8467697\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h88gg"
Jul 24 17:11:31.485900 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:31.485787    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/546de716-d864-4169-ac8a-c9bae8467697-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-h88gg\" (UID: \"546de716-d864-4169-ac8a-c9bae8467697\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h88gg"
Jul 24 17:11:31.586195 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:31.586155    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/546de716-d864-4169-ac8a-c9bae8467697-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-h88gg\" (UID: \"546de716-d864-4169-ac8a-c9bae8467697\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h88gg"
Jul 24 17:11:31.586379 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:31.586202    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/546de716-d864-4169-ac8a-c9bae8467697-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-h88gg\" (UID: \"546de716-d864-4169-ac8a-c9bae8467697\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h88gg"
Jul 24 17:11:31.586379 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:31.586271    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/546de716-d864-4169-ac8a-c9bae8467697-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-h88gg\" (UID: \"546de716-d864-4169-ac8a-c9bae8467697\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h88gg"
Jul 24 17:11:31.586379 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:31.586320    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6kmh\" (UniqueName: \"kubernetes.io/projected/546de716-d864-4169-ac8a-c9bae8467697-kube-api-access-n6kmh\") pod \"prometheus-operator-84d69f794d-h88gg\" (UID: \"546de716-d864-4169-ac8a-c9bae8467697\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h88gg"
Jul 24 17:11:31.586551 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:31.586451    2581 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 24 17:11:31.586551 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:31.586514    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/546de716-d864-4169-ac8a-c9bae8467697-prometheus-operator-tls podName:546de716-d864-4169-ac8a-c9bae8467697 nodeName:}" failed. No retries permitted until 2026-07-24 17:11:32.086497563 +0000 UTC m=+59.288546817 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/546de716-d864-4169-ac8a-c9bae8467697-prometheus-operator-tls") pod "prometheus-operator-84d69f794d-h88gg" (UID: "546de716-d864-4169-ac8a-c9bae8467697") : secret "prometheus-operator-tls" not found
Jul 24 17:11:31.587027 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:31.587007    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/546de716-d864-4169-ac8a-c9bae8467697-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-h88gg\" (UID: \"546de716-d864-4169-ac8a-c9bae8467697\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h88gg"
Jul 24 17:11:31.589915 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:31.589868    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/546de716-d864-4169-ac8a-c9bae8467697-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-h88gg\" (UID: \"546de716-d864-4169-ac8a-c9bae8467697\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h88gg"
Jul 24 17:11:31.595573 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:31.595540    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6kmh\" (UniqueName: \"kubernetes.io/projected/546de716-d864-4169-ac8a-c9bae8467697-kube-api-access-n6kmh\") pod \"prometheus-operator-84d69f794d-h88gg\" (UID: \"546de716-d864-4169-ac8a-c9bae8467697\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h88gg"
Jul 24 17:11:31.841607 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:31.841529    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:31.898174 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:31.898138    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:32.091718 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:32.091608    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/546de716-d864-4169-ac8a-c9bae8467697-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-h88gg\" (UID: \"546de716-d864-4169-ac8a-c9bae8467697\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h88gg"
Jul 24 17:11:32.095640 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:32.095616    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/546de716-d864-4169-ac8a-c9bae8467697-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-h88gg\" (UID: \"546de716-d864-4169-ac8a-c9bae8467697\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h88gg"
Jul 24 17:11:32.368027 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:32.367944    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-h88gg"
Jul 24 17:11:32.583278 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:32.583206    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-h88gg"]
Jul 24 17:11:32.586625 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:11:32.586600    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod546de716_d864_4169_ac8a_c9bae8467697.slice/crio-8b7996e5a54eb4d2a835579230181af0704677621bff834b7a5b435a96b0684a WatchSource:0}: Error finding container 8b7996e5a54eb4d2a835579230181af0704677621bff834b7a5b435a96b0684a: Status 404 returned error can't find the container with id 8b7996e5a54eb4d2a835579230181af0704677621bff834b7a5b435a96b0684a
Jul 24 17:11:32.606224 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:32.606200    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9lnl2"
Jul 24 17:11:32.701354 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:32.701319    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-h88gg" event={"ID":"546de716-d864-4169-ac8a-c9bae8467697","Type":"ContainerStarted","Data":"8b7996e5a54eb4d2a835579230181af0704677621bff834b7a5b435a96b0684a"}
Jul 24 17:11:32.703287 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:32.703266    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8x79m" event={"ID":"44c6ba96-c4bc-42c6-b993-43a864544dff","Type":"ContainerStarted","Data":"d5af365fa82b4833357054a892dbd5c98f26ddd18a234bbed1c61644ea4a38ef"}
Jul 24 17:11:32.757012 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:32.756983    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:32.857239 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:32.857201    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:32.957138 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:32.956977    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:34.713007 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:34.712970    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-h88gg" event={"ID":"546de716-d864-4169-ac8a-c9bae8467697","Type":"ContainerStarted","Data":"a276ff0246c6eb54e719479fd4393d0d6722c2afba6b6dd24c76a39ff7b16cba"}
Jul 24 17:11:34.862827 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:34.862741    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:36.723136 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:36.723060    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-h88gg" event={"ID":"546de716-d864-4169-ac8a-c9bae8467697","Type":"ContainerStarted","Data":"c43ddc042b96ec2e212d26cc64690c84b08fd5691fad23a42553838d7d83d56b"}
Jul 24 17:11:36.741808 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:36.741450    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-h88gg" podStartSLOduration=3.800473174 podStartE2EDuration="5.741435104s" totalImagesPullingTime="1.94096193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:11:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:11:36.740863727 +0000 UTC m=+63.942913000" watchObservedRunningTime="2026-07-24 17:11:36.741435104 +0000 UTC m=+63.943484369"
Jul 24 17:11:37.611477 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:37.611437    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:37.731680 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:37.731641    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8x79m" event={"ID":"44c6ba96-c4bc-42c6-b993-43a864544dff","Type":"ContainerStarted","Data":"b13113ad9c0101a8cf29eaca5975084fba4b4b01dc9f2666bba3648a14f3146d"}
Jul 24 17:11:37.750542 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:37.750479    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-8x79m" podStartSLOduration=1.937998248 podStartE2EDuration="16.75046023s" totalImagesPullingTime="14.812461982s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:11:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:11:37.749136281 +0000 UTC m=+64.951185568" watchObservedRunningTime="2026-07-24 17:11:37.75046023 +0000 UTC m=+64.952509505"
Jul 24 17:11:37.753511 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:37.753483    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:37.852670 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:37.852619    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:38.531767 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.531719    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:38.690681 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.690653    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-tt7pb"
Jul 24 17:11:38.752257 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.752214    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:38.778748 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.778720    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-8pb55"]
Jul 24 17:11:38.781399 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.781377    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8pb55"
Jul 24 17:11:38.784240 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.784165    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 24 17:11:38.784378 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.784356    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 24 17:11:38.784580 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.784562    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-b5kwq\""
Jul 24 17:11:38.790820 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.790800    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-f9qj2"]
Jul 24 17:11:38.793337 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.793317    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-f9qj2"
Jul 24 17:11:38.796330 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.796309    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-5lsq6\""
Jul 24 17:11:38.796869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.796848    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 24 17:11:38.797085 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.797066    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 24 17:11:38.797623 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.797605    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 24 17:11:38.797717 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.797643    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-8pb55"]
Jul 24 17:11:38.804907 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.804887    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-dvz8j"]
Jul 24 17:11:38.807658 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.807635    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dvz8j"
Jul 24 17:11:38.808766 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.808723    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-f9qj2"]
Jul 24 17:11:38.810514 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.810497    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 24 17:11:38.810810 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.810791    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 24 17:11:38.810810 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.810802    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 24 17:11:38.811145 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.811129    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-kgtc8\""
Jul 24 17:11:38.846797 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.846765    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:38.847749 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.847724    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d8e047ab-f0c3-48cc-84c2-19559698ab11-root\") pod \"node-exporter-dvz8j\" (UID: \"d8e047ab-f0c3-48cc-84c2-19559698ab11\") " pod="openshift-monitoring/node-exporter-dvz8j"
Jul 24 17:11:38.847881 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.847756    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6lsf\" (UniqueName: \"kubernetes.io/projected/d8e047ab-f0c3-48cc-84c2-19559698ab11-kube-api-access-d6lsf\") pod \"node-exporter-dvz8j\" (UID: \"d8e047ab-f0c3-48cc-84c2-19559698ab11\") " pod="openshift-monitoring/node-exporter-dvz8j"
Jul 24 17:11:38.847881 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.847776    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/fac1bdfa-4c1a-480d-9b1a-e3b025e473bb-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-f9qj2\" (UID: \"fac1bdfa-4c1a-480d-9b1a-e3b025e473bb\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-f9qj2"
Jul 24 17:11:38.847881 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.847798    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fac1bdfa-4c1a-480d-9b1a-e3b025e473bb-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-f9qj2\" (UID: \"fac1bdfa-4c1a-480d-9b1a-e3b025e473bb\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-f9qj2"
Jul 24 17:11:38.848067 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.847889    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2596065-10a4-4cca-bdca-04d5aa1208a2-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-8pb55\" (UID: \"a2596065-10a4-4cca-bdca-04d5aa1208a2\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8pb55"
Jul 24 17:11:38.848067 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.847907    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d8e047ab-f0c3-48cc-84c2-19559698ab11-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dvz8j\" (UID: \"d8e047ab-f0c3-48cc-84c2-19559698ab11\") " pod="openshift-monitoring/node-exporter-dvz8j"
Jul 24 17:11:38.848067 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.847940    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d8e047ab-f0c3-48cc-84c2-19559698ab11-node-exporter-tls\") pod \"node-exporter-dvz8j\" (UID: \"d8e047ab-f0c3-48cc-84c2-19559698ab11\") " pod="openshift-monitoring/node-exporter-dvz8j"
Jul 24 17:11:38.848067 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.847970    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d8e047ab-f0c3-48cc-84c2-19559698ab11-node-exporter-wtmp\") pod \"node-exporter-dvz8j\" (UID: \"d8e047ab-f0c3-48cc-84c2-19559698ab11\") " pod="openshift-monitoring/node-exporter-dvz8j"
Jul 24 17:11:38.848067 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.847995    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a2596065-10a4-4cca-bdca-04d5aa1208a2-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-8pb55\" (UID: \"a2596065-10a4-4cca-bdca-04d5aa1208a2\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8pb55"
Jul 24 17:11:38.848067 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.848021    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2596065-10a4-4cca-bdca-04d5aa1208a2-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-8pb55\" (UID: \"a2596065-10a4-4cca-bdca-04d5aa1208a2\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8pb55"
Jul 24 17:11:38.848314 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.848075    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/fac1bdfa-4c1a-480d-9b1a-e3b025e473bb-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-f9qj2\" (UID: \"fac1bdfa-4c1a-480d-9b1a-e3b025e473bb\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-f9qj2"
Jul 24 17:11:38.848314 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.848111    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5hcm\" (UniqueName: \"kubernetes.io/projected/fac1bdfa-4c1a-480d-9b1a-e3b025e473bb-kube-api-access-b5hcm\") pod \"kube-state-metrics-56d9f944b4-f9qj2\" (UID: \"fac1bdfa-4c1a-480d-9b1a-e3b025e473bb\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-f9qj2"
Jul 24 17:11:38.848314 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.848147    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fac1bdfa-4c1a-480d-9b1a-e3b025e473bb-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-f9qj2\" (UID: \"fac1bdfa-4c1a-480d-9b1a-e3b025e473bb\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-f9qj2"
Jul 24 17:11:38.848314 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.848214    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d8e047ab-f0c3-48cc-84c2-19559698ab11-node-exporter-textfile\") pod \"node-exporter-dvz8j\" (UID: \"d8e047ab-f0c3-48cc-84c2-19559698ab11\") " pod="openshift-monitoring/node-exporter-dvz8j"
Jul 24 17:11:38.848314 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.848252    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9426n\" (UniqueName: \"kubernetes.io/projected/a2596065-10a4-4cca-bdca-04d5aa1208a2-kube-api-access-9426n\") pod \"openshift-state-metrics-b8f49f5c6-8pb55\" (UID: \"a2596065-10a4-4cca-bdca-04d5aa1208a2\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8pb55"
Jul 24 17:11:38.848314 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.848280    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8e047ab-f0c3-48cc-84c2-19559698ab11-sys\") pod \"node-exporter-dvz8j\" (UID: \"d8e047ab-f0c3-48cc-84c2-19559698ab11\") " pod="openshift-monitoring/node-exporter-dvz8j"
Jul 24 17:11:38.848523 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.848325    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fac1bdfa-4c1a-480d-9b1a-e3b025e473bb-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-f9qj2\" (UID: \"fac1bdfa-4c1a-480d-9b1a-e3b025e473bb\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-f9qj2"
Jul 24 17:11:38.848523 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.848387    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d8e047ab-f0c3-48cc-84c2-19559698ab11-node-exporter-accelerators-collector-config\") pod \"node-exporter-dvz8j\" (UID: \"d8e047ab-f0c3-48cc-84c2-19559698ab11\") " pod="openshift-monitoring/node-exporter-dvz8j"
Jul 24 17:11:38.848523 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.848417    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d8e047ab-f0c3-48cc-84c2-19559698ab11-metrics-client-ca\") pod \"node-exporter-dvz8j\" (UID: \"d8e047ab-f0c3-48cc-84c2-19559698ab11\") " pod="openshift-monitoring/node-exporter-dvz8j"
Jul 24 17:11:38.917029 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.916996    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:38.949693 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.949665    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2596065-10a4-4cca-bdca-04d5aa1208a2-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-8pb55\" (UID: \"a2596065-10a4-4cca-bdca-04d5aa1208a2\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8pb55"
Jul 24 17:11:38.949863 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.949709    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d8e047ab-f0c3-48cc-84c2-19559698ab11-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dvz8j\" (UID: \"d8e047ab-f0c3-48cc-84c2-19559698ab11\") " pod="openshift-monitoring/node-exporter-dvz8j"
Jul 24 17:11:38.949863 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.949733    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d8e047ab-f0c3-48cc-84c2-19559698ab11-node-exporter-tls\") pod \"node-exporter-dvz8j\" (UID: \"d8e047ab-f0c3-48cc-84c2-19559698ab11\") " pod="openshift-monitoring/node-exporter-dvz8j"
Jul 24 17:11:38.949863 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.949757    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d8e047ab-f0c3-48cc-84c2-19559698ab11-node-exporter-wtmp\") pod \"node-exporter-dvz8j\" (UID: \"d8e047ab-f0c3-48cc-84c2-19559698ab11\") " pod="openshift-monitoring/node-exporter-dvz8j"
Jul 24 17:11:38.949863 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.949779    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a2596065-10a4-4cca-bdca-04d5aa1208a2-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-8pb55\" (UID: \"a2596065-10a4-4cca-bdca-04d5aa1208a2\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8pb55"
Jul 24 17:11:38.949863 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.949802    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2596065-10a4-4cca-bdca-04d5aa1208a2-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-8pb55\" (UID: \"a2596065-10a4-4cca-bdca-04d5aa1208a2\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8pb55"
Jul 24 17:11:38.949863 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:38.949814    2581 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 24 17:11:38.949863 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.949828    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/fac1bdfa-4c1a-480d-9b1a-e3b025e473bb-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-f9qj2\" (UID: \"fac1bdfa-4c1a-480d-9b1a-e3b025e473bb\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-f9qj2"
Jul 24 17:11:38.949863 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.949853    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5hcm\" (UniqueName: \"kubernetes.io/projected/fac1bdfa-4c1a-480d-9b1a-e3b025e473bb-kube-api-access-b5hcm\") pod \"kube-state-metrics-56d9f944b4-f9qj2\" (UID: \"fac1bdfa-4c1a-480d-9b1a-e3b025e473bb\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-f9qj2"
Jul 24 17:11:38.950289 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:38.949891    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2596065-10a4-4cca-bdca-04d5aa1208a2-openshift-state-metrics-tls podName:a2596065-10a4-4cca-bdca-04d5aa1208a2 nodeName:}" failed. No retries permitted until 2026-07-24 17:11:39.449869895 +0000 UTC m=+66.651919147 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/a2596065-10a4-4cca-bdca-04d5aa1208a2-openshift-state-metrics-tls") pod "openshift-state-metrics-b8f49f5c6-8pb55" (UID: "a2596065-10a4-4cca-bdca-04d5aa1208a2") : secret "openshift-state-metrics-tls" not found
Jul 24 17:11:38.950289 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:38.949894    2581 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 24 17:11:38.950289 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.949917    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fac1bdfa-4c1a-480d-9b1a-e3b025e473bb-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-f9qj2\" (UID: \"fac1bdfa-4c1a-480d-9b1a-e3b025e473bb\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-f9qj2"
Jul 24 17:11:38.950289 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.949942    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d8e047ab-f0c3-48cc-84c2-19559698ab11-node-exporter-wtmp\") pod \"node-exporter-dvz8j\" (UID: \"d8e047ab-f0c3-48cc-84c2-19559698ab11\") " pod="openshift-monitoring/node-exporter-dvz8j"
Jul 24 17:11:38.950289 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:38.949958    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8e047ab-f0c3-48cc-84c2-19559698ab11-node-exporter-tls podName:d8e047ab-f0c3-48cc-84c2-19559698ab11 nodeName:}" failed. No retries permitted until 2026-07-24 17:11:39.449942219 +0000 UTC m=+66.651991474 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/d8e047ab-f0c3-48cc-84c2-19559698ab11-node-exporter-tls") pod "node-exporter-dvz8j" (UID: "d8e047ab-f0c3-48cc-84c2-19559698ab11") : secret "node-exporter-tls" not found
Jul 24 17:11:38.950289 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.949978    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d8e047ab-f0c3-48cc-84c2-19559698ab11-node-exporter-textfile\") pod \"node-exporter-dvz8j\" (UID: \"d8e047ab-f0c3-48cc-84c2-19559698ab11\") " pod="openshift-monitoring/node-exporter-dvz8j"
Jul 24 17:11:38.950289 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.950015    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9426n\" (UniqueName: \"kubernetes.io/projected/a2596065-10a4-4cca-bdca-04d5aa1208a2-kube-api-access-9426n\") pod \"openshift-state-metrics-b8f49f5c6-8pb55\" (UID: \"a2596065-10a4-4cca-bdca-04d5aa1208a2\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8pb55"
Jul 24 17:11:38.950289 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.950058    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8e047ab-f0c3-48cc-84c2-19559698ab11-sys\") pod \"node-exporter-dvz8j\" (UID: \"d8e047ab-f0c3-48cc-84c2-19559698ab11\") " pod="openshift-monitoring/node-exporter-dvz8j"
Jul 24 17:11:38.950289 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.950092    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fac1bdfa-4c1a-480d-9b1a-e3b025e473bb-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-f9qj2\" (UID: \"fac1bdfa-4c1a-480d-9b1a-e3b025e473bb\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-f9qj2"
Jul 24 17:11:38.950289 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.950120    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d8e047ab-f0c3-48cc-84c2-19559698ab11-node-exporter-accelerators-collector-config\") pod \"node-exporter-dvz8j\" (UID: \"d8e047ab-f0c3-48cc-84c2-19559698ab11\") " pod="openshift-monitoring/node-exporter-dvz8j"
Jul 24 17:11:38.950289 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.950144    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d8e047ab-f0c3-48cc-84c2-19559698ab11-metrics-client-ca\") pod \"node-exporter-dvz8j\" (UID: \"d8e047ab-f0c3-48cc-84c2-19559698ab11\") " pod="openshift-monitoring/node-exporter-dvz8j"
Jul 24 17:11:38.950289 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.950191    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d8e047ab-f0c3-48cc-84c2-19559698ab11-root\") pod \"node-exporter-dvz8j\" (UID: \"d8e047ab-f0c3-48cc-84c2-19559698ab11\") " pod="openshift-monitoring/node-exporter-dvz8j"
Jul 24 17:11:38.950289 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.950214    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6lsf\" (UniqueName: \"kubernetes.io/projected/d8e047ab-f0c3-48cc-84c2-19559698ab11-kube-api-access-d6lsf\") pod \"node-exporter-dvz8j\" (UID: \"d8e047ab-f0c3-48cc-84c2-19559698ab11\") " pod="openshift-monitoring/node-exporter-dvz8j"
Jul 24 17:11:38.950289 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.950239    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/fac1bdfa-4c1a-480d-9b1a-e3b025e473bb-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-f9qj2\" (UID: \"fac1bdfa-4c1a-480d-9b1a-e3b025e473bb\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-f9qj2"
Jul 24 17:11:38.950289 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.950277    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fac1bdfa-4c1a-480d-9b1a-e3b025e473bb-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-f9qj2\" (UID: \"fac1bdfa-4c1a-480d-9b1a-e3b025e473bb\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-f9qj2"
Jul 24 17:11:38.951018 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:38.950385    2581 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 24 17:11:38.951018 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:38.950437    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fac1bdfa-4c1a-480d-9b1a-e3b025e473bb-kube-state-metrics-tls podName:fac1bdfa-4c1a-480d-9b1a-e3b025e473bb nodeName:}" failed. No retries permitted until 2026-07-24 17:11:39.450423398 +0000 UTC m=+66.652472650 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/fac1bdfa-4c1a-480d-9b1a-e3b025e473bb-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-f9qj2" (UID: "fac1bdfa-4c1a-480d-9b1a-e3b025e473bb") : secret "kube-state-metrics-tls" not found
Jul 24 17:11:38.951018 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.950720    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d8e047ab-f0c3-48cc-84c2-19559698ab11-node-exporter-textfile\") pod \"node-exporter-dvz8j\" (UID: \"d8e047ab-f0c3-48cc-84c2-19559698ab11\") " pod="openshift-monitoring/node-exporter-dvz8j"
Jul 24 17:11:38.951018 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.951004    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2596065-10a4-4cca-bdca-04d5aa1208a2-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-8pb55\" (UID: \"a2596065-10a4-4cca-bdca-04d5aa1208a2\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8pb55"
Jul 24 17:11:38.951018 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.951015    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fac1bdfa-4c1a-480d-9b1a-e3b025e473bb-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-f9qj2\" (UID: \"fac1bdfa-4c1a-480d-9b1a-e3b025e473bb\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-f9qj2"
Jul 24 17:11:38.951292 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.951241    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d8e047ab-f0c3-48cc-84c2-19559698ab11-root\") pod \"node-exporter-dvz8j\" (UID: \"d8e047ab-f0c3-48cc-84c2-19559698ab11\") " pod="openshift-monitoring/node-exporter-dvz8j"
Jul 24 17:11:38.951349 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.951294    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8e047ab-f0c3-48cc-84c2-19559698ab11-sys\") pod \"node-exporter-dvz8j\" (UID: \"d8e047ab-f0c3-48cc-84c2-19559698ab11\") " pod="openshift-monitoring/node-exporter-dvz8j"
Jul 24 17:11:38.951568 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.951545    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d8e047ab-f0c3-48cc-84c2-19559698ab11-metrics-client-ca\") pod \"node-exporter-dvz8j\" (UID: \"d8e047ab-f0c3-48cc-84c2-19559698ab11\") " pod="openshift-monitoring/node-exporter-dvz8j"
Jul 24 17:11:38.951669 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.951643    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/fac1bdfa-4c1a-480d-9b1a-e3b025e473bb-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-f9qj2\" (UID: \"fac1bdfa-4c1a-480d-9b1a-e3b025e473bb\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-f9qj2"
Jul 24 17:11:38.951816 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.951778    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d8e047ab-f0c3-48cc-84c2-19559698ab11-node-exporter-accelerators-collector-config\") pod \"node-exporter-dvz8j\" (UID: \"d8e047ab-f0c3-48cc-84c2-19559698ab11\") " pod="openshift-monitoring/node-exporter-dvz8j"
Jul 24 17:11:38.952086 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.952022    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/fac1bdfa-4c1a-480d-9b1a-e3b025e473bb-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-f9qj2\" (UID: \"fac1bdfa-4c1a-480d-9b1a-e3b025e473bb\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-f9qj2"
Jul 24 17:11:38.953002 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.952981    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a2596065-10a4-4cca-bdca-04d5aa1208a2-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-8pb55\" (UID: \"a2596065-10a4-4cca-bdca-04d5aa1208a2\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8pb55"
Jul 24 17:11:38.953141 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.953119    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d8e047ab-f0c3-48cc-84c2-19559698ab11-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dvz8j\" (UID: \"d8e047ab-f0c3-48cc-84c2-19559698ab11\") " pod="openshift-monitoring/node-exporter-dvz8j"
Jul 24 17:11:38.954273 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.954248    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fac1bdfa-4c1a-480d-9b1a-e3b025e473bb-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-f9qj2\" (UID: \"fac1bdfa-4c1a-480d-9b1a-e3b025e473bb\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-f9qj2"
Jul 24 17:11:38.960353 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.960329    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5hcm\" (UniqueName: \"kubernetes.io/projected/fac1bdfa-4c1a-480d-9b1a-e3b025e473bb-kube-api-access-b5hcm\") pod \"kube-state-metrics-56d9f944b4-f9qj2\" (UID: \"fac1bdfa-4c1a-480d-9b1a-e3b025e473bb\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-f9qj2"
Jul 24 17:11:38.960431 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.960406    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6lsf\" (UniqueName: \"kubernetes.io/projected/d8e047ab-f0c3-48cc-84c2-19559698ab11-kube-api-access-d6lsf\") pod \"node-exporter-dvz8j\" (UID: \"d8e047ab-f0c3-48cc-84c2-19559698ab11\") " pod="openshift-monitoring/node-exporter-dvz8j"
Jul 24 17:11:38.960760 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:38.960736    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9426n\" (UniqueName: \"kubernetes.io/projected/a2596065-10a4-4cca-bdca-04d5aa1208a2-kube-api-access-9426n\") pod \"openshift-state-metrics-b8f49f5c6-8pb55\" (UID: \"a2596065-10a4-4cca-bdca-04d5aa1208a2\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8pb55"
Jul 24 17:11:39.016753 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.016723    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:39.051703 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.051626    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/24d6b216-f67b-4d45-bd22-c9f159255177-original-pull-secret\") pod \"global-pull-secret-syncer-v42pr\" (UID: \"24d6b216-f67b-4d45-bd22-c9f159255177\") " pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:11:39.051891 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.051745    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c99646a-9e3e-4e60-a750-a96ab5aab0bf-metrics-certs\") pod \"network-metrics-daemon-d42hh\" (UID: \"5c99646a-9e3e-4e60-a750-a96ab5aab0bf\") " pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:11:39.054478 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.054454    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 17:11:39.054607 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.054532    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 17:11:39.065615 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.065593    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/24d6b216-f67b-4d45-bd22-c9f159255177-original-pull-secret\") pod \"global-pull-secret-syncer-v42pr\" (UID: \"24d6b216-f67b-4d45-bd22-c9f159255177\") " pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:11:39.065912 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.065886    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c99646a-9e3e-4e60-a750-a96ab5aab0bf-metrics-certs\") pod \"network-metrics-daemon-d42hh\" (UID: \"5c99646a-9e3e-4e60-a750-a96ab5aab0bf\") " pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:11:39.120067 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.120020    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:39.152326 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.152290    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4j6q\" (UniqueName: \"kubernetes.io/projected/bcf18baa-0b35-4089-9570-df68ef7a456a-kube-api-access-d4j6q\") pod \"network-check-target-t2znt\" (UID: \"bcf18baa-0b35-4089-9570-df68ef7a456a\") " pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:11:39.155231 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.155211    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 24 17:11:39.165219 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.165200    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 24 17:11:39.176401 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.176377    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4j6q\" (UniqueName: \"kubernetes.io/projected/bcf18baa-0b35-4089-9570-df68ef7a456a-kube-api-access-d4j6q\") pod \"network-check-target-t2znt\" (UID: \"bcf18baa-0b35-4089-9570-df68ef7a456a\") " pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:11:39.192597 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.192568    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-56frs\""
Jul 24 17:11:39.197170 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.197055    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v42pr"
Jul 24 17:11:39.201140 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.201111    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d42hh"
Jul 24 17:11:39.206453 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.206423    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gkgpl\""
Jul 24 17:11:39.214058 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.213998    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:11:39.217627 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.217590    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:39.277799 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.276550    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56ffd5f7b5-vvmhd"]
Jul 24 17:11:39.281137 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.280141    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56ffd5f7b5-vvmhd"
Jul 24 17:11:39.284464 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.284312    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 24 17:11:39.284583 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.284563    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 24 17:11:39.284927 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.284906    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 24 17:11:39.285233 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.285213    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 24 17:11:39.285765 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.285603    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 24 17:11:39.288394 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.288126    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-j5hxv\""
Jul 24 17:11:39.298063 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.298018    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56ffd5f7b5-vvmhd"]
Jul 24 17:11:39.305737 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.305671    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:39.354830 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.354603    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b111fa18-0380-470b-9d3c-ec7992007d20-console-config\") pod \"console-56ffd5f7b5-vvmhd\" (UID: \"b111fa18-0380-470b-9d3c-ec7992007d20\") " pod="openshift-console/console-56ffd5f7b5-vvmhd"
Jul 24 17:11:39.354830 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.354650    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b111fa18-0380-470b-9d3c-ec7992007d20-console-serving-cert\") pod \"console-56ffd5f7b5-vvmhd\" (UID: \"b111fa18-0380-470b-9d3c-ec7992007d20\") " pod="openshift-console/console-56ffd5f7b5-vvmhd"
Jul 24 17:11:39.354830 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.354679    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b111fa18-0380-470b-9d3c-ec7992007d20-service-ca\") pod \"console-56ffd5f7b5-vvmhd\" (UID: \"b111fa18-0380-470b-9d3c-ec7992007d20\") " pod="openshift-console/console-56ffd5f7b5-vvmhd"
Jul 24 17:11:39.354830 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.354711    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52hxg\" (UniqueName: \"kubernetes.io/projected/b111fa18-0380-470b-9d3c-ec7992007d20-kube-api-access-52hxg\") pod \"console-56ffd5f7b5-vvmhd\" (UID: \"b111fa18-0380-470b-9d3c-ec7992007d20\") " pod="openshift-console/console-56ffd5f7b5-vvmhd"
Jul 24 17:11:39.354830 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.354732    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b111fa18-0380-470b-9d3c-ec7992007d20-oauth-serving-cert\") pod \"console-56ffd5f7b5-vvmhd\" (UID: \"b111fa18-0380-470b-9d3c-ec7992007d20\") " pod="openshift-console/console-56ffd5f7b5-vvmhd"
Jul 24 17:11:39.354830 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.354764    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b111fa18-0380-470b-9d3c-ec7992007d20-console-oauth-config\") pod \"console-56ffd5f7b5-vvmhd\" (UID: \"b111fa18-0380-470b-9d3c-ec7992007d20\") " pod="openshift-console/console-56ffd5f7b5-vvmhd"
Jul 24 17:11:39.365478 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.365447    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-v42pr"]
Jul 24 17:11:39.388021 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.387865    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-d42hh"]
Jul 24 17:11:39.391054 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:11:39.391015    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c99646a_9e3e_4e60_a750_a96ab5aab0bf.slice/crio-1805f1e7deac3ba7b2838c27daa96dc77051d4c38da59332cab7978e81b90371 WatchSource:0}: Error finding container 1805f1e7deac3ba7b2838c27daa96dc77051d4c38da59332cab7978e81b90371: Status 404 returned error can't find the container with id 1805f1e7deac3ba7b2838c27daa96dc77051d4c38da59332cab7978e81b90371
Jul 24 17:11:39.408971 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.408941    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-t2znt"]
Jul 24 17:11:39.411613 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:11:39.411586    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcf18baa_0b35_4089_9570_df68ef7a456a.slice/crio-47d9a4cad1679c62817f4aeef8462cec48b6e5c0caa4b0254b372032834f65a0 WatchSource:0}: Error finding container 47d9a4cad1679c62817f4aeef8462cec48b6e5c0caa4b0254b372032834f65a0: Status 404 returned error can't find the container with id 47d9a4cad1679c62817f4aeef8462cec48b6e5c0caa4b0254b372032834f65a0
Jul 24 17:11:39.455731 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.455701    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d8e047ab-f0c3-48cc-84c2-19559698ab11-node-exporter-tls\") pod \"node-exporter-dvz8j\" (UID: \"d8e047ab-f0c3-48cc-84c2-19559698ab11\") " pod="openshift-monitoring/node-exporter-dvz8j"
Jul 24 17:11:39.455731 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.455737    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b111fa18-0380-470b-9d3c-ec7992007d20-service-ca\") pod \"console-56ffd5f7b5-vvmhd\" (UID: \"b111fa18-0380-470b-9d3c-ec7992007d20\") " pod="openshift-console/console-56ffd5f7b5-vvmhd"
Jul 24 17:11:39.455975 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.455781    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52hxg\" (UniqueName: \"kubernetes.io/projected/b111fa18-0380-470b-9d3c-ec7992007d20-kube-api-access-52hxg\") pod \"console-56ffd5f7b5-vvmhd\" (UID: \"b111fa18-0380-470b-9d3c-ec7992007d20\") " pod="openshift-console/console-56ffd5f7b5-vvmhd"
Jul 24 17:11:39.455975 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.455828    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b111fa18-0380-470b-9d3c-ec7992007d20-oauth-serving-cert\") pod \"console-56ffd5f7b5-vvmhd\" (UID: \"b111fa18-0380-470b-9d3c-ec7992007d20\") " pod="openshift-console/console-56ffd5f7b5-vvmhd"
Jul 24 17:11:39.455975 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.455870    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fac1bdfa-4c1a-480d-9b1a-e3b025e473bb-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-f9qj2\" (UID: \"fac1bdfa-4c1a-480d-9b1a-e3b025e473bb\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-f9qj2"
Jul 24 17:11:39.455975 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.455900    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b111fa18-0380-470b-9d3c-ec7992007d20-console-oauth-config\") pod \"console-56ffd5f7b5-vvmhd\" (UID: \"b111fa18-0380-470b-9d3c-ec7992007d20\") " pod="openshift-console/console-56ffd5f7b5-vvmhd"
Jul 24 17:11:39.455975 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.455929    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b111fa18-0380-470b-9d3c-ec7992007d20-console-config\") pod \"console-56ffd5f7b5-vvmhd\" (UID: \"b111fa18-0380-470b-9d3c-ec7992007d20\") " pod="openshift-console/console-56ffd5f7b5-vvmhd"
Jul 24 17:11:39.455975 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.455959    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2596065-10a4-4cca-bdca-04d5aa1208a2-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-8pb55\" (UID: \"a2596065-10a4-4cca-bdca-04d5aa1208a2\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8pb55"
Jul 24 17:11:39.456287 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.455982    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b111fa18-0380-470b-9d3c-ec7992007d20-console-serving-cert\") pod \"console-56ffd5f7b5-vvmhd\" (UID: \"b111fa18-0380-470b-9d3c-ec7992007d20\") " pod="openshift-console/console-56ffd5f7b5-vvmhd"
Jul 24 17:11:39.456742 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.456554    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b111fa18-0380-470b-9d3c-ec7992007d20-service-ca\") pod \"console-56ffd5f7b5-vvmhd\" (UID: \"b111fa18-0380-470b-9d3c-ec7992007d20\") " pod="openshift-console/console-56ffd5f7b5-vvmhd"
Jul 24 17:11:39.456903 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.456788    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b111fa18-0380-470b-9d3c-ec7992007d20-console-config\") pod \"console-56ffd5f7b5-vvmhd\" (UID: \"b111fa18-0380-470b-9d3c-ec7992007d20\") " pod="openshift-console/console-56ffd5f7b5-vvmhd"
Jul 24 17:11:39.457005 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.456976    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b111fa18-0380-470b-9d3c-ec7992007d20-oauth-serving-cert\") pod \"console-56ffd5f7b5-vvmhd\" (UID: \"b111fa18-0380-470b-9d3c-ec7992007d20\") " pod="openshift-console/console-56ffd5f7b5-vvmhd"
Jul 24 17:11:39.459278 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.459253    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d8e047ab-f0c3-48cc-84c2-19559698ab11-node-exporter-tls\") pod \"node-exporter-dvz8j\" (UID: \"d8e047ab-f0c3-48cc-84c2-19559698ab11\") " pod="openshift-monitoring/node-exporter-dvz8j"
Jul 24 17:11:39.459406 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.459319    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2596065-10a4-4cca-bdca-04d5aa1208a2-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-8pb55\" (UID: \"a2596065-10a4-4cca-bdca-04d5aa1208a2\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8pb55"
Jul 24 17:11:39.459406 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.459337    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b111fa18-0380-470b-9d3c-ec7992007d20-console-serving-cert\") pod \"console-56ffd5f7b5-vvmhd\" (UID: \"b111fa18-0380-470b-9d3c-ec7992007d20\") " pod="openshift-console/console-56ffd5f7b5-vvmhd"
Jul 24 17:11:39.459771 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.459748    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fac1bdfa-4c1a-480d-9b1a-e3b025e473bb-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-f9qj2\" (UID: \"fac1bdfa-4c1a-480d-9b1a-e3b025e473bb\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-f9qj2"
Jul 24 17:11:39.459861 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.459829    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b111fa18-0380-470b-9d3c-ec7992007d20-console-oauth-config\") pod \"console-56ffd5f7b5-vvmhd\" (UID: \"b111fa18-0380-470b-9d3c-ec7992007d20\") " pod="openshift-console/console-56ffd5f7b5-vvmhd"
Jul 24 17:11:39.464345 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.464323    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52hxg\" (UniqueName: \"kubernetes.io/projected/b111fa18-0380-470b-9d3c-ec7992007d20-kube-api-access-52hxg\") pod \"console-56ffd5f7b5-vvmhd\" (UID: \"b111fa18-0380-470b-9d3c-ec7992007d20\") " pod="openshift-console/console-56ffd5f7b5-vvmhd"
Jul 24 17:11:39.595908 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.595826    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56ffd5f7b5-vvmhd"
Jul 24 17:11:39.692496 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.692456    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8pb55"
Jul 24 17:11:39.704843 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.704810    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-f9qj2"
Jul 24 17:11:39.718749 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.718724    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dvz8j"
Jul 24 17:11:39.744394 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.744360    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-d42hh" event={"ID":"5c99646a-9e3e-4e60-a750-a96ab5aab0bf","Type":"ContainerStarted","Data":"1805f1e7deac3ba7b2838c27daa96dc77051d4c38da59332cab7978e81b90371"}
Jul 24 17:11:39.745660 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.745633    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-v42pr" event={"ID":"24d6b216-f67b-4d45-bd22-c9f159255177","Type":"ContainerStarted","Data":"e63ec1be29f22d1d7df68d81d6138e1ca0704415b43d2bb3767ae01f5622a938"}
Jul 24 17:11:39.746924 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.746892    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-t2znt" event={"ID":"bcf18baa-0b35-4089-9570-df68ef7a456a","Type":"ContainerStarted","Data":"47d9a4cad1679c62817f4aeef8462cec48b6e5c0caa4b0254b372032834f65a0"}
Jul 24 17:11:39.763250 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.763208    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:39.839800 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.839763    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:39.844110 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.844077    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 17:11:39.847143 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.847084    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:39.850095 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.850070    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 24 17:11:39.850265 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.850219    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 24 17:11:39.850456 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.850281    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-fz926\""
Jul 24 17:11:39.850456 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.850137    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 24 17:11:39.850875 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.850489    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 24 17:11:39.850875 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.850738    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 24 17:11:39.851012 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.850922    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 24 17:11:39.851012 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.850992    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 24 17:11:39.851152 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.851136    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 24 17:11:39.851238 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.850882    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 24 17:11:39.867769 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.867739    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 17:11:39.959232 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.959202    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/284bf1c9-f105-4053-b478-b8bd4fd6625f-config-out\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:39.959417 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.959238    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-web-config\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:39.959417 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.959258    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/284bf1c9-f105-4053-b478-b8bd4fd6625f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:39.959417 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.959290    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:39.959417 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.959323    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:39.959417 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.959394    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/284bf1c9-f105-4053-b478-b8bd4fd6625f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:39.959683 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.959436    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:39.959683 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.959476    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/284bf1c9-f105-4053-b478-b8bd4fd6625f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:39.959683 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.959509    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-config-volume\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:39.959683 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.959544    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:39.959683 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.959570    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/284bf1c9-f105-4053-b478-b8bd4fd6625f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:39.959683 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.959605    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdv9n\" (UniqueName: \"kubernetes.io/projected/284bf1c9-f105-4053-b478-b8bd4fd6625f-kube-api-access-mdv9n\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:39.959991 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.959689    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:39.962721 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:39.962688    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:40.060422 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:40.060374    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:40.060600 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:40.060430    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/284bf1c9-f105-4053-b478-b8bd4fd6625f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:40.060600 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:40.060468    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdv9n\" (UniqueName: \"kubernetes.io/projected/284bf1c9-f105-4053-b478-b8bd4fd6625f-kube-api-access-mdv9n\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:40.060600 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:40.060500    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:40.060600 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:40.060531    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/284bf1c9-f105-4053-b478-b8bd4fd6625f-config-out\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:40.060600 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:40.060558    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-web-config\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:40.060833 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:40.060771    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/284bf1c9-f105-4053-b478-b8bd4fd6625f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:40.060833 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:40.060825    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:40.060945 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:40.060856    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:40.060945 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:40.060887    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/284bf1c9-f105-4053-b478-b8bd4fd6625f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:40.060945 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:40.060927    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:40.061132 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:40.060965    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/284bf1c9-f105-4053-b478-b8bd4fd6625f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:40.061132 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:40.060999    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-config-volume\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:40.061132 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:40.061001    2581 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 24 17:11:40.061132 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:40.061093    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-secret-alertmanager-main-tls podName:284bf1c9-f105-4053-b478-b8bd4fd6625f nodeName:}" failed. No retries permitted until 2026-07-24 17:11:40.561072771 +0000 UTC m=+67.763122037 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "284bf1c9-f105-4053-b478-b8bd4fd6625f") : secret "alertmanager-main-tls" not found
Jul 24 17:11:40.061844 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:40.061446    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/284bf1c9-f105-4053-b478-b8bd4fd6625f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:40.061844 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:40.061774    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/284bf1c9-f105-4053-b478-b8bd4fd6625f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:40.062534 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:40.062059    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/284bf1c9-f105-4053-b478-b8bd4fd6625f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:40.063519 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:40.063494    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/284bf1c9-f105-4053-b478-b8bd4fd6625f-config-out\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:40.063835 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:40.063815    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:40.064725 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:40.064703    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:40.065009 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:40.064984    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:40.065123 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:40.065099    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-web-config\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:40.065402 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:40.065376    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:40.065608 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:40.065588    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/284bf1c9-f105-4053-b478-b8bd4fd6625f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:40.067131 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:40.067113    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-config-volume\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:40.070602 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:40.070573    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdv9n\" (UniqueName: \"kubernetes.io/projected/284bf1c9-f105-4053-b478-b8bd4fd6625f-kube-api-access-mdv9n\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:40.164703 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:40.164669    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:40.441962 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:40.441870    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:40.505389 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:40.505332    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:40.565567 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:40.565530    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:40.568423 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:40.568399    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:40.758598 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:40.758505    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:11:41.673240 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:41.673206    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:41.752087 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:41.752029    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:41.758213 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:41.758170    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-dd94759f-6lss7"]
Jul 24 17:11:41.780166 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:41.780130    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-dd94759f-6lss7"]
Jul 24 17:11:41.780334 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:41.780282    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-dd94759f-6lss7"
Jul 24 17:11:41.783643 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:41.782898    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 24 17:11:41.783643 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:41.783157    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-g5q8h\""
Jul 24 17:11:41.783643 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:41.783228    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 24 17:11:41.783643 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:41.783364    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 24 17:11:41.783643 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:41.783474    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 24 17:11:41.783643 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:41.783527    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 24 17:11:41.783914 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:41.783718    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-71202a644018p\""
Jul 24 17:11:41.807604 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:41.807565    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:41.874733 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:41.874682    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cce52072-dc7f-43b7-ac2d-318e50963272-metrics-client-ca\") pod \"thanos-querier-dd94759f-6lss7\" (UID: \"cce52072-dc7f-43b7-ac2d-318e50963272\") " pod="openshift-monitoring/thanos-querier-dd94759f-6lss7"
Jul 24 17:11:41.874924 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:41.874747    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cce52072-dc7f-43b7-ac2d-318e50963272-secret-grpc-tls\") pod \"thanos-querier-dd94759f-6lss7\" (UID: \"cce52072-dc7f-43b7-ac2d-318e50963272\") " pod="openshift-monitoring/thanos-querier-dd94759f-6lss7"
Jul 24 17:11:41.874924 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:41.874831    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngfr8\" (UniqueName: \"kubernetes.io/projected/cce52072-dc7f-43b7-ac2d-318e50963272-kube-api-access-ngfr8\") pod \"thanos-querier-dd94759f-6lss7\" (UID: \"cce52072-dc7f-43b7-ac2d-318e50963272\") " pod="openshift-monitoring/thanos-querier-dd94759f-6lss7"
Jul 24 17:11:41.874924 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:41.874879    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/cce52072-dc7f-43b7-ac2d-318e50963272-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-dd94759f-6lss7\" (UID: \"cce52072-dc7f-43b7-ac2d-318e50963272\") " pod="openshift-monitoring/thanos-querier-dd94759f-6lss7"
Jul 24 17:11:41.874924 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:41.874915    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cce52072-dc7f-43b7-ac2d-318e50963272-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-dd94759f-6lss7\" (UID: \"cce52072-dc7f-43b7-ac2d-318e50963272\") " pod="openshift-monitoring/thanos-querier-dd94759f-6lss7"
Jul 24 17:11:41.875159 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:41.874971    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/cce52072-dc7f-43b7-ac2d-318e50963272-secret-thanos-querier-tls\") pod \"thanos-querier-dd94759f-6lss7\" (UID: \"cce52072-dc7f-43b7-ac2d-318e50963272\") " pod="openshift-monitoring/thanos-querier-dd94759f-6lss7"
Jul 24 17:11:41.875159 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:41.875007    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cce52072-dc7f-43b7-ac2d-318e50963272-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-dd94759f-6lss7\" (UID: \"cce52072-dc7f-43b7-ac2d-318e50963272\") " pod="openshift-monitoring/thanos-querier-dd94759f-6lss7"
Jul 24 17:11:41.875159 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:41.875033    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/cce52072-dc7f-43b7-ac2d-318e50963272-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-dd94759f-6lss7\" (UID: \"cce52072-dc7f-43b7-ac2d-318e50963272\") " pod="openshift-monitoring/thanos-querier-dd94759f-6lss7"
Jul 24 17:11:41.935231 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:41.935145    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:41.976300 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:41.976267    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/cce52072-dc7f-43b7-ac2d-318e50963272-secret-thanos-querier-tls\") pod \"thanos-querier-dd94759f-6lss7\" (UID: \"cce52072-dc7f-43b7-ac2d-318e50963272\") " pod="openshift-monitoring/thanos-querier-dd94759f-6lss7"
Jul 24 17:11:41.976300 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:41.976309    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cce52072-dc7f-43b7-ac2d-318e50963272-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-dd94759f-6lss7\" (UID: \"cce52072-dc7f-43b7-ac2d-318e50963272\") " pod="openshift-monitoring/thanos-querier-dd94759f-6lss7"
Jul 24 17:11:41.976542 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:41.976337    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/cce52072-dc7f-43b7-ac2d-318e50963272-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-dd94759f-6lss7\" (UID: \"cce52072-dc7f-43b7-ac2d-318e50963272\") " pod="openshift-monitoring/thanos-querier-dd94759f-6lss7"
Jul 24 17:11:41.976542 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:41.976384    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cce52072-dc7f-43b7-ac2d-318e50963272-metrics-client-ca\") pod \"thanos-querier-dd94759f-6lss7\" (UID: \"cce52072-dc7f-43b7-ac2d-318e50963272\") " pod="openshift-monitoring/thanos-querier-dd94759f-6lss7"
Jul 24 17:11:41.976542 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:41.976427    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cce52072-dc7f-43b7-ac2d-318e50963272-secret-grpc-tls\") pod \"thanos-querier-dd94759f-6lss7\" (UID: \"cce52072-dc7f-43b7-ac2d-318e50963272\") " pod="openshift-monitoring/thanos-querier-dd94759f-6lss7"
Jul 24 17:11:41.976703 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:41.976536    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngfr8\" (UniqueName: \"kubernetes.io/projected/cce52072-dc7f-43b7-ac2d-318e50963272-kube-api-access-ngfr8\") pod \"thanos-querier-dd94759f-6lss7\" (UID: \"cce52072-dc7f-43b7-ac2d-318e50963272\") " pod="openshift-monitoring/thanos-querier-dd94759f-6lss7"
Jul 24 17:11:41.976703 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:41.976589    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/cce52072-dc7f-43b7-ac2d-318e50963272-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-dd94759f-6lss7\" (UID: \"cce52072-dc7f-43b7-ac2d-318e50963272\") " pod="openshift-monitoring/thanos-querier-dd94759f-6lss7"
Jul 24 17:11:41.976703 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:41.976626    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cce52072-dc7f-43b7-ac2d-318e50963272-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-dd94759f-6lss7\" (UID: \"cce52072-dc7f-43b7-ac2d-318e50963272\") " pod="openshift-monitoring/thanos-querier-dd94759f-6lss7"
Jul 24 17:11:41.977824 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:41.977767    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cce52072-dc7f-43b7-ac2d-318e50963272-metrics-client-ca\") pod \"thanos-querier-dd94759f-6lss7\" (UID: \"cce52072-dc7f-43b7-ac2d-318e50963272\") " pod="openshift-monitoring/thanos-querier-dd94759f-6lss7"
Jul 24 17:11:41.979560 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:41.979534    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/cce52072-dc7f-43b7-ac2d-318e50963272-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-dd94759f-6lss7\" (UID: \"cce52072-dc7f-43b7-ac2d-318e50963272\") " pod="openshift-monitoring/thanos-querier-dd94759f-6lss7"
Jul 24 17:11:41.979708 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:41.979685    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/cce52072-dc7f-43b7-ac2d-318e50963272-secret-thanos-querier-tls\") pod \"thanos-querier-dd94759f-6lss7\" (UID: \"cce52072-dc7f-43b7-ac2d-318e50963272\") " pod="openshift-monitoring/thanos-querier-dd94759f-6lss7"
Jul 24 17:11:41.980111 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:41.980084    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/cce52072-dc7f-43b7-ac2d-318e50963272-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-dd94759f-6lss7\" (UID: \"cce52072-dc7f-43b7-ac2d-318e50963272\") " pod="openshift-monitoring/thanos-querier-dd94759f-6lss7"
Jul 24 17:11:41.980208 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:41.980173    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cce52072-dc7f-43b7-ac2d-318e50963272-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-dd94759f-6lss7\" (UID: \"cce52072-dc7f-43b7-ac2d-318e50963272\") " pod="openshift-monitoring/thanos-querier-dd94759f-6lss7"
Jul 24 17:11:41.980332 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:41.980312    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cce52072-dc7f-43b7-ac2d-318e50963272-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-dd94759f-6lss7\" (UID: \"cce52072-dc7f-43b7-ac2d-318e50963272\") " pod="openshift-monitoring/thanos-querier-dd94759f-6lss7"
Jul 24 17:11:41.980424 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:41.980333    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cce52072-dc7f-43b7-ac2d-318e50963272-secret-grpc-tls\") pod \"thanos-querier-dd94759f-6lss7\" (UID: \"cce52072-dc7f-43b7-ac2d-318e50963272\") " pod="openshift-monitoring/thanos-querier-dd94759f-6lss7"
Jul 24 17:11:41.985694 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:41.985671    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngfr8\" (UniqueName: \"kubernetes.io/projected/cce52072-dc7f-43b7-ac2d-318e50963272-kube-api-access-ngfr8\") pod \"thanos-querier-dd94759f-6lss7\" (UID: \"cce52072-dc7f-43b7-ac2d-318e50963272\") " pod="openshift-monitoring/thanos-querier-dd94759f-6lss7"
Jul 24 17:11:42.091497 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:42.091458    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-dd94759f-6lss7"
Jul 24 17:11:43.174769 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.174730    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5dc5d7846d-8hxgs"]
Jul 24 17:11:43.177207 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.177187    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5dc5d7846d-8hxgs"
Jul 24 17:11:43.181919 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.181655    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-54d7ujtsoidf2\""
Jul 24 17:11:43.181919 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.181672    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 24 17:11:43.181919 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.181728    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 24 17:11:43.181919 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.181755    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 24 17:11:43.181919 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.181814    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 24 17:11:43.182306 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.182021    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-gpzz9\""
Jul 24 17:11:43.189839 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.189818    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5dc5d7846d-8hxgs"]
Jul 24 17:11:43.288473 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.288438    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e076379a-b0e0-4fa9-9cef-5180aede4d56-audit-log\") pod \"metrics-server-5dc5d7846d-8hxgs\" (UID: \"e076379a-b0e0-4fa9-9cef-5180aede4d56\") " pod="openshift-monitoring/metrics-server-5dc5d7846d-8hxgs"
Jul 24 17:11:43.288647 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.288481    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e076379a-b0e0-4fa9-9cef-5180aede4d56-client-ca-bundle\") pod \"metrics-server-5dc5d7846d-8hxgs\" (UID: \"e076379a-b0e0-4fa9-9cef-5180aede4d56\") " pod="openshift-monitoring/metrics-server-5dc5d7846d-8hxgs"
Jul 24 17:11:43.288647 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.288511    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e076379a-b0e0-4fa9-9cef-5180aede4d56-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5dc5d7846d-8hxgs\" (UID: \"e076379a-b0e0-4fa9-9cef-5180aede4d56\") " pod="openshift-monitoring/metrics-server-5dc5d7846d-8hxgs"
Jul 24 17:11:43.288647 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.288533    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e076379a-b0e0-4fa9-9cef-5180aede4d56-metrics-server-audit-profiles\") pod \"metrics-server-5dc5d7846d-8hxgs\" (UID: \"e076379a-b0e0-4fa9-9cef-5180aede4d56\") " pod="openshift-monitoring/metrics-server-5dc5d7846d-8hxgs"
Jul 24 17:11:43.288764 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.288648    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e076379a-b0e0-4fa9-9cef-5180aede4d56-secret-metrics-server-client-certs\") pod \"metrics-server-5dc5d7846d-8hxgs\" (UID: \"e076379a-b0e0-4fa9-9cef-5180aede4d56\") " pod="openshift-monitoring/metrics-server-5dc5d7846d-8hxgs"
Jul 24 17:11:43.288764 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.288694    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e076379a-b0e0-4fa9-9cef-5180aede4d56-secret-metrics-server-tls\") pod \"metrics-server-5dc5d7846d-8hxgs\" (UID: \"e076379a-b0e0-4fa9-9cef-5180aede4d56\") " pod="openshift-monitoring/metrics-server-5dc5d7846d-8hxgs"
Jul 24 17:11:43.288764 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.288724    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5gjs\" (UniqueName: \"kubernetes.io/projected/e076379a-b0e0-4fa9-9cef-5180aede4d56-kube-api-access-p5gjs\") pod \"metrics-server-5dc5d7846d-8hxgs\" (UID: \"e076379a-b0e0-4fa9-9cef-5180aede4d56\") " pod="openshift-monitoring/metrics-server-5dc5d7846d-8hxgs"
Jul 24 17:11:43.389862 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.389833    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e076379a-b0e0-4fa9-9cef-5180aede4d56-audit-log\") pod \"metrics-server-5dc5d7846d-8hxgs\" (UID: \"e076379a-b0e0-4fa9-9cef-5180aede4d56\") " pod="openshift-monitoring/metrics-server-5dc5d7846d-8hxgs"
Jul 24 17:11:43.389862 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.389866    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e076379a-b0e0-4fa9-9cef-5180aede4d56-client-ca-bundle\") pod \"metrics-server-5dc5d7846d-8hxgs\" (UID: \"e076379a-b0e0-4fa9-9cef-5180aede4d56\") " pod="openshift-monitoring/metrics-server-5dc5d7846d-8hxgs"
Jul 24 17:11:43.390108 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.389884    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e076379a-b0e0-4fa9-9cef-5180aede4d56-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5dc5d7846d-8hxgs\" (UID: \"e076379a-b0e0-4fa9-9cef-5180aede4d56\") " pod="openshift-monitoring/metrics-server-5dc5d7846d-8hxgs"
Jul 24 17:11:43.390108 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.389900    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e076379a-b0e0-4fa9-9cef-5180aede4d56-metrics-server-audit-profiles\") pod \"metrics-server-5dc5d7846d-8hxgs\" (UID: \"e076379a-b0e0-4fa9-9cef-5180aede4d56\") " pod="openshift-monitoring/metrics-server-5dc5d7846d-8hxgs"
Jul 24 17:11:43.390108 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.389955    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e076379a-b0e0-4fa9-9cef-5180aede4d56-secret-metrics-server-client-certs\") pod \"metrics-server-5dc5d7846d-8hxgs\" (UID: \"e076379a-b0e0-4fa9-9cef-5180aede4d56\") " pod="openshift-monitoring/metrics-server-5dc5d7846d-8hxgs"
Jul 24 17:11:43.390108 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.389992    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e076379a-b0e0-4fa9-9cef-5180aede4d56-secret-metrics-server-tls\") pod \"metrics-server-5dc5d7846d-8hxgs\" (UID: \"e076379a-b0e0-4fa9-9cef-5180aede4d56\") " pod="openshift-monitoring/metrics-server-5dc5d7846d-8hxgs"
Jul 24 17:11:43.390108 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.390027    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5gjs\" (UniqueName: \"kubernetes.io/projected/e076379a-b0e0-4fa9-9cef-5180aede4d56-kube-api-access-p5gjs\") pod \"metrics-server-5dc5d7846d-8hxgs\" (UID: \"e076379a-b0e0-4fa9-9cef-5180aede4d56\") " pod="openshift-monitoring/metrics-server-5dc5d7846d-8hxgs"
Jul 24 17:11:43.390344 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.390321    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e076379a-b0e0-4fa9-9cef-5180aede4d56-audit-log\") pod \"metrics-server-5dc5d7846d-8hxgs\" (UID: \"e076379a-b0e0-4fa9-9cef-5180aede4d56\") " pod="openshift-monitoring/metrics-server-5dc5d7846d-8hxgs"
Jul 24 17:11:43.390976 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.390923    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e076379a-b0e0-4fa9-9cef-5180aede4d56-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5dc5d7846d-8hxgs\" (UID: \"e076379a-b0e0-4fa9-9cef-5180aede4d56\") " pod="openshift-monitoring/metrics-server-5dc5d7846d-8hxgs"
Jul 24 17:11:43.391130 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.391067    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e076379a-b0e0-4fa9-9cef-5180aede4d56-metrics-server-audit-profiles\") pod \"metrics-server-5dc5d7846d-8hxgs\" (UID: \"e076379a-b0e0-4fa9-9cef-5180aede4d56\") " pod="openshift-monitoring/metrics-server-5dc5d7846d-8hxgs"
Jul 24 17:11:43.393222 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.393194    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e076379a-b0e0-4fa9-9cef-5180aede4d56-secret-metrics-server-tls\") pod \"metrics-server-5dc5d7846d-8hxgs\" (UID: \"e076379a-b0e0-4fa9-9cef-5180aede4d56\") " pod="openshift-monitoring/metrics-server-5dc5d7846d-8hxgs"
Jul 24 17:11:43.393348 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.393259    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e076379a-b0e0-4fa9-9cef-5180aede4d56-secret-metrics-server-client-certs\") pod \"metrics-server-5dc5d7846d-8hxgs\" (UID: \"e076379a-b0e0-4fa9-9cef-5180aede4d56\") " pod="openshift-monitoring/metrics-server-5dc5d7846d-8hxgs"
Jul 24 17:11:43.393348 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.393271    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e076379a-b0e0-4fa9-9cef-5180aede4d56-client-ca-bundle\") pod \"metrics-server-5dc5d7846d-8hxgs\" (UID: \"e076379a-b0e0-4fa9-9cef-5180aede4d56\") " pod="openshift-monitoring/metrics-server-5dc5d7846d-8hxgs"
Jul 24 17:11:43.401582 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.401560    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5gjs\" (UniqueName: \"kubernetes.io/projected/e076379a-b0e0-4fa9-9cef-5180aede4d56-kube-api-access-p5gjs\") pod \"metrics-server-5dc5d7846d-8hxgs\" (UID: \"e076379a-b0e0-4fa9-9cef-5180aede4d56\") " pod="openshift-monitoring/metrics-server-5dc5d7846d-8hxgs"
Jul 24 17:11:43.487991 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.487898    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5dc5d7846d-8hxgs"
Jul 24 17:11:43.540827 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.540794    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:43.550818 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.550791    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-m72fg"]
Jul 24 17:11:43.555006 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.554984    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-m72fg"
Jul 24 17:11:43.557736 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.557716    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-dbgxx\""
Jul 24 17:11:43.557867 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.557847    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 24 17:11:43.566766 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.566711    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-m72fg"]
Jul 24 17:11:43.637349 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.637319    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:43.687139 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.687106    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:43.692777 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.692751    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29ql7\" (UniqueName: \"kubernetes.io/projected/1aaa688e-4aad-4120-b7f0-64f7574e4aad-kube-api-access-29ql7\") pod \"monitoring-plugin-68b7fc9574-m72fg\" (UID: \"1aaa688e-4aad-4120-b7f0-64f7574e4aad\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-m72fg"
Jul 24 17:11:43.692957 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.692851    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1aaa688e-4aad-4120-b7f0-64f7574e4aad-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-m72fg\" (UID: \"1aaa688e-4aad-4120-b7f0-64f7574e4aad\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-m72fg"
Jul 24 17:11:43.793728 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.793630    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1aaa688e-4aad-4120-b7f0-64f7574e4aad-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-m72fg\" (UID: \"1aaa688e-4aad-4120-b7f0-64f7574e4aad\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-m72fg"
Jul 24 17:11:43.793728 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.793697    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29ql7\" (UniqueName: \"kubernetes.io/projected/1aaa688e-4aad-4120-b7f0-64f7574e4aad-kube-api-access-29ql7\") pod \"monitoring-plugin-68b7fc9574-m72fg\" (UID: \"1aaa688e-4aad-4120-b7f0-64f7574e4aad\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-m72fg"
Jul 24 17:11:43.793934 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:43.793802    2581 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 24 17:11:43.793934 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:11:43.793884    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1aaa688e-4aad-4120-b7f0-64f7574e4aad-monitoring-plugin-cert podName:1aaa688e-4aad-4120-b7f0-64f7574e4aad nodeName:}" failed. No retries permitted until 2026-07-24 17:11:44.293863014 +0000 UTC m=+71.495912266 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/1aaa688e-4aad-4120-b7f0-64f7574e4aad-monitoring-plugin-cert") pod "monitoring-plugin-68b7fc9574-m72fg" (UID: "1aaa688e-4aad-4120-b7f0-64f7574e4aad") : secret "monitoring-plugin-cert" not found
Jul 24 17:11:43.805019 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.804991    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29ql7\" (UniqueName: \"kubernetes.io/projected/1aaa688e-4aad-4120-b7f0-64f7574e4aad-kube-api-access-29ql7\") pod \"monitoring-plugin-68b7fc9574-m72fg\" (UID: \"1aaa688e-4aad-4120-b7f0-64f7574e4aad\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-m72fg"
Jul 24 17:11:43.844327 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.844287    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:43.978907 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.978870    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7569b5b876-z2trz"]
Jul 24 17:11:43.981749 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.981727    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7569b5b876-z2trz"
Jul 24 17:11:43.987584 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.987564    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 24 17:11:43.987874 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.987855    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 24 17:11:43.988699 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.988678    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 24 17:11:43.988840 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.988711    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-5nrnl\""
Jul 24 17:11:43.988840 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.988759    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 24 17:11:43.989006 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.988988    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 24 17:11:43.998812 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:43.998787    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 24 17:11:44.008496 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.008471    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7569b5b876-z2trz"]
Jul 24 17:11:44.095991 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.095895    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/31e30a4b-cb9c-42ea-a5c0-484a75fe6a12-telemeter-client-tls\") pod \"telemeter-client-7569b5b876-z2trz\" (UID: \"31e30a4b-cb9c-42ea-a5c0-484a75fe6a12\") " pod="openshift-monitoring/telemeter-client-7569b5b876-z2trz"
Jul 24 17:11:44.095991 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.095936    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31e30a4b-cb9c-42ea-a5c0-484a75fe6a12-serving-certs-ca-bundle\") pod \"telemeter-client-7569b5b876-z2trz\" (UID: \"31e30a4b-cb9c-42ea-a5c0-484a75fe6a12\") " pod="openshift-monitoring/telemeter-client-7569b5b876-z2trz"
Jul 24 17:11:44.096240 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.096014    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8758r\" (UniqueName: \"kubernetes.io/projected/31e30a4b-cb9c-42ea-a5c0-484a75fe6a12-kube-api-access-8758r\") pod \"telemeter-client-7569b5b876-z2trz\" (UID: \"31e30a4b-cb9c-42ea-a5c0-484a75fe6a12\") " pod="openshift-monitoring/telemeter-client-7569b5b876-z2trz"
Jul 24 17:11:44.096240 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.096108    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31e30a4b-cb9c-42ea-a5c0-484a75fe6a12-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7569b5b876-z2trz\" (UID: \"31e30a4b-cb9c-42ea-a5c0-484a75fe6a12\") " pod="openshift-monitoring/telemeter-client-7569b5b876-z2trz"
Jul 24 17:11:44.096240 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.096166    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/31e30a4b-cb9c-42ea-a5c0-484a75fe6a12-secret-telemeter-client\") pod \"telemeter-client-7569b5b876-z2trz\" (UID: \"31e30a4b-cb9c-42ea-a5c0-484a75fe6a12\") " pod="openshift-monitoring/telemeter-client-7569b5b876-z2trz"
Jul 24 17:11:44.096240 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.096201    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/31e30a4b-cb9c-42ea-a5c0-484a75fe6a12-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7569b5b876-z2trz\" (UID: \"31e30a4b-cb9c-42ea-a5c0-484a75fe6a12\") " pod="openshift-monitoring/telemeter-client-7569b5b876-z2trz"
Jul 24 17:11:44.096453 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.096244    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/31e30a4b-cb9c-42ea-a5c0-484a75fe6a12-federate-client-tls\") pod \"telemeter-client-7569b5b876-z2trz\" (UID: \"31e30a4b-cb9c-42ea-a5c0-484a75fe6a12\") " pod="openshift-monitoring/telemeter-client-7569b5b876-z2trz"
Jul 24 17:11:44.096453 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.096346    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/31e30a4b-cb9c-42ea-a5c0-484a75fe6a12-metrics-client-ca\") pod \"telemeter-client-7569b5b876-z2trz\" (UID: \"31e30a4b-cb9c-42ea-a5c0-484a75fe6a12\") " pod="openshift-monitoring/telemeter-client-7569b5b876-z2trz"
Jul 24 17:11:44.197568 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.197531    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/31e30a4b-cb9c-42ea-a5c0-484a75fe6a12-secret-telemeter-client\") pod \"telemeter-client-7569b5b876-z2trz\" (UID: \"31e30a4b-cb9c-42ea-a5c0-484a75fe6a12\") " pod="openshift-monitoring/telemeter-client-7569b5b876-z2trz"
Jul 24 17:11:44.197568 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.197572    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/31e30a4b-cb9c-42ea-a5c0-484a75fe6a12-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7569b5b876-z2trz\" (UID: \"31e30a4b-cb9c-42ea-a5c0-484a75fe6a12\") " pod="openshift-monitoring/telemeter-client-7569b5b876-z2trz"
Jul 24 17:11:44.198132 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.197607    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/31e30a4b-cb9c-42ea-a5c0-484a75fe6a12-federate-client-tls\") pod \"telemeter-client-7569b5b876-z2trz\" (UID: \"31e30a4b-cb9c-42ea-a5c0-484a75fe6a12\") " pod="openshift-monitoring/telemeter-client-7569b5b876-z2trz"
Jul 24 17:11:44.198132 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.197669    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/31e30a4b-cb9c-42ea-a5c0-484a75fe6a12-metrics-client-ca\") pod \"telemeter-client-7569b5b876-z2trz\" (UID: \"31e30a4b-cb9c-42ea-a5c0-484a75fe6a12\") " pod="openshift-monitoring/telemeter-client-7569b5b876-z2trz"
Jul 24 17:11:44.198132 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.197715    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/31e30a4b-cb9c-42ea-a5c0-484a75fe6a12-telemeter-client-tls\") pod \"telemeter-client-7569b5b876-z2trz\" (UID: \"31e30a4b-cb9c-42ea-a5c0-484a75fe6a12\") " pod="openshift-monitoring/telemeter-client-7569b5b876-z2trz"
Jul 24 17:11:44.198132 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.197746    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31e30a4b-cb9c-42ea-a5c0-484a75fe6a12-serving-certs-ca-bundle\") pod \"telemeter-client-7569b5b876-z2trz\" (UID: \"31e30a4b-cb9c-42ea-a5c0-484a75fe6a12\") " pod="openshift-monitoring/telemeter-client-7569b5b876-z2trz"
Jul 24 17:11:44.198132 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.197775    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8758r\" (UniqueName: \"kubernetes.io/projected/31e30a4b-cb9c-42ea-a5c0-484a75fe6a12-kube-api-access-8758r\") pod \"telemeter-client-7569b5b876-z2trz\" (UID: \"31e30a4b-cb9c-42ea-a5c0-484a75fe6a12\") " pod="openshift-monitoring/telemeter-client-7569b5b876-z2trz"
Jul 24 17:11:44.198132 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.197800    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31e30a4b-cb9c-42ea-a5c0-484a75fe6a12-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7569b5b876-z2trz\" (UID: \"31e30a4b-cb9c-42ea-a5c0-484a75fe6a12\") " pod="openshift-monitoring/telemeter-client-7569b5b876-z2trz"
Jul 24 17:11:44.199372 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.198692    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31e30a4b-cb9c-42ea-a5c0-484a75fe6a12-serving-certs-ca-bundle\") pod \"telemeter-client-7569b5b876-z2trz\" (UID: \"31e30a4b-cb9c-42ea-a5c0-484a75fe6a12\") " pod="openshift-monitoring/telemeter-client-7569b5b876-z2trz"
Jul 24 17:11:44.199372 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.198717    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31e30a4b-cb9c-42ea-a5c0-484a75fe6a12-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7569b5b876-z2trz\" (UID: \"31e30a4b-cb9c-42ea-a5c0-484a75fe6a12\") " pod="openshift-monitoring/telemeter-client-7569b5b876-z2trz"
Jul 24 17:11:44.199372 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.199303    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/31e30a4b-cb9c-42ea-a5c0-484a75fe6a12-metrics-client-ca\") pod \"telemeter-client-7569b5b876-z2trz\" (UID: \"31e30a4b-cb9c-42ea-a5c0-484a75fe6a12\") " pod="openshift-monitoring/telemeter-client-7569b5b876-z2trz"
Jul 24 17:11:44.200672 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.200644    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/31e30a4b-cb9c-42ea-a5c0-484a75fe6a12-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7569b5b876-z2trz\" (UID: \"31e30a4b-cb9c-42ea-a5c0-484a75fe6a12\") " pod="openshift-monitoring/telemeter-client-7569b5b876-z2trz"
Jul 24 17:11:44.200672 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.200666    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/31e30a4b-cb9c-42ea-a5c0-484a75fe6a12-secret-telemeter-client\") pod \"telemeter-client-7569b5b876-z2trz\" (UID: \"31e30a4b-cb9c-42ea-a5c0-484a75fe6a12\") " pod="openshift-monitoring/telemeter-client-7569b5b876-z2trz"
Jul 24 17:11:44.200834 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.200695    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/31e30a4b-cb9c-42ea-a5c0-484a75fe6a12-federate-client-tls\") pod \"telemeter-client-7569b5b876-z2trz\" (UID: \"31e30a4b-cb9c-42ea-a5c0-484a75fe6a12\") " pod="openshift-monitoring/telemeter-client-7569b5b876-z2trz"
Jul 24 17:11:44.202210 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.202190    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/31e30a4b-cb9c-42ea-a5c0-484a75fe6a12-telemeter-client-tls\") pod \"telemeter-client-7569b5b876-z2trz\" (UID: \"31e30a4b-cb9c-42ea-a5c0-484a75fe6a12\") " pod="openshift-monitoring/telemeter-client-7569b5b876-z2trz"
Jul 24 17:11:44.208237 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.208198    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8758r\" (UniqueName: \"kubernetes.io/projected/31e30a4b-cb9c-42ea-a5c0-484a75fe6a12-kube-api-access-8758r\") pod \"telemeter-client-7569b5b876-z2trz\" (UID: \"31e30a4b-cb9c-42ea-a5c0-484a75fe6a12\") " pod="openshift-monitoring/telemeter-client-7569b5b876-z2trz"
Jul 24 17:11:44.293063 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.293018    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7569b5b876-z2trz"
Jul 24 17:11:44.299198 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.299171    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1aaa688e-4aad-4120-b7f0-64f7574e4aad-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-m72fg\" (UID: \"1aaa688e-4aad-4120-b7f0-64f7574e4aad\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-m72fg"
Jul 24 17:11:44.301635 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.301616    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1aaa688e-4aad-4120-b7f0-64f7574e4aad-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-m72fg\" (UID: \"1aaa688e-4aad-4120-b7f0-64f7574e4aad\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-m72fg"
Jul 24 17:11:44.467857 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.467822    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-m72fg"
Jul 24 17:11:44.932578 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.932507    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:44.940925 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.940893    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 17:11:44.948073 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.948021    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:44.952276 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.952248    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 24 17:11:44.952443 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.952415    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 24 17:11:44.952443 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.952322    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 24 17:11:44.952607 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.952415    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 24 17:11:44.952607 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.952425    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 24 17:11:44.952607 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.952322    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 24 17:11:44.953322 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.953303    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 24 17:11:44.953393 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.953319    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-8db87\""
Jul 24 17:11:44.953577 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.953563    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 24 17:11:44.953800 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.953787    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 24 17:11:44.953910 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.953884    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 24 17:11:44.954008 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.953980    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 24 17:11:44.954138 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.954121    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 24 17:11:44.954233 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.954216    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 24 17:11:44.954465 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.954434    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-28kkn82cat3cn\""
Jul 24 17:11:44.955596 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.955370    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 24 17:11:44.963062 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:44.963019    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 17:11:45.005026 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.004989    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.005026 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.005025    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-config\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.005288 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.005082    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.005288 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.005124    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pg7hv\" (UniqueName: \"kubernetes.io/projected/2d8f8104-653f-4c52-850b-56e496c30884-kube-api-access-pg7hv\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.005288 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.005160    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2d8f8104-653f-4c52-850b-56e496c30884-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.005288 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.005206    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2d8f8104-653f-4c52-850b-56e496c30884-config-out\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.005288 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.005231    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2d8f8104-653f-4c52-850b-56e496c30884-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.005288 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.005254    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.005288 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.005282    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.005630 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.005309    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.005630 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.005338    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.005630 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.005380    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.005630 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.005407    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-web-config\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.005630 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.005443    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.005630 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.005472    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.005630 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.005496    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.005630 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.005522    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.005630 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.005559    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.005630 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.005585    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.006017 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.005645    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.011256 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.011201    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:45.107337 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.107282    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2d8f8104-653f-4c52-850b-56e496c30884-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.107565 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.107380    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2d8f8104-653f-4c52-850b-56e496c30884-config-out\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.107565 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.107417    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2d8f8104-653f-4c52-850b-56e496c30884-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.107565 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.107452    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.107565 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.107482    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.107565 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.107517    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.107565 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.107551    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.108012 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.107597    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.108012 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.107630    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-web-config\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.108012 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.107666    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.108012 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.107702    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.108012 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.107728    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.108012 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.107764    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.108799 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.108774    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.110586 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.110239    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.118226 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.112715    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.118226 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.112784    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.118226 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.112830    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.118226 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.112936    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.118226 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.112978    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-config\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.118226 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.113219    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.118226 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.113256    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pg7hv\" (UniqueName: \"kubernetes.io/projected/2d8f8104-653f-4c52-850b-56e496c30884-kube-api-access-pg7hv\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.118226 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.113525    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.118226 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.115174    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2d8f8104-653f-4c52-850b-56e496c30884-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.118226 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.115156    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.118226 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.116061    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.118226 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.116275    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:45.118226 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.116228    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.121384 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.119258    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.121384 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.119754    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.121384 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.120474    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.121384 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.120938    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.124729 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.123460    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-config\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.124952 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.124922    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2d8f8104-653f-4c52-850b-56e496c30884-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.125420 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.125395    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pg7hv\" (UniqueName: \"kubernetes.io/projected/2d8f8104-653f-4c52-850b-56e496c30884-kube-api-access-pg7hv\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.127738 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.127698    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-web-config\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.128157 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.128133    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.128416 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.128361    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.128680 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.128660    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2d8f8104-653f-4c52-850b-56e496c30884-config-out\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.129080 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.129062    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.131469 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.131437    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:45.264447 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:45.264367    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:11:46.329622 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:46.329586    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:46.429565 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:46.429530    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:47.127905 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:47.127766    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:47.195348 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:47.194877    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:47.315450 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:47.312669    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-8pb55"]
Jul 24 17:11:47.320784 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:11:47.320749    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2596065_10a4_4cca_bdca_04d5aa1208a2.slice/crio-9b8f180ba001670f5bfe964913e8024217d2031c86fe34715b4c0bf9614c7824 WatchSource:0}: Error finding container 9b8f180ba001670f5bfe964913e8024217d2031c86fe34715b4c0bf9614c7824: Status 404 returned error can't find the container with id 9b8f180ba001670f5bfe964913e8024217d2031c86fe34715b4c0bf9614c7824
Jul 24 17:11:47.336624 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:47.336554    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56ffd5f7b5-vvmhd"]
Jul 24 17:11:47.340480 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:47.340356    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-dd94759f-6lss7"]
Jul 24 17:11:47.342600 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:47.342575    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7569b5b876-z2trz"]
Jul 24 17:11:47.427719 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:47.427697    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 17:11:47.429028 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:11:47.429000    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod284bf1c9_f105_4053_b478_b8bd4fd6625f.slice/crio-4e3f8e2ba15d389fd5d43fe620e044c6c43ed2313fa9e7de9e2d0dc77b2fd7c4 WatchSource:0}: Error finding container 4e3f8e2ba15d389fd5d43fe620e044c6c43ed2313fa9e7de9e2d0dc77b2fd7c4: Status 404 returned error can't find the container with id 4e3f8e2ba15d389fd5d43fe620e044c6c43ed2313fa9e7de9e2d0dc77b2fd7c4
Jul 24 17:11:47.570672 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:47.568114    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-m72fg"]
Jul 24 17:11:47.578650 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:47.578379    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5dc5d7846d-8hxgs"]
Jul 24 17:11:47.579143 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:11:47.578792    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aaa688e_4aad_4120_b7f0_64f7574e4aad.slice/crio-42c074fe7b1ff0bec8254e286655d48d16471dd7cd19546d4c8464192fcab095 WatchSource:0}: Error finding container 42c074fe7b1ff0bec8254e286655d48d16471dd7cd19546d4c8464192fcab095: Status 404 returned error can't find the container with id 42c074fe7b1ff0bec8254e286655d48d16471dd7cd19546d4c8464192fcab095
Jul 24 17:11:47.579143 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:47.579064    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-f9qj2"]
Jul 24 17:11:47.580268 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:11:47.580235    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode076379a_b0e0_4fa9_9cef_5180aede4d56.slice/crio-9ac49ead85c12d31a9e1f8234e26bc284dd200affe778ebaa7c90b49de5b1e95 WatchSource:0}: Error finding container 9ac49ead85c12d31a9e1f8234e26bc284dd200affe778ebaa7c90b49de5b1e95: Status 404 returned error can't find the container with id 9ac49ead85c12d31a9e1f8234e26bc284dd200affe778ebaa7c90b49de5b1e95
Jul 24 17:11:47.583210 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:47.582872    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 17:11:47.787406 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:47.787259    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d8f8104-653f-4c52-850b-56e496c30884","Type":"ContainerStarted","Data":"b6995ad6a876c419645ebbd238422c58a963062cf5ff3f6004bf65a169e9c314"}
Jul 24 17:11:47.796054 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:47.795262    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-wf48c" event={"ID":"d625ce61-acef-427e-9d65-75720151fcd7","Type":"ContainerStarted","Data":"bb77773815d989da8cf61ec34a2a173288a4ce573d6ffc65d4fb1bad0929d51d"}
Jul 24 17:11:47.796267 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:47.796222    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-wf48c"
Jul 24 17:11:47.799150 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:47.798647    2581 patch_prober.go:28] interesting pod/downloads-7c97dc4555-wf48c container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.134.0.10:8080/\": dial tcp 10.134.0.10:8080: connect: connection refused" start-of-body=
Jul 24 17:11:47.799150 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:47.798695    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-7c97dc4555-wf48c" podUID="d625ce61-acef-427e-9d65-75720151fcd7" containerName="download-server" probeResult="failure" output="Get \"http://10.134.0.10:8080/\": dial tcp 10.134.0.10:8080: connect: connection refused"
Jul 24 17:11:47.800280 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:47.800195    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"284bf1c9-f105-4053-b478-b8bd4fd6625f","Type":"ContainerStarted","Data":"4e3f8e2ba15d389fd5d43fe620e044c6c43ed2313fa9e7de9e2d0dc77b2fd7c4"}
Jul 24 17:11:47.803375 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:47.803311    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dvz8j" event={"ID":"d8e047ab-f0c3-48cc-84c2-19559698ab11","Type":"ContainerStarted","Data":"dfedd3bc95f413823cf69e4191e31d329b3dd2ceeb305f1b41e07b106ae35a87"}
Jul 24 17:11:47.808175 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:47.808121    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56ffd5f7b5-vvmhd" event={"ID":"b111fa18-0380-470b-9d3c-ec7992007d20","Type":"ContainerStarted","Data":"4c3a4cfa11db711e60b72ce7fafd9a9a4c7f4b6ebd29b7dd48d4ef80875fc62e"}
Jul 24 17:11:47.810961 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:47.810916    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-m72fg" event={"ID":"1aaa688e-4aad-4120-b7f0-64f7574e4aad","Type":"ContainerStarted","Data":"42c074fe7b1ff0bec8254e286655d48d16471dd7cd19546d4c8464192fcab095"}
Jul 24 17:11:47.819144 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:47.818492    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7569b5b876-z2trz" event={"ID":"31e30a4b-cb9c-42ea-a5c0-484a75fe6a12","Type":"ContainerStarted","Data":"6f873bc3c507ba0912b0c0c36380e157560df4884dccfd9ec6d09bab11cc886e"}
Jul 24 17:11:47.823428 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:47.823339    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-dd94759f-6lss7" event={"ID":"cce52072-dc7f-43b7-ac2d-318e50963272","Type":"ContainerStarted","Data":"f6a95e9ec807b2f22b59750405674714eb53e44ce22872497e91cbcb05494d54"}
Jul 24 17:11:47.825646 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:47.825595    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5dc5d7846d-8hxgs" event={"ID":"e076379a-b0e0-4fa9-9cef-5180aede4d56","Type":"ContainerStarted","Data":"9ac49ead85c12d31a9e1f8234e26bc284dd200affe778ebaa7c90b49de5b1e95"}
Jul 24 17:11:47.829791 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:47.829768    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8pb55" event={"ID":"a2596065-10a4-4cca-bdca-04d5aa1208a2","Type":"ContainerStarted","Data":"b89568c8cbd17b235718c34df9aeeef7ecbb90bffd8aa142029e5608492b987e"}
Jul 24 17:11:47.829947 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:47.829929    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8pb55" event={"ID":"a2596065-10a4-4cca-bdca-04d5aa1208a2","Type":"ContainerStarted","Data":"91f7d3274e9600198b147cc159b8f9c84e85ea87f112a8a027f906b990b17450"}
Jul 24 17:11:47.830130 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:47.830094    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8pb55" event={"ID":"a2596065-10a4-4cca-bdca-04d5aa1208a2","Type":"ContainerStarted","Data":"9b8f180ba001670f5bfe964913e8024217d2031c86fe34715b4c0bf9614c7824"}
Jul 24 17:11:47.834931 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:47.834860    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-f9qj2" event={"ID":"fac1bdfa-4c1a-480d-9b1a-e3b025e473bb","Type":"ContainerStarted","Data":"aa4533744d2255d49f0d684d103178189a60bd74f43deee58e9de4ab6c309a22"}
Jul 24 17:11:48.869023 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:48.868967    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-wf48c"
Jul 24 17:11:48.894503 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:48.892905    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-wf48c" podStartSLOduration=2.540684353 podStartE2EDuration="20.892885109s" totalImagesPullingTime="18.352200756s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:11:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:11:47.81724523 +0000 UTC m=+75.019294505" watchObservedRunningTime="2026-07-24 17:11:48.892885109 +0000 UTC m=+76.094934383"
Jul 24 17:11:49.447645 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:49.447545    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:49.540965 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:49.540672    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:52.672760 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:52.672712    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:52.772947 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:52.772871    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:53.848614 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:53.848461    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:53.913896 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:53.913849    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:55.382525 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:55.382493    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:55.436121 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:55.436088    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:11:59.732749 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:59.729954    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56ffd5f7b5-vvmhd"]
Jul 24 17:11:59.922265 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:11:59.922210    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:12:00.023105 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:00.022985    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:12:01.245799 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:01.245742    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:12:01.294461 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:01.294430    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:12:02.950262 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:02.950144    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-v42pr" event={"ID":"24d6b216-f67b-4d45-bd22-c9f159255177","Type":"ContainerStarted","Data":"dabbbcd33acb390fcd188aef2f84a48ff3983d3a56d24ee923f9d13c8a11c2c8"}
Jul 24 17:12:02.953056 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:02.952485    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d8f8104-653f-4c52-850b-56e496c30884","Type":"ContainerStarted","Data":"c16969c06171d5e3126c475a855fb259c859a2f8c35c6c4552ffe65284b20b95"}
Jul 24 17:12:02.960560 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:02.960539    2581 generic.go:356] "Generic (PLEG): container finished" podID="284bf1c9-f105-4053-b478-b8bd4fd6625f" containerID="091cd8ed99eb1a5bd646af9e4f8d62f867c20f46029a886957d5611c0a814b2e" exitCode=0
Jul 24 17:12:02.960714 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:02.960698    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"284bf1c9-f105-4053-b478-b8bd4fd6625f","Type":"ContainerDied","Data":"091cd8ed99eb1a5bd646af9e4f8d62f867c20f46029a886957d5611c0a814b2e"}
Jul 24 17:12:02.976232 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:02.976199    2581 generic.go:356] "Generic (PLEG): container finished" podID="d8e047ab-f0c3-48cc-84c2-19559698ab11" containerID="80a1fc9aa174e3d0aead0523fe9a3a17553e4cce821f7b9486599906208ee0b8" exitCode=0
Jul 24 17:12:02.976402 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:02.976327    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dvz8j" event={"ID":"d8e047ab-f0c3-48cc-84c2-19559698ab11","Type":"ContainerDied","Data":"80a1fc9aa174e3d0aead0523fe9a3a17553e4cce821f7b9486599906208ee0b8"}
Jul 24 17:12:02.985565 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:02.984874    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56ffd5f7b5-vvmhd" event={"ID":"b111fa18-0380-470b-9d3c-ec7992007d20","Type":"ContainerStarted","Data":"92497ce52b93fd0211a8d817134422b27858cba5a75ffe4e7bec8994988e3a4b"}
Jul 24 17:12:03.001055 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:02.997783    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-m72fg" event={"ID":"1aaa688e-4aad-4120-b7f0-64f7574e4aad","Type":"ContainerStarted","Data":"59e6370c2821f90b8214afb2905e58b8e3406191869a571ecd29bd28c9ab117e"}
Jul 24 17:12:03.020823 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:03.020538    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7569b5b876-z2trz" event={"ID":"31e30a4b-cb9c-42ea-a5c0-484a75fe6a12","Type":"ContainerStarted","Data":"f5ebe7e8488b34f6e237f1c3a1672c02ec66d3b065e5217b643486df1bb241e7"}
Jul 24 17:12:03.020823 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:03.020605    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7569b5b876-z2trz" event={"ID":"31e30a4b-cb9c-42ea-a5c0-484a75fe6a12","Type":"ContainerStarted","Data":"9ddbb8dbf3f35b62c704cc0e6877cc0b2f17407e769ea073c240da5479a695e5"}
Jul 24 17:12:03.022289 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:03.022243    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-m72fg"
Jul 24 17:12:03.024927 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:03.024851    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-d42hh" event={"ID":"5c99646a-9e3e-4e60-a750-a96ab5aab0bf","Type":"ContainerStarted","Data":"fc2e05daad36c7ed0fabd557356170ea31e03346efb50a72424bed72e19b73bc"}
Jul 24 17:12:03.024927 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:03.024897    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-d42hh" event={"ID":"5c99646a-9e3e-4e60-a750-a96ab5aab0bf","Type":"ContainerStarted","Data":"29f3cca99c058ae91e1ba4ab85737923ab74072afbc02c107842b897b2016078"}
Jul 24 17:12:03.031643 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:03.031519    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-m72fg"
Jul 24 17:12:03.032990 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:03.032310    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-t2znt" event={"ID":"bcf18baa-0b35-4089-9570-df68ef7a456a","Type":"ContainerStarted","Data":"b68f57aef1e2e84bd360dd11fc56739d665f11e1061bb11cf8f0d7818247783d"}
Jul 24 17:12:03.032990 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:03.032746    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:12:03.035241 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:03.034899    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-dd94759f-6lss7" event={"ID":"cce52072-dc7f-43b7-ac2d-318e50963272","Type":"ContainerStarted","Data":"50fd488c1d50cccd67ea770425db17642b56c787df119245d09d6bb85674c867"}
Jul 24 17:12:03.037888 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:03.037447    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5dc5d7846d-8hxgs" event={"ID":"e076379a-b0e0-4fa9-9cef-5180aede4d56","Type":"ContainerStarted","Data":"c7628bef333ff0b1f24f468e8a3e0f1160763fca12a5d5a4b433ca8608fbf6eb"}
Jul 24 17:12:03.049980 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:03.049434    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8pb55" event={"ID":"a2596065-10a4-4cca-bdca-04d5aa1208a2","Type":"ContainerStarted","Data":"95d69d6d3ceafe09852abb5189b11234e11ca9aae8479b20885f1841e3f20eee"}
Jul 24 17:12:03.052193 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:03.052095    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-f9qj2" event={"ID":"fac1bdfa-4c1a-480d-9b1a-e3b025e473bb","Type":"ContainerStarted","Data":"73ff14b1e354a070382c2d410bc27d04d6de7edc88b2cb602d61cabb1ea81e9e"}
Jul 24 17:12:03.052193 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:03.052147    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-f9qj2" event={"ID":"fac1bdfa-4c1a-480d-9b1a-e3b025e473bb","Type":"ContainerStarted","Data":"be1f0db343b850784016dd5fbb23221a752c83053e10bf1e454b59b84878fb72"}
Jul 24 17:12:03.070860 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:03.070312    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-v42pr" podStartSLOduration=67.065664309 podStartE2EDuration="1m30.070292347s" totalImagesPullingTime="23.004628038s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:10:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:12:03.048008512 +0000 UTC m=+90.250057787" watchObservedRunningTime="2026-07-24 17:12:03.070292347 +0000 UTC m=+90.272341616"
Jul 24 17:12:03.094621 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:03.092603    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56ffd5f7b5-vvmhd" podStartSLOduration=9.065331987 podStartE2EDuration="24.092583469s" totalImagesPullingTime="15.027251482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:11:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:12:03.090752389 +0000 UTC m=+90.292801663" watchObservedRunningTime="2026-07-24 17:12:03.092583469 +0000 UTC m=+90.294632746"
Jul 24 17:12:03.134534 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:03.133477    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5dc5d7846d-8hxgs" podStartSLOduration=5.340079948 podStartE2EDuration="20.133454062s" totalImagesPullingTime="14.793374114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:11:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:12:03.133212272 +0000 UTC m=+90.335261547" watchObservedRunningTime="2026-07-24 17:12:03.133454062 +0000 UTC m=+90.335503338"
Jul 24 17:12:03.155078 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:03.155006    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-m72fg" podStartSLOduration=5.653592514 podStartE2EDuration="20.154985697s" totalImagesPullingTime="14.501393183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:11:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:12:03.151934552 +0000 UTC m=+90.353983827" watchObservedRunningTime="2026-07-24 17:12:03.154985697 +0000 UTC m=+90.357034971"
Jul 24 17:12:03.170703 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:03.170651    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-d42hh" podStartSLOduration=67.442621348 podStartE2EDuration="1m30.170636279s" totalImagesPullingTime="22.728014931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:10:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:12:03.170584973 +0000 UTC m=+90.372634247" watchObservedRunningTime="2026-07-24 17:12:03.170636279 +0000 UTC m=+90.372685549"
Jul 24 17:12:03.187456 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:03.186981    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-t2znt" podStartSLOduration=67.512630524 podStartE2EDuration="1m30.186958704s" totalImagesPullingTime="22.67432818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:10:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:12:03.18554194 +0000 UTC m=+90.387591215" watchObservedRunningTime="2026-07-24 17:12:03.186958704 +0000 UTC m=+90.389007979"
Jul 24 17:12:03.215882 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:03.215826    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8pb55" podStartSLOduration=10.794778977 podStartE2EDuration="25.215804338s" totalImagesPullingTime="14.421025361s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:11:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:12:03.214887738 +0000 UTC m=+90.416937013" watchObservedRunningTime="2026-07-24 17:12:03.215804338 +0000 UTC m=+90.417853613"
Jul 24 17:12:03.271186 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:03.271149    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:12:03.351077 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:03.351029    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:12:03.488831 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:03.488792    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5dc5d7846d-8hxgs"
Jul 24 17:12:03.489018 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:03.488864    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5dc5d7846d-8hxgs"
Jul 24 17:12:04.060501 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:04.060462    2581 generic.go:356] "Generic (PLEG): container finished" podID="2d8f8104-653f-4c52-850b-56e496c30884" containerID="c16969c06171d5e3126c475a855fb259c859a2f8c35c6c4552ffe65284b20b95" exitCode=0
Jul 24 17:12:04.061250 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:04.060533    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d8f8104-653f-4c52-850b-56e496c30884","Type":"ContainerDied","Data":"c16969c06171d5e3126c475a855fb259c859a2f8c35c6c4552ffe65284b20b95"}
Jul 24 17:12:04.064017 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:04.063907    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dvz8j" event={"ID":"d8e047ab-f0c3-48cc-84c2-19559698ab11","Type":"ContainerStarted","Data":"17a4726daa0a37401997426f128e0421bc2631e6af973ebf9e2337b7f4e6e17d"}
Jul 24 17:12:04.064017 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:04.063980    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dvz8j" event={"ID":"d8e047ab-f0c3-48cc-84c2-19559698ab11","Type":"ContainerStarted","Data":"c45c1be8e74a0a0ee447c013c854a4532819580c7238699badb72aa27f34488c"}
Jul 24 17:12:04.070676 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:04.070650    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7569b5b876-z2trz" event={"ID":"31e30a4b-cb9c-42ea-a5c0-484a75fe6a12","Type":"ContainerStarted","Data":"8c778cbeee6ed5cc45584056b8d431abe968f1a2f45c495a469ceed86e2b6ba8"}
Jul 24 17:12:04.074183 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:04.074156    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-dd94759f-6lss7" event={"ID":"cce52072-dc7f-43b7-ac2d-318e50963272","Type":"ContainerStarted","Data":"61bd0a3be1b0c3ca537ff023dbe2184970822e569f23e1da470eeac334b2d1cc"}
Jul 24 17:12:04.074315 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:04.074228    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-dd94759f-6lss7" event={"ID":"cce52072-dc7f-43b7-ac2d-318e50963272","Type":"ContainerStarted","Data":"793d7fc54b27d8d5731f9e3c9a16de06d45520acdbeac2054d967baa1416000c"}
Jul 24 17:12:04.079930 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:04.079396    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-f9qj2" event={"ID":"fac1bdfa-4c1a-480d-9b1a-e3b025e473bb","Type":"ContainerStarted","Data":"9be5a3b9c4f45f5d447ab5da2da0bc6990d43f02a06e68c6a75b4d83a2fabb4a"}
Jul 24 17:12:04.084011 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:04.083730    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-dvz8j" podStartSLOduration=10.956761643 podStartE2EDuration="26.083711119s" totalImagesPullingTime="14.963126444s" totalInitContainerRuntime="163.823032ms" isStatefulPod=true podCreationTimestamp="2026-07-24 17:11:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:12:04.082662297 +0000 UTC m=+91.284711574" watchObservedRunningTime="2026-07-24 17:12:04.083711119 +0000 UTC m=+91.285760392"
Jul 24 17:12:04.104969 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:04.104077    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-f9qj2" podStartSLOduration=11.319715045 podStartE2EDuration="26.104033116s" totalImagesPullingTime="14.784318071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:11:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:12:04.102479555 +0000 UTC m=+91.304528842" watchObservedRunningTime="2026-07-24 17:12:04.104033116 +0000 UTC m=+91.306082391"
Jul 24 17:12:04.129198 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:04.129137    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7569b5b876-z2trz" podStartSLOduration=6.103496804 podStartE2EDuration="21.129122326s" totalImagesPullingTime="15.025625522s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:11:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:12:04.126482297 +0000 UTC m=+91.328531574" watchObservedRunningTime="2026-07-24 17:12:04.129122326 +0000 UTC m=+91.331171600"
Jul 24 17:12:06.093458 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:06.093419    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"284bf1c9-f105-4053-b478-b8bd4fd6625f","Type":"ContainerStarted","Data":"d19670244761d96695127747d504a879cb7a42e6bd5bd74ab611f2a50fea84a4"}
Jul 24 17:12:06.096204 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:06.096168    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-dd94759f-6lss7" event={"ID":"cce52072-dc7f-43b7-ac2d-318e50963272","Type":"ContainerStarted","Data":"a18117eb599c7ce506cf3174acc0bfe8e809bb1aaeedfecff7e294121c993d6a"}
Jul 24 17:12:06.770861 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:06.770785    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c548b8c6d-drlkk"]
Jul 24 17:12:06.792067 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:06.792021    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c548b8c6d-drlkk"]
Jul 24 17:12:06.792216 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:06.792155    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c548b8c6d-drlkk"
Jul 24 17:12:06.800980 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:06.800955    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 24 17:12:06.837380 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:06.837344    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5498695e-9a6a-4e67-9fca-ae3c027a48e6-console-config\") pod \"console-6c548b8c6d-drlkk\" (UID: \"5498695e-9a6a-4e67-9fca-ae3c027a48e6\") " pod="openshift-console/console-6c548b8c6d-drlkk"
Jul 24 17:12:06.837539 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:06.837396    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5498695e-9a6a-4e67-9fca-ae3c027a48e6-console-serving-cert\") pod \"console-6c548b8c6d-drlkk\" (UID: \"5498695e-9a6a-4e67-9fca-ae3c027a48e6\") " pod="openshift-console/console-6c548b8c6d-drlkk"
Jul 24 17:12:06.837539 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:06.837414    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5498695e-9a6a-4e67-9fca-ae3c027a48e6-console-oauth-config\") pod \"console-6c548b8c6d-drlkk\" (UID: \"5498695e-9a6a-4e67-9fca-ae3c027a48e6\") " pod="openshift-console/console-6c548b8c6d-drlkk"
Jul 24 17:12:06.837539 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:06.837451    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5498695e-9a6a-4e67-9fca-ae3c027a48e6-trusted-ca-bundle\") pod \"console-6c548b8c6d-drlkk\" (UID: \"5498695e-9a6a-4e67-9fca-ae3c027a48e6\") " pod="openshift-console/console-6c548b8c6d-drlkk"
Jul 24 17:12:06.837539 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:06.837468    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5498695e-9a6a-4e67-9fca-ae3c027a48e6-service-ca\") pod \"console-6c548b8c6d-drlkk\" (UID: \"5498695e-9a6a-4e67-9fca-ae3c027a48e6\") " pod="openshift-console/console-6c548b8c6d-drlkk"
Jul 24 17:12:06.837539 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:06.837485    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5498695e-9a6a-4e67-9fca-ae3c027a48e6-oauth-serving-cert\") pod \"console-6c548b8c6d-drlkk\" (UID: \"5498695e-9a6a-4e67-9fca-ae3c027a48e6\") " pod="openshift-console/console-6c548b8c6d-drlkk"
Jul 24 17:12:06.837699 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:06.837569    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fqvt\" (UniqueName: \"kubernetes.io/projected/5498695e-9a6a-4e67-9fca-ae3c027a48e6-kube-api-access-2fqvt\") pod \"console-6c548b8c6d-drlkk\" (UID: \"5498695e-9a6a-4e67-9fca-ae3c027a48e6\") " pod="openshift-console/console-6c548b8c6d-drlkk"
Jul 24 17:12:06.842937 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:06.842912    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:12:06.933450 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:06.933426    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:12:06.938120 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:06.938091    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5498695e-9a6a-4e67-9fca-ae3c027a48e6-console-config\") pod \"console-6c548b8c6d-drlkk\" (UID: \"5498695e-9a6a-4e67-9fca-ae3c027a48e6\") " pod="openshift-console/console-6c548b8c6d-drlkk"
Jul 24 17:12:06.938238 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:06.938167    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5498695e-9a6a-4e67-9fca-ae3c027a48e6-console-serving-cert\") pod \"console-6c548b8c6d-drlkk\" (UID: \"5498695e-9a6a-4e67-9fca-ae3c027a48e6\") " pod="openshift-console/console-6c548b8c6d-drlkk"
Jul 24 17:12:06.938238 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:06.938202    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5498695e-9a6a-4e67-9fca-ae3c027a48e6-console-oauth-config\") pod \"console-6c548b8c6d-drlkk\" (UID: \"5498695e-9a6a-4e67-9fca-ae3c027a48e6\") " pod="openshift-console/console-6c548b8c6d-drlkk"
Jul 24 17:12:06.938350 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:06.938262    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5498695e-9a6a-4e67-9fca-ae3c027a48e6-trusted-ca-bundle\") pod \"console-6c548b8c6d-drlkk\" (UID: \"5498695e-9a6a-4e67-9fca-ae3c027a48e6\") " pod="openshift-console/console-6c548b8c6d-drlkk"
Jul 24 17:12:06.938350 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:06.938293    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5498695e-9a6a-4e67-9fca-ae3c027a48e6-service-ca\") pod \"console-6c548b8c6d-drlkk\" (UID: \"5498695e-9a6a-4e67-9fca-ae3c027a48e6\") " pod="openshift-console/console-6c548b8c6d-drlkk"
Jul 24 17:12:06.938350 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:06.938321    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5498695e-9a6a-4e67-9fca-ae3c027a48e6-oauth-serving-cert\") pod \"console-6c548b8c6d-drlkk\" (UID: \"5498695e-9a6a-4e67-9fca-ae3c027a48e6\") " pod="openshift-console/console-6c548b8c6d-drlkk"
Jul 24 17:12:06.938502 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:06.938363    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fqvt\" (UniqueName: \"kubernetes.io/projected/5498695e-9a6a-4e67-9fca-ae3c027a48e6-kube-api-access-2fqvt\") pod \"console-6c548b8c6d-drlkk\" (UID: \"5498695e-9a6a-4e67-9fca-ae3c027a48e6\") " pod="openshift-console/console-6c548b8c6d-drlkk"
Jul 24 17:12:06.939054 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:06.939007    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5498695e-9a6a-4e67-9fca-ae3c027a48e6-console-config\") pod \"console-6c548b8c6d-drlkk\" (UID: \"5498695e-9a6a-4e67-9fca-ae3c027a48e6\") " pod="openshift-console/console-6c548b8c6d-drlkk"
Jul 24 17:12:06.939148 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:06.939028    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5498695e-9a6a-4e67-9fca-ae3c027a48e6-service-ca\") pod \"console-6c548b8c6d-drlkk\" (UID: \"5498695e-9a6a-4e67-9fca-ae3c027a48e6\") " pod="openshift-console/console-6c548b8c6d-drlkk"
Jul 24 17:12:06.939253 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:06.939233    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5498695e-9a6a-4e67-9fca-ae3c027a48e6-trusted-ca-bundle\") pod \"console-6c548b8c6d-drlkk\" (UID: \"5498695e-9a6a-4e67-9fca-ae3c027a48e6\") " pod="openshift-console/console-6c548b8c6d-drlkk"
Jul 24 17:12:06.939409 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:06.939390    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5498695e-9a6a-4e67-9fca-ae3c027a48e6-oauth-serving-cert\") pod \"console-6c548b8c6d-drlkk\" (UID: \"5498695e-9a6a-4e67-9fca-ae3c027a48e6\") " pod="openshift-console/console-6c548b8c6d-drlkk"
Jul 24 17:12:06.940726 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:06.940708    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5498695e-9a6a-4e67-9fca-ae3c027a48e6-console-serving-cert\") pod \"console-6c548b8c6d-drlkk\" (UID: \"5498695e-9a6a-4e67-9fca-ae3c027a48e6\") " pod="openshift-console/console-6c548b8c6d-drlkk"
Jul 24 17:12:06.941226 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:06.941209    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5498695e-9a6a-4e67-9fca-ae3c027a48e6-console-oauth-config\") pod \"console-6c548b8c6d-drlkk\" (UID: \"5498695e-9a6a-4e67-9fca-ae3c027a48e6\") " pod="openshift-console/console-6c548b8c6d-drlkk"
Jul 24 17:12:06.952668 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:06.952645    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fqvt\" (UniqueName: \"kubernetes.io/projected/5498695e-9a6a-4e67-9fca-ae3c027a48e6-kube-api-access-2fqvt\") pod \"console-6c548b8c6d-drlkk\" (UID: \"5498695e-9a6a-4e67-9fca-ae3c027a48e6\") " pod="openshift-console/console-6c548b8c6d-drlkk"
Jul 24 17:12:07.084892 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:07.084861    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:12:07.101434 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:07.101279    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c548b8c6d-drlkk"
Jul 24 17:12:07.103535 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:07.103385    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d8f8104-653f-4c52-850b-56e496c30884","Type":"ContainerStarted","Data":"61aeae10049aa845078ce01582f6264a477edd96823ec4433c915634e71d1fea"}
Jul 24 17:12:07.107615 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:07.107379    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"284bf1c9-f105-4053-b478-b8bd4fd6625f","Type":"ContainerStarted","Data":"7a310e1073cfa69399179bd136fa276967c52923b80d1a874e022d4b7add19e2"}
Jul 24 17:12:07.107615 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:07.107464    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"284bf1c9-f105-4053-b478-b8bd4fd6625f","Type":"ContainerStarted","Data":"f14f459b2bd6ceea1ae130fb2c0aae5699a4efb166d965fb37d52e43ae4e73ac"}
Jul 24 17:12:07.107615 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:07.107508    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"284bf1c9-f105-4053-b478-b8bd4fd6625f","Type":"ContainerStarted","Data":"009fb2a0a4bde6e5898539b03f01eb37fd82898856a1e9a835cba5de618f682f"}
Jul 24 17:12:07.107615 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:07.107562    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"284bf1c9-f105-4053-b478-b8bd4fd6625f","Type":"ContainerStarted","Data":"56241c7ac0216b720152963434b213c86dbec534a1b9556f6f008f5030b89024"}
Jul 24 17:12:07.111073 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:07.110910    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-dd94759f-6lss7" event={"ID":"cce52072-dc7f-43b7-ac2d-318e50963272","Type":"ContainerStarted","Data":"202e8289d82024dc2fa762af70b13ca18d45faae173f382136987356f08840a4"}
Jul 24 17:12:07.111073 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:07.110993    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-dd94759f-6lss7" event={"ID":"cce52072-dc7f-43b7-ac2d-318e50963272","Type":"ContainerStarted","Data":"cc93336d3ce26b5408c0d51942051424fff3f6ae772cc4cc1f039cc117fd707f"}
Jul 24 17:12:07.111574 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:07.111558    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-dd94759f-6lss7"
Jul 24 17:12:07.140303 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:07.139999    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-dd94759f-6lss7" podStartSLOduration=9.168572092 podStartE2EDuration="26.139975869s" totalImagesPullingTime="16.971403777s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:11:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:12:07.138946407 +0000 UTC m=+94.340995695" watchObservedRunningTime="2026-07-24 17:12:07.139975869 +0000 UTC m=+94.342025139"
Jul 24 17:12:07.203920 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:07.203882    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:12:07.299189 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:07.298981    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c548b8c6d-drlkk"]
Jul 24 17:12:07.301379 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:12:07.301327    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5498695e_9a6a_4e67_9fca_ae3c027a48e6.slice/crio-a5715000e2522f22da357fedceeff383f79026c3aeb1238c262ba54343ac1a8e WatchSource:0}: Error finding container a5715000e2522f22da357fedceeff383f79026c3aeb1238c262ba54343ac1a8e: Status 404 returned error can't find the container with id a5715000e2522f22da357fedceeff383f79026c3aeb1238c262ba54343ac1a8e
Jul 24 17:12:07.301554 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:07.301448    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:12:08.119072 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:08.119013    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"284bf1c9-f105-4053-b478-b8bd4fd6625f","Type":"ContainerStarted","Data":"daf2426b7dc7a99a0acc80af8e435df7a8579eb56f1222da0a009419edabe5a3"}
Jul 24 17:12:08.120646 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:08.120624    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c548b8c6d-drlkk" event={"ID":"5498695e-9a6a-4e67-9fca-ae3c027a48e6","Type":"ContainerStarted","Data":"64f5452dbb5416dd80fadfecfb7b8f8b09a2fff068840f023015b894d079201c"}
Jul 24 17:12:08.120775 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:08.120661    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c548b8c6d-drlkk" event={"ID":"5498695e-9a6a-4e67-9fca-ae3c027a48e6","Type":"ContainerStarted","Data":"a5715000e2522f22da357fedceeff383f79026c3aeb1238c262ba54343ac1a8e"}
Jul 24 17:12:08.123741 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:08.123714    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d8f8104-653f-4c52-850b-56e496c30884","Type":"ContainerStarted","Data":"65f8e05ab790aeded00fd275a7c7bf4d610ca5284044e60aea5b384d93e0556c"}
Jul 24 17:12:08.123844 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:08.123786    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d8f8104-653f-4c52-850b-56e496c30884","Type":"ContainerStarted","Data":"e91d6684aac1046ff4df10d33b6ecf519ae53ced15e25171d6a6e582a1c4ea0c"}
Jul 24 17:12:08.123844 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:08.123827    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d8f8104-653f-4c52-850b-56e496c30884","Type":"ContainerStarted","Data":"828371c79c36ed7658a3577fc6bd6996a79123f8c39d75b78f2c76fc8b6a799f"}
Jul 24 17:12:08.123915 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:08.123868    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d8f8104-653f-4c52-850b-56e496c30884","Type":"ContainerStarted","Data":"058dd73dd5d71eba7411e7837e86e8f89ffdb4a84e36690223974e26aa1cdf7b"}
Jul 24 17:12:08.123915 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:08.123908    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d8f8104-653f-4c52-850b-56e496c30884","Type":"ContainerStarted","Data":"06f613e9d037295d6cd807636b0bab5f9942d9ced65105df8c49c1c6dfad057d"}
Jul 24 17:12:08.129633 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:08.129615    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-dd94759f-6lss7"
Jul 24 17:12:08.149623 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:08.149577    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=12.320023195 podStartE2EDuration="29.149562488s" totalImagesPullingTime="16.803049178s" totalInitContainerRuntime="26.490115ms" isStatefulPod=false podCreationTimestamp="2026-07-24 17:11:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:12:08.147608364 +0000 UTC m=+95.349657678" watchObservedRunningTime="2026-07-24 17:12:08.149562488 +0000 UTC m=+95.351611762"
Jul 24 17:12:08.202357 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:08.202296    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=5.409293536 podStartE2EDuration="24.202275116s" totalImagesPullingTime="18.726580257s" totalInitContainerRuntime="66.401323ms" isStatefulPod=false podCreationTimestamp="2026-07-24 17:11:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:12:08.198871379 +0000 UTC m=+95.400920680" watchObservedRunningTime="2026-07-24 17:12:08.202275116 +0000 UTC m=+95.404324391"
Jul 24 17:12:09.596077 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:09.596011    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56ffd5f7b5-vvmhd"
Jul 24 17:12:10.264645 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:10.264608    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:12:14.921417 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:14.921382    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:12:14.974522 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:14.974487    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:12:17.102016 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:17.101978    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c548b8c6d-drlkk"
Jul 24 17:12:17.102517 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:17.102082    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c548b8c6d-drlkk"
Jul 24 17:12:17.107213 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:17.107192    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c548b8c6d-drlkk"
Jul 24 17:12:17.127900 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:17.127850    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c548b8c6d-drlkk" podStartSLOduration=11.127833648 podStartE2EDuration="11.127833648s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:12:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:12:08.216951431 +0000 UTC m=+95.419000705" watchObservedRunningTime="2026-07-24 17:12:17.127833648 +0000 UTC m=+104.329882922"
Jul 24 17:12:17.169685 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:17.169657    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c548b8c6d-drlkk"
Jul 24 17:12:17.311257 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:17.311225    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:12:17.362244 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:17.362164    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:12:17.413112 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:17.413075    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:12:21.483750 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:21.483716    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:12:23.493751 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:23.493720    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5dc5d7846d-8hxgs"
Jul 24 17:12:23.497704 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:23.497682    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5dc5d7846d-8hxgs"
Jul 24 17:12:28.019991 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:28.019922    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-56ffd5f7b5-vvmhd" podUID="b111fa18-0380-470b-9d3c-ec7992007d20" containerName="console" containerID="cri-o://92497ce52b93fd0211a8d817134422b27858cba5a75ffe4e7bec8994988e3a4b" gracePeriod=15
Jul 24 17:12:28.217936 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:28.217899    2581 generic.go:356] "Generic (PLEG): container finished" podID="b111fa18-0380-470b-9d3c-ec7992007d20" containerID="92497ce52b93fd0211a8d817134422b27858cba5a75ffe4e7bec8994988e3a4b" exitCode=0
Jul 24 17:12:28.218111 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:28.218032    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56ffd5f7b5-vvmhd" event={"ID":"b111fa18-0380-470b-9d3c-ec7992007d20","Type":"ContainerDied","Data":"92497ce52b93fd0211a8d817134422b27858cba5a75ffe4e7bec8994988e3a4b"}
Jul 24 17:12:28.323107 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:28.322490    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56ffd5f7b5-vvmhd"
Jul 24 17:12:28.436635 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:28.436593    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b111fa18-0380-470b-9d3c-ec7992007d20-console-config\") pod \"b111fa18-0380-470b-9d3c-ec7992007d20\" (UID: \"b111fa18-0380-470b-9d3c-ec7992007d20\") "
Jul 24 17:12:28.436826 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:28.436668    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-52hxg\" (UniqueName: \"kubernetes.io/projected/b111fa18-0380-470b-9d3c-ec7992007d20-kube-api-access-52hxg\") pod \"b111fa18-0380-470b-9d3c-ec7992007d20\" (UID: \"b111fa18-0380-470b-9d3c-ec7992007d20\") "
Jul 24 17:12:28.436826 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:28.436705    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b111fa18-0380-470b-9d3c-ec7992007d20-console-serving-cert\") pod \"b111fa18-0380-470b-9d3c-ec7992007d20\" (UID: \"b111fa18-0380-470b-9d3c-ec7992007d20\") "
Jul 24 17:12:28.436826 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:28.436750    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b111fa18-0380-470b-9d3c-ec7992007d20-console-oauth-config\") pod \"b111fa18-0380-470b-9d3c-ec7992007d20\" (UID: \"b111fa18-0380-470b-9d3c-ec7992007d20\") "
Jul 24 17:12:28.436826 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:28.436807    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b111fa18-0380-470b-9d3c-ec7992007d20-service-ca\") pod \"b111fa18-0380-470b-9d3c-ec7992007d20\" (UID: \"b111fa18-0380-470b-9d3c-ec7992007d20\") "
Jul 24 17:12:28.437132 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:28.436859    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b111fa18-0380-470b-9d3c-ec7992007d20-oauth-serving-cert\") pod \"b111fa18-0380-470b-9d3c-ec7992007d20\" (UID: \"b111fa18-0380-470b-9d3c-ec7992007d20\") "
Jul 24 17:12:28.438215 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:28.438179    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b111fa18-0380-470b-9d3c-ec7992007d20-console-config" (OuterVolumeSpecName: "console-config") pod "b111fa18-0380-470b-9d3c-ec7992007d20" (UID: "b111fa18-0380-470b-9d3c-ec7992007d20"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:12:28.438785 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:28.438757    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b111fa18-0380-470b-9d3c-ec7992007d20-service-ca" (OuterVolumeSpecName: "service-ca") pod "b111fa18-0380-470b-9d3c-ec7992007d20" (UID: "b111fa18-0380-470b-9d3c-ec7992007d20"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:12:28.439055 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:28.439013    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b111fa18-0380-470b-9d3c-ec7992007d20-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b111fa18-0380-470b-9d3c-ec7992007d20" (UID: "b111fa18-0380-470b-9d3c-ec7992007d20"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:12:28.448155 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:28.447178    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b111fa18-0380-470b-9d3c-ec7992007d20-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b111fa18-0380-470b-9d3c-ec7992007d20" (UID: "b111fa18-0380-470b-9d3c-ec7992007d20"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:12:28.448853 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:28.448820    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b111fa18-0380-470b-9d3c-ec7992007d20-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b111fa18-0380-470b-9d3c-ec7992007d20" (UID: "b111fa18-0380-470b-9d3c-ec7992007d20"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:12:28.448968 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:28.448941    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b111fa18-0380-470b-9d3c-ec7992007d20-kube-api-access-52hxg" (OuterVolumeSpecName: "kube-api-access-52hxg") pod "b111fa18-0380-470b-9d3c-ec7992007d20" (UID: "b111fa18-0380-470b-9d3c-ec7992007d20"). InnerVolumeSpecName "kube-api-access-52hxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:12:28.538222 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:28.538151    2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b111fa18-0380-470b-9d3c-ec7992007d20-oauth-serving-cert\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:12:28.538222 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:28.538178    2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b111fa18-0380-470b-9d3c-ec7992007d20-console-config\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:12:28.538222 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:28.538188    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52hxg\" (UniqueName: \"kubernetes.io/projected/b111fa18-0380-470b-9d3c-ec7992007d20-kube-api-access-52hxg\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:12:28.538222 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:28.538198    2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b111fa18-0380-470b-9d3c-ec7992007d20-console-serving-cert\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:12:28.538222 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:28.538206    2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b111fa18-0380-470b-9d3c-ec7992007d20-console-oauth-config\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:12:28.538222 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:28.538215    2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b111fa18-0380-470b-9d3c-ec7992007d20-service-ca\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:12:29.224111 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:29.224088    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56ffd5f7b5-vvmhd"
Jul 24 17:12:29.224665 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:29.224078    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56ffd5f7b5-vvmhd" event={"ID":"b111fa18-0380-470b-9d3c-ec7992007d20","Type":"ContainerDied","Data":"4c3a4cfa11db711e60b72ce7fafd9a9a4c7f4b6ebd29b7dd48d4ef80875fc62e"}
Jul 24 17:12:29.224665 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:29.224254    2581 scope.go:117] "RemoveContainer" containerID="92497ce52b93fd0211a8d817134422b27858cba5a75ffe4e7bec8994988e3a4b"
Jul 24 17:12:29.246298 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:29.246273    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56ffd5f7b5-vvmhd"]
Jul 24 17:12:29.254484 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:29.251020    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-56ffd5f7b5-vvmhd"]
Jul 24 17:12:29.383985 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:29.383953    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b111fa18-0380-470b-9d3c-ec7992007d20" path="/var/lib/kubelet/pods/b111fa18-0380-470b-9d3c-ec7992007d20/volumes"
Jul 24 17:12:35.090027 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:35.089992    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-t2znt"
Jul 24 17:12:45.265563 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:45.265520    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:12:45.341948 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:45.341921    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:12:45.357873 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:45.357851    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:12:54.800803 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:54.800761    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:12:54.850969 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:54.850938    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:12:59.033351 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:59.033318    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 17:12:59.033758 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:59.033615    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="284bf1c9-f105-4053-b478-b8bd4fd6625f" containerName="alertmanager" containerID="cri-o://d19670244761d96695127747d504a879cb7a42e6bd5bd74ab611f2a50fea84a4" gracePeriod=120
Jul 24 17:12:59.033758 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:59.033670    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="284bf1c9-f105-4053-b478-b8bd4fd6625f" containerName="kube-rbac-proxy-metric" containerID="cri-o://7a310e1073cfa69399179bd136fa276967c52923b80d1a874e022d4b7add19e2" gracePeriod=120
Jul 24 17:12:59.033883 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:59.033750    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="284bf1c9-f105-4053-b478-b8bd4fd6625f" containerName="prom-label-proxy" containerID="cri-o://daf2426b7dc7a99a0acc80af8e435df7a8579eb56f1222da0a009419edabe5a3" gracePeriod=120
Jul 24 17:12:59.033883 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:59.033756    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="284bf1c9-f105-4053-b478-b8bd4fd6625f" containerName="kube-rbac-proxy-web" containerID="cri-o://009fb2a0a4bde6e5898539b03f01eb37fd82898856a1e9a835cba5de618f682f" gracePeriod=120
Jul 24 17:12:59.033883 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:59.033791    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="284bf1c9-f105-4053-b478-b8bd4fd6625f" containerName="config-reloader" containerID="cri-o://56241c7ac0216b720152963434b213c86dbec534a1b9556f6f008f5030b89024" gracePeriod=120
Jul 24 17:12:59.033883 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:59.033756    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="284bf1c9-f105-4053-b478-b8bd4fd6625f" containerName="kube-rbac-proxy" containerID="cri-o://f14f459b2bd6ceea1ae130fb2c0aae5699a4efb166d965fb37d52e43ae4e73ac" gracePeriod=120
Jul 24 17:12:59.351970 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:59.351888    2581 generic.go:356] "Generic (PLEG): container finished" podID="284bf1c9-f105-4053-b478-b8bd4fd6625f" containerID="daf2426b7dc7a99a0acc80af8e435df7a8579eb56f1222da0a009419edabe5a3" exitCode=0
Jul 24 17:12:59.351970 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:59.351913    2581 generic.go:356] "Generic (PLEG): container finished" podID="284bf1c9-f105-4053-b478-b8bd4fd6625f" containerID="f14f459b2bd6ceea1ae130fb2c0aae5699a4efb166d965fb37d52e43ae4e73ac" exitCode=0
Jul 24 17:12:59.351970 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:59.351921    2581 generic.go:356] "Generic (PLEG): container finished" podID="284bf1c9-f105-4053-b478-b8bd4fd6625f" containerID="56241c7ac0216b720152963434b213c86dbec534a1b9556f6f008f5030b89024" exitCode=0
Jul 24 17:12:59.351970 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:59.351927    2581 generic.go:356] "Generic (PLEG): container finished" podID="284bf1c9-f105-4053-b478-b8bd4fd6625f" containerID="d19670244761d96695127747d504a879cb7a42e6bd5bd74ab611f2a50fea84a4" exitCode=0
Jul 24 17:12:59.351970 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:59.351957    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"284bf1c9-f105-4053-b478-b8bd4fd6625f","Type":"ContainerDied","Data":"daf2426b7dc7a99a0acc80af8e435df7a8579eb56f1222da0a009419edabe5a3"}
Jul 24 17:12:59.352255 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:59.352015    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"284bf1c9-f105-4053-b478-b8bd4fd6625f","Type":"ContainerDied","Data":"f14f459b2bd6ceea1ae130fb2c0aae5699a4efb166d965fb37d52e43ae4e73ac"}
Jul 24 17:12:59.352255 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:59.352070    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"284bf1c9-f105-4053-b478-b8bd4fd6625f","Type":"ContainerDied","Data":"56241c7ac0216b720152963434b213c86dbec534a1b9556f6f008f5030b89024"}
Jul 24 17:12:59.352255 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:12:59.352118    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"284bf1c9-f105-4053-b478-b8bd4fd6625f","Type":"ContainerDied","Data":"d19670244761d96695127747d504a879cb7a42e6bd5bd74ab611f2a50fea84a4"}
Jul 24 17:13:00.274712 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.274649    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.315208 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.315177    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/284bf1c9-f105-4053-b478-b8bd4fd6625f-config-out\") pod \"284bf1c9-f105-4053-b478-b8bd4fd6625f\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") "
Jul 24 17:13:00.315388 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.315287    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-config-volume\") pod \"284bf1c9-f105-4053-b478-b8bd4fd6625f\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") "
Jul 24 17:13:00.315388 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.315340    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-secret-alertmanager-kube-rbac-proxy\") pod \"284bf1c9-f105-4053-b478-b8bd4fd6625f\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") "
Jul 24 17:13:00.315517 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.315420    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/284bf1c9-f105-4053-b478-b8bd4fd6625f-metrics-client-ca\") pod \"284bf1c9-f105-4053-b478-b8bd4fd6625f\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") "
Jul 24 17:13:00.315517 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.315467    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-secret-alertmanager-main-tls\") pod \"284bf1c9-f105-4053-b478-b8bd4fd6625f\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") "
Jul 24 17:13:00.315517 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.315494    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"284bf1c9-f105-4053-b478-b8bd4fd6625f\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") "
Jul 24 17:13:00.315669 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.315555    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/284bf1c9-f105-4053-b478-b8bd4fd6625f-alertmanager-trusted-ca-bundle\") pod \"284bf1c9-f105-4053-b478-b8bd4fd6625f\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") "
Jul 24 17:13:00.315669 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.315599    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mdv9n\" (UniqueName: \"kubernetes.io/projected/284bf1c9-f105-4053-b478-b8bd4fd6625f-kube-api-access-mdv9n\") pod \"284bf1c9-f105-4053-b478-b8bd4fd6625f\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") "
Jul 24 17:13:00.315669 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.315652    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-web-config\") pod \"284bf1c9-f105-4053-b478-b8bd4fd6625f\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") "
Jul 24 17:13:00.315857 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.315678    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-secret-alertmanager-kube-rbac-proxy-web\") pod \"284bf1c9-f105-4053-b478-b8bd4fd6625f\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") "
Jul 24 17:13:00.315857 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.315728    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/284bf1c9-f105-4053-b478-b8bd4fd6625f-tls-assets\") pod \"284bf1c9-f105-4053-b478-b8bd4fd6625f\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") "
Jul 24 17:13:00.315857 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.315778    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-cluster-tls-config\") pod \"284bf1c9-f105-4053-b478-b8bd4fd6625f\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") "
Jul 24 17:13:00.315857 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.315806    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/284bf1c9-f105-4053-b478-b8bd4fd6625f-alertmanager-main-db\") pod \"284bf1c9-f105-4053-b478-b8bd4fd6625f\" (UID: \"284bf1c9-f105-4053-b478-b8bd4fd6625f\") "
Jul 24 17:13:00.316078 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.315890    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/284bf1c9-f105-4053-b478-b8bd4fd6625f-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "284bf1c9-f105-4053-b478-b8bd4fd6625f" (UID: "284bf1c9-f105-4053-b478-b8bd4fd6625f"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:13:00.316208 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.316186    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/284bf1c9-f105-4053-b478-b8bd4fd6625f-metrics-client-ca\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:00.316452 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.316429    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/284bf1c9-f105-4053-b478-b8bd4fd6625f-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "284bf1c9-f105-4053-b478-b8bd4fd6625f" (UID: "284bf1c9-f105-4053-b478-b8bd4fd6625f"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:13:00.318352 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.318287    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-config-volume" (OuterVolumeSpecName: "config-volume") pod "284bf1c9-f105-4053-b478-b8bd4fd6625f" (UID: "284bf1c9-f105-4053-b478-b8bd4fd6625f"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:13:00.318468 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.318368    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/284bf1c9-f105-4053-b478-b8bd4fd6625f-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "284bf1c9-f105-4053-b478-b8bd4fd6625f" (UID: "284bf1c9-f105-4053-b478-b8bd4fd6625f"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:13:00.323937 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.323902    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/284bf1c9-f105-4053-b478-b8bd4fd6625f-config-out" (OuterVolumeSpecName: "config-out") pod "284bf1c9-f105-4053-b478-b8bd4fd6625f" (UID: "284bf1c9-f105-4053-b478-b8bd4fd6625f"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:13:00.324061 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.323964    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "284bf1c9-f105-4053-b478-b8bd4fd6625f" (UID: "284bf1c9-f105-4053-b478-b8bd4fd6625f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:13:00.324492 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.324455    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "284bf1c9-f105-4053-b478-b8bd4fd6625f" (UID: "284bf1c9-f105-4053-b478-b8bd4fd6625f"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:13:00.324492 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.324459    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/284bf1c9-f105-4053-b478-b8bd4fd6625f-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "284bf1c9-f105-4053-b478-b8bd4fd6625f" (UID: "284bf1c9-f105-4053-b478-b8bd4fd6625f"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:13:00.325246 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.325096    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/284bf1c9-f105-4053-b478-b8bd4fd6625f-kube-api-access-mdv9n" (OuterVolumeSpecName: "kube-api-access-mdv9n") pod "284bf1c9-f105-4053-b478-b8bd4fd6625f" (UID: "284bf1c9-f105-4053-b478-b8bd4fd6625f"). InnerVolumeSpecName "kube-api-access-mdv9n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:13:00.325246 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.325123    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "284bf1c9-f105-4053-b478-b8bd4fd6625f" (UID: "284bf1c9-f105-4053-b478-b8bd4fd6625f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:13:00.325350 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.325246    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "284bf1c9-f105-4053-b478-b8bd4fd6625f" (UID: "284bf1c9-f105-4053-b478-b8bd4fd6625f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:13:00.326988 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.326961    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "284bf1c9-f105-4053-b478-b8bd4fd6625f" (UID: "284bf1c9-f105-4053-b478-b8bd4fd6625f"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:13:00.337192 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.337160    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-web-config" (OuterVolumeSpecName: "web-config") pod "284bf1c9-f105-4053-b478-b8bd4fd6625f" (UID: "284bf1c9-f105-4053-b478-b8bd4fd6625f"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:13:00.361995 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.361919    2581 generic.go:356] "Generic (PLEG): container finished" podID="284bf1c9-f105-4053-b478-b8bd4fd6625f" containerID="7a310e1073cfa69399179bd136fa276967c52923b80d1a874e022d4b7add19e2" exitCode=0
Jul 24 17:13:00.361995 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.361945    2581 generic.go:356] "Generic (PLEG): container finished" podID="284bf1c9-f105-4053-b478-b8bd4fd6625f" containerID="009fb2a0a4bde6e5898539b03f01eb37fd82898856a1e9a835cba5de618f682f" exitCode=0
Jul 24 17:13:00.362186 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.362007    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.362186 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.361998    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"284bf1c9-f105-4053-b478-b8bd4fd6625f","Type":"ContainerDied","Data":"7a310e1073cfa69399179bd136fa276967c52923b80d1a874e022d4b7add19e2"}
Jul 24 17:13:00.362186 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.362146    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"284bf1c9-f105-4053-b478-b8bd4fd6625f","Type":"ContainerDied","Data":"009fb2a0a4bde6e5898539b03f01eb37fd82898856a1e9a835cba5de618f682f"}
Jul 24 17:13:00.362186 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.362173    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"284bf1c9-f105-4053-b478-b8bd4fd6625f","Type":"ContainerDied","Data":"4e3f8e2ba15d389fd5d43fe620e044c6c43ed2313fa9e7de9e2d0dc77b2fd7c4"}
Jul 24 17:13:00.362307 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.362204    2581 scope.go:117] "RemoveContainer" containerID="daf2426b7dc7a99a0acc80af8e435df7a8579eb56f1222da0a009419edabe5a3"
Jul 24 17:13:00.369790 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.369772    2581 scope.go:117] "RemoveContainer" containerID="7a310e1073cfa69399179bd136fa276967c52923b80d1a874e022d4b7add19e2"
Jul 24 17:13:00.377065 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.377029    2581 scope.go:117] "RemoveContainer" containerID="f14f459b2bd6ceea1ae130fb2c0aae5699a4efb166d965fb37d52e43ae4e73ac"
Jul 24 17:13:00.383784 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.383767    2581 scope.go:117] "RemoveContainer" containerID="009fb2a0a4bde6e5898539b03f01eb37fd82898856a1e9a835cba5de618f682f"
Jul 24 17:13:00.390836 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.390813    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 17:13:00.392073 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.392032    2581 scope.go:117] "RemoveContainer" containerID="56241c7ac0216b720152963434b213c86dbec534a1b9556f6f008f5030b89024"
Jul 24 17:13:00.399060 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.399027    2581 scope.go:117] "RemoveContainer" containerID="d19670244761d96695127747d504a879cb7a42e6bd5bd74ab611f2a50fea84a4"
Jul 24 17:13:00.403530 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.403506    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 17:13:00.408752 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.408732    2581 scope.go:117] "RemoveContainer" containerID="091cd8ed99eb1a5bd646af9e4f8d62f867c20f46029a886957d5611c0a814b2e"
Jul 24 17:13:00.415754 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.415740    2581 scope.go:117] "RemoveContainer" containerID="daf2426b7dc7a99a0acc80af8e435df7a8579eb56f1222da0a009419edabe5a3"
Jul 24 17:13:00.416013 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:13:00.415990    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"daf2426b7dc7a99a0acc80af8e435df7a8579eb56f1222da0a009419edabe5a3\": container with ID starting with daf2426b7dc7a99a0acc80af8e435df7a8579eb56f1222da0a009419edabe5a3 not found: ID does not exist" containerID="daf2426b7dc7a99a0acc80af8e435df7a8579eb56f1222da0a009419edabe5a3"
Jul 24 17:13:00.416117 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.416019    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"daf2426b7dc7a99a0acc80af8e435df7a8579eb56f1222da0a009419edabe5a3"} err="failed to get container status \"daf2426b7dc7a99a0acc80af8e435df7a8579eb56f1222da0a009419edabe5a3\": rpc error: code = NotFound desc = could not find container \"daf2426b7dc7a99a0acc80af8e435df7a8579eb56f1222da0a009419edabe5a3\": container with ID starting with daf2426b7dc7a99a0acc80af8e435df7a8579eb56f1222da0a009419edabe5a3 not found: ID does not exist"
Jul 24 17:13:00.416117 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.416074    2581 scope.go:117] "RemoveContainer" containerID="7a310e1073cfa69399179bd136fa276967c52923b80d1a874e022d4b7add19e2"
Jul 24 17:13:00.416298 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:13:00.416281    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a310e1073cfa69399179bd136fa276967c52923b80d1a874e022d4b7add19e2\": container with ID starting with 7a310e1073cfa69399179bd136fa276967c52923b80d1a874e022d4b7add19e2 not found: ID does not exist" containerID="7a310e1073cfa69399179bd136fa276967c52923b80d1a874e022d4b7add19e2"
Jul 24 17:13:00.416338 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.416304    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a310e1073cfa69399179bd136fa276967c52923b80d1a874e022d4b7add19e2"} err="failed to get container status \"7a310e1073cfa69399179bd136fa276967c52923b80d1a874e022d4b7add19e2\": rpc error: code = NotFound desc = could not find container \"7a310e1073cfa69399179bd136fa276967c52923b80d1a874e022d4b7add19e2\": container with ID starting with 7a310e1073cfa69399179bd136fa276967c52923b80d1a874e022d4b7add19e2 not found: ID does not exist"
Jul 24 17:13:00.416338 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.416323    2581 scope.go:117] "RemoveContainer" containerID="f14f459b2bd6ceea1ae130fb2c0aae5699a4efb166d965fb37d52e43ae4e73ac"
Jul 24 17:13:00.416573 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:13:00.416560    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f14f459b2bd6ceea1ae130fb2c0aae5699a4efb166d965fb37d52e43ae4e73ac\": container with ID starting with f14f459b2bd6ceea1ae130fb2c0aae5699a4efb166d965fb37d52e43ae4e73ac not found: ID does not exist" containerID="f14f459b2bd6ceea1ae130fb2c0aae5699a4efb166d965fb37d52e43ae4e73ac"
Jul 24 17:13:00.416620 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.416576    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f14f459b2bd6ceea1ae130fb2c0aae5699a4efb166d965fb37d52e43ae4e73ac"} err="failed to get container status \"f14f459b2bd6ceea1ae130fb2c0aae5699a4efb166d965fb37d52e43ae4e73ac\": rpc error: code = NotFound desc = could not find container \"f14f459b2bd6ceea1ae130fb2c0aae5699a4efb166d965fb37d52e43ae4e73ac\": container with ID starting with f14f459b2bd6ceea1ae130fb2c0aae5699a4efb166d965fb37d52e43ae4e73ac not found: ID does not exist"
Jul 24 17:13:00.416620 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.416587    2581 scope.go:117] "RemoveContainer" containerID="009fb2a0a4bde6e5898539b03f01eb37fd82898856a1e9a835cba5de618f682f"
Jul 24 17:13:00.416697 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.416630    2581 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/284bf1c9-f105-4053-b478-b8bd4fd6625f-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:00.416697 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.416646    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdv9n\" (UniqueName: \"kubernetes.io/projected/284bf1c9-f105-4053-b478-b8bd4fd6625f-kube-api-access-mdv9n\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:00.416697 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.416655    2581 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-web-config\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:00.416697 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.416664    2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:00.416697 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.416673    2581 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/284bf1c9-f105-4053-b478-b8bd4fd6625f-tls-assets\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:00.416697 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.416682    2581 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-cluster-tls-config\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:00.416697 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.416691    2581 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/284bf1c9-f105-4053-b478-b8bd4fd6625f-alertmanager-main-db\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:00.416950 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.416704    2581 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/284bf1c9-f105-4053-b478-b8bd4fd6625f-config-out\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:00.416950 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.416719    2581 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-config-volume\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:00.416950 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.416732    2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:00.416950 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.416745    2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-secret-alertmanager-main-tls\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:00.416950 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.416758    2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/284bf1c9-f105-4053-b478-b8bd4fd6625f-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:00.416950 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:13:00.416788    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"009fb2a0a4bde6e5898539b03f01eb37fd82898856a1e9a835cba5de618f682f\": container with ID starting with 009fb2a0a4bde6e5898539b03f01eb37fd82898856a1e9a835cba5de618f682f not found: ID does not exist" containerID="009fb2a0a4bde6e5898539b03f01eb37fd82898856a1e9a835cba5de618f682f"
Jul 24 17:13:00.416950 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.416804    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"009fb2a0a4bde6e5898539b03f01eb37fd82898856a1e9a835cba5de618f682f"} err="failed to get container status \"009fb2a0a4bde6e5898539b03f01eb37fd82898856a1e9a835cba5de618f682f\": rpc error: code = NotFound desc = could not find container \"009fb2a0a4bde6e5898539b03f01eb37fd82898856a1e9a835cba5de618f682f\": container with ID starting with 009fb2a0a4bde6e5898539b03f01eb37fd82898856a1e9a835cba5de618f682f not found: ID does not exist"
Jul 24 17:13:00.416950 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.416820    2581 scope.go:117] "RemoveContainer" containerID="56241c7ac0216b720152963434b213c86dbec534a1b9556f6f008f5030b89024"
Jul 24 17:13:00.417327 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:13:00.417085    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56241c7ac0216b720152963434b213c86dbec534a1b9556f6f008f5030b89024\": container with ID starting with 56241c7ac0216b720152963434b213c86dbec534a1b9556f6f008f5030b89024 not found: ID does not exist" containerID="56241c7ac0216b720152963434b213c86dbec534a1b9556f6f008f5030b89024"
Jul 24 17:13:00.417327 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.417115    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56241c7ac0216b720152963434b213c86dbec534a1b9556f6f008f5030b89024"} err="failed to get container status \"56241c7ac0216b720152963434b213c86dbec534a1b9556f6f008f5030b89024\": rpc error: code = NotFound desc = could not find container \"56241c7ac0216b720152963434b213c86dbec534a1b9556f6f008f5030b89024\": container with ID starting with 56241c7ac0216b720152963434b213c86dbec534a1b9556f6f008f5030b89024 not found: ID does not exist"
Jul 24 17:13:00.417327 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.417127    2581 scope.go:117] "RemoveContainer" containerID="d19670244761d96695127747d504a879cb7a42e6bd5bd74ab611f2a50fea84a4"
Jul 24 17:13:00.417481 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:13:00.417351    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d19670244761d96695127747d504a879cb7a42e6bd5bd74ab611f2a50fea84a4\": container with ID starting with d19670244761d96695127747d504a879cb7a42e6bd5bd74ab611f2a50fea84a4 not found: ID does not exist" containerID="d19670244761d96695127747d504a879cb7a42e6bd5bd74ab611f2a50fea84a4"
Jul 24 17:13:00.417481 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.417381    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d19670244761d96695127747d504a879cb7a42e6bd5bd74ab611f2a50fea84a4"} err="failed to get container status \"d19670244761d96695127747d504a879cb7a42e6bd5bd74ab611f2a50fea84a4\": rpc error: code = NotFound desc = could not find container \"d19670244761d96695127747d504a879cb7a42e6bd5bd74ab611f2a50fea84a4\": container with ID starting with d19670244761d96695127747d504a879cb7a42e6bd5bd74ab611f2a50fea84a4 not found: ID does not exist"
Jul 24 17:13:00.417481 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.417403    2581 scope.go:117] "RemoveContainer" containerID="091cd8ed99eb1a5bd646af9e4f8d62f867c20f46029a886957d5611c0a814b2e"
Jul 24 17:13:00.417647 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:13:00.417632    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"091cd8ed99eb1a5bd646af9e4f8d62f867c20f46029a886957d5611c0a814b2e\": container with ID starting with 091cd8ed99eb1a5bd646af9e4f8d62f867c20f46029a886957d5611c0a814b2e not found: ID does not exist" containerID="091cd8ed99eb1a5bd646af9e4f8d62f867c20f46029a886957d5611c0a814b2e"
Jul 24 17:13:00.417683 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.417651    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"091cd8ed99eb1a5bd646af9e4f8d62f867c20f46029a886957d5611c0a814b2e"} err="failed to get container status \"091cd8ed99eb1a5bd646af9e4f8d62f867c20f46029a886957d5611c0a814b2e\": rpc error: code = NotFound desc = could not find container \"091cd8ed99eb1a5bd646af9e4f8d62f867c20f46029a886957d5611c0a814b2e\": container with ID starting with 091cd8ed99eb1a5bd646af9e4f8d62f867c20f46029a886957d5611c0a814b2e not found: ID does not exist"
Jul 24 17:13:00.417683 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.417669    2581 scope.go:117] "RemoveContainer" containerID="daf2426b7dc7a99a0acc80af8e435df7a8579eb56f1222da0a009419edabe5a3"
Jul 24 17:13:00.417882 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.417864    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"daf2426b7dc7a99a0acc80af8e435df7a8579eb56f1222da0a009419edabe5a3"} err="failed to get container status \"daf2426b7dc7a99a0acc80af8e435df7a8579eb56f1222da0a009419edabe5a3\": rpc error: code = NotFound desc = could not find container \"daf2426b7dc7a99a0acc80af8e435df7a8579eb56f1222da0a009419edabe5a3\": container with ID starting with daf2426b7dc7a99a0acc80af8e435df7a8579eb56f1222da0a009419edabe5a3 not found: ID does not exist"
Jul 24 17:13:00.417925 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.417884    2581 scope.go:117] "RemoveContainer" containerID="7a310e1073cfa69399179bd136fa276967c52923b80d1a874e022d4b7add19e2"
Jul 24 17:13:00.418094 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.418079    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a310e1073cfa69399179bd136fa276967c52923b80d1a874e022d4b7add19e2"} err="failed to get container status \"7a310e1073cfa69399179bd136fa276967c52923b80d1a874e022d4b7add19e2\": rpc error: code = NotFound desc = could not find container \"7a310e1073cfa69399179bd136fa276967c52923b80d1a874e022d4b7add19e2\": container with ID starting with 7a310e1073cfa69399179bd136fa276967c52923b80d1a874e022d4b7add19e2 not found: ID does not exist"
Jul 24 17:13:00.418152 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.418095    2581 scope.go:117] "RemoveContainer" containerID="f14f459b2bd6ceea1ae130fb2c0aae5699a4efb166d965fb37d52e43ae4e73ac"
Jul 24 17:13:00.418271 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.418253    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f14f459b2bd6ceea1ae130fb2c0aae5699a4efb166d965fb37d52e43ae4e73ac"} err="failed to get container status \"f14f459b2bd6ceea1ae130fb2c0aae5699a4efb166d965fb37d52e43ae4e73ac\": rpc error: code = NotFound desc = could not find container \"f14f459b2bd6ceea1ae130fb2c0aae5699a4efb166d965fb37d52e43ae4e73ac\": container with ID starting with f14f459b2bd6ceea1ae130fb2c0aae5699a4efb166d965fb37d52e43ae4e73ac not found: ID does not exist"
Jul 24 17:13:00.418337 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.418272    2581 scope.go:117] "RemoveContainer" containerID="009fb2a0a4bde6e5898539b03f01eb37fd82898856a1e9a835cba5de618f682f"
Jul 24 17:13:00.418493 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.418476    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"009fb2a0a4bde6e5898539b03f01eb37fd82898856a1e9a835cba5de618f682f"} err="failed to get container status \"009fb2a0a4bde6e5898539b03f01eb37fd82898856a1e9a835cba5de618f682f\": rpc error: code = NotFound desc = could not find container \"009fb2a0a4bde6e5898539b03f01eb37fd82898856a1e9a835cba5de618f682f\": container with ID starting with 009fb2a0a4bde6e5898539b03f01eb37fd82898856a1e9a835cba5de618f682f not found: ID does not exist"
Jul 24 17:13:00.418538 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.418493    2581 scope.go:117] "RemoveContainer" containerID="56241c7ac0216b720152963434b213c86dbec534a1b9556f6f008f5030b89024"
Jul 24 17:13:00.418717 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.418698    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56241c7ac0216b720152963434b213c86dbec534a1b9556f6f008f5030b89024"} err="failed to get container status \"56241c7ac0216b720152963434b213c86dbec534a1b9556f6f008f5030b89024\": rpc error: code = NotFound desc = could not find container \"56241c7ac0216b720152963434b213c86dbec534a1b9556f6f008f5030b89024\": container with ID starting with 56241c7ac0216b720152963434b213c86dbec534a1b9556f6f008f5030b89024 not found: ID does not exist"
Jul 24 17:13:00.418717 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.418716    2581 scope.go:117] "RemoveContainer" containerID="d19670244761d96695127747d504a879cb7a42e6bd5bd74ab611f2a50fea84a4"
Jul 24 17:13:00.418900 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.418884    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d19670244761d96695127747d504a879cb7a42e6bd5bd74ab611f2a50fea84a4"} err="failed to get container status \"d19670244761d96695127747d504a879cb7a42e6bd5bd74ab611f2a50fea84a4\": rpc error: code = NotFound desc = could not find container \"d19670244761d96695127747d504a879cb7a42e6bd5bd74ab611f2a50fea84a4\": container with ID starting with d19670244761d96695127747d504a879cb7a42e6bd5bd74ab611f2a50fea84a4 not found: ID does not exist"
Jul 24 17:13:00.418946 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.418901    2581 scope.go:117] "RemoveContainer" containerID="091cd8ed99eb1a5bd646af9e4f8d62f867c20f46029a886957d5611c0a814b2e"
Jul 24 17:13:00.419141 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.419123    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"091cd8ed99eb1a5bd646af9e4f8d62f867c20f46029a886957d5611c0a814b2e"} err="failed to get container status \"091cd8ed99eb1a5bd646af9e4f8d62f867c20f46029a886957d5611c0a814b2e\": rpc error: code = NotFound desc = could not find container \"091cd8ed99eb1a5bd646af9e4f8d62f867c20f46029a886957d5611c0a814b2e\": container with ID starting with 091cd8ed99eb1a5bd646af9e4f8d62f867c20f46029a886957d5611c0a814b2e not found: ID does not exist"
Jul 24 17:13:00.453694 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.453664    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 17:13:00.454002 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.453981    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="284bf1c9-f105-4053-b478-b8bd4fd6625f" containerName="prom-label-proxy"
Jul 24 17:13:00.454002 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.453998    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="284bf1c9-f105-4053-b478-b8bd4fd6625f" containerName="prom-label-proxy"
Jul 24 17:13:00.454388 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.454011    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="284bf1c9-f105-4053-b478-b8bd4fd6625f" containerName="alertmanager"
Jul 24 17:13:00.454388 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.454016    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="284bf1c9-f105-4053-b478-b8bd4fd6625f" containerName="alertmanager"
Jul 24 17:13:00.454388 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.454025    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b111fa18-0380-470b-9d3c-ec7992007d20" containerName="console"
Jul 24 17:13:00.454388 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.454049    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b111fa18-0380-470b-9d3c-ec7992007d20" containerName="console"
Jul 24 17:13:00.454388 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.454064    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="284bf1c9-f105-4053-b478-b8bd4fd6625f" containerName="config-reloader"
Jul 24 17:13:00.454388 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.454072    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="284bf1c9-f105-4053-b478-b8bd4fd6625f" containerName="config-reloader"
Jul 24 17:13:00.454388 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.454085    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="284bf1c9-f105-4053-b478-b8bd4fd6625f" containerName="kube-rbac-proxy-web"
Jul 24 17:13:00.454388 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.454094    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="284bf1c9-f105-4053-b478-b8bd4fd6625f" containerName="kube-rbac-proxy-web"
Jul 24 17:13:00.454388 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.454111    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="284bf1c9-f105-4053-b478-b8bd4fd6625f" containerName="init-config-reloader"
Jul 24 17:13:00.454388 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.454119    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="284bf1c9-f105-4053-b478-b8bd4fd6625f" containerName="init-config-reloader"
Jul 24 17:13:00.454388 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.454129    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="284bf1c9-f105-4053-b478-b8bd4fd6625f" containerName="kube-rbac-proxy"
Jul 24 17:13:00.454388 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.454136    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="284bf1c9-f105-4053-b478-b8bd4fd6625f" containerName="kube-rbac-proxy"
Jul 24 17:13:00.454388 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.454147    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="284bf1c9-f105-4053-b478-b8bd4fd6625f" containerName="kube-rbac-proxy-metric"
Jul 24 17:13:00.454388 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.454155    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="284bf1c9-f105-4053-b478-b8bd4fd6625f" containerName="kube-rbac-proxy-metric"
Jul 24 17:13:00.454388 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.454245    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="284bf1c9-f105-4053-b478-b8bd4fd6625f" containerName="kube-rbac-proxy-web"
Jul 24 17:13:00.454388 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.454258    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="284bf1c9-f105-4053-b478-b8bd4fd6625f" containerName="config-reloader"
Jul 24 17:13:00.454388 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.454268    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b111fa18-0380-470b-9d3c-ec7992007d20" containerName="console"
Jul 24 17:13:00.454388 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.454278    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="284bf1c9-f105-4053-b478-b8bd4fd6625f" containerName="kube-rbac-proxy-metric"
Jul 24 17:13:00.454388 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.454286    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="284bf1c9-f105-4053-b478-b8bd4fd6625f" containerName="alertmanager"
Jul 24 17:13:00.454388 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.454294    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="284bf1c9-f105-4053-b478-b8bd4fd6625f" containerName="kube-rbac-proxy"
Jul 24 17:13:00.454388 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.454303    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="284bf1c9-f105-4053-b478-b8bd4fd6625f" containerName="prom-label-proxy"
Jul 24 17:13:00.458310 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.458294    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.461261 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.461238    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 24 17:13:00.461362 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.461265    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 24 17:13:00.461362 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.461238    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 24 17:13:00.461362 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.461238    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-fz926\""
Jul 24 17:13:00.461496 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.461423    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 24 17:13:00.461752 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.461733    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 24 17:13:00.461879 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.461789    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 24 17:13:00.461933 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.461907    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 24 17:13:00.461986 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.461974    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 24 17:13:00.466403 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.466387    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 24 17:13:00.475380 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.475359    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 17:13:00.518110 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.518082    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.518110 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.518116    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-web-config\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.518373 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.518145    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.518373 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.518167    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.518373 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.518186    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.518373 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.518250    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.518373 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.518301    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-config-out\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.518373 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.518339    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-779m2\" (UniqueName: \"kubernetes.io/projected/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-kube-api-access-779m2\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.518373 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.518366    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.518685 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.518413    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-config-volume\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.518685 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.518445    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.518685 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.518473    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.518685 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.518528    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.619506 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.619426    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-config-out\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.619506 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.619467    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-779m2\" (UniqueName: \"kubernetes.io/projected/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-kube-api-access-779m2\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.619506 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.619486    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.619772 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.619506    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-config-volume\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.619772 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.619540    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.619772 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.619568    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.619772 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.619604    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.619772 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.619643    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.619772 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.619669    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-web-config\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.619772 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.619711    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.619772 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.619740    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.619772 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.619767    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.620243 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.619806    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.620950 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.620425    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.620950 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.620516    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.621567 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.621540    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.622735 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.622695    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-config-volume\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.622830 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.622744    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-config-out\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.622949 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.622926    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.623059 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.623017    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.623290 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.623263    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.623381 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.623369    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-web-config\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.623585 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.623568    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.623815 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.623797    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.624665 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.624650    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.628133 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.628117    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-779m2\" (UniqueName: \"kubernetes.io/projected/209f3273-8d94-4c95-8b73-dbe24cdfe4d2-kube-api-access-779m2\") pod \"alertmanager-main-0\" (UID: \"209f3273-8d94-4c95-8b73-dbe24cdfe4d2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.767849 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.767814    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:13:00.905447 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:00.905422    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 17:13:00.908218 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:13:00.908173    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod209f3273_8d94_4c95_8b73_dbe24cdfe4d2.slice/crio-f3134fc0838c8691b781b03652c9ab3b2c7c7142a408ab3878ce41f2ff46ffa8 WatchSource:0}: Error finding container f3134fc0838c8691b781b03652c9ab3b2c7c7142a408ab3878ce41f2ff46ffa8: Status 404 returned error can't find the container with id f3134fc0838c8691b781b03652c9ab3b2c7c7142a408ab3878ce41f2ff46ffa8
Jul 24 17:13:01.368801 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:01.368700    2581 generic.go:356] "Generic (PLEG): container finished" podID="209f3273-8d94-4c95-8b73-dbe24cdfe4d2" containerID="b0cbf4938d896506940760212bb9ac7420436cca23a245f406fa6beb27acdbba" exitCode=0
Jul 24 17:13:01.368801 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:01.368770    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"209f3273-8d94-4c95-8b73-dbe24cdfe4d2","Type":"ContainerDied","Data":"b0cbf4938d896506940760212bb9ac7420436cca23a245f406fa6beb27acdbba"}
Jul 24 17:13:01.369265 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:01.368837    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"209f3273-8d94-4c95-8b73-dbe24cdfe4d2","Type":"ContainerStarted","Data":"f3134fc0838c8691b781b03652c9ab3b2c7c7142a408ab3878ce41f2ff46ffa8"}
Jul 24 17:13:01.383430 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:01.383402    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="284bf1c9-f105-4053-b478-b8bd4fd6625f" path="/var/lib/kubelet/pods/284bf1c9-f105-4053-b478-b8bd4fd6625f/volumes"
Jul 24 17:13:02.305113 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:02.305076    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:13:02.371699 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:02.371655    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:13:02.376439 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:02.376413    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"209f3273-8d94-4c95-8b73-dbe24cdfe4d2","Type":"ContainerStarted","Data":"c348f1eb8e388d9d83ce7670bedc43c81a711f6bf714e6864a26bb67775120e3"}
Jul 24 17:13:02.376540 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:02.376520    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"209f3273-8d94-4c95-8b73-dbe24cdfe4d2","Type":"ContainerStarted","Data":"59e481889f38aec251eaee6b0ddce1d1123943b2c6a4c5bd32c721e42a52ae33"}
Jul 24 17:13:02.376590 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:02.376566    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"209f3273-8d94-4c95-8b73-dbe24cdfe4d2","Type":"ContainerStarted","Data":"371b2fbc6daf4587ec53df1c4b59bba6acf6f05a1778f547368eb73aace386f7"}
Jul 24 17:13:02.376630 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:02.376603    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"209f3273-8d94-4c95-8b73-dbe24cdfe4d2","Type":"ContainerStarted","Data":"5161df1d3b05623c658ea0b6dc33f65079ee85cf1659b9ed5ca8837fc089db74"}
Jul 24 17:13:02.376668 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:02.376638    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"209f3273-8d94-4c95-8b73-dbe24cdfe4d2","Type":"ContainerStarted","Data":"327f2117900d3a085a64c7a8a9ce7b34bbe8bade001d35a3bc2323088ef7273b"}
Jul 24 17:13:02.376707 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:02.376680    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"209f3273-8d94-4c95-8b73-dbe24cdfe4d2","Type":"ContainerStarted","Data":"3ee5b10449c7c133e2dde163d6a1e47106f5e6c5882334f21f7a643922f09d51"}
Jul 24 17:13:02.407435 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:02.407380    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.391547657 podStartE2EDuration="2.407362551s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.814894ms" isStatefulPod=false podCreationTimestamp="2026-07-24 17:13:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:13:02.404640481 +0000 UTC m=+149.606689779" watchObservedRunningTime="2026-07-24 17:13:02.407362551 +0000 UTC m=+149.609411857"
Jul 24 17:13:03.253386 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.253352    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 17:13:03.253721 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.253672    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2d8f8104-653f-4c52-850b-56e496c30884" containerName="kube-rbac-proxy" containerID="cri-o://e91d6684aac1046ff4df10d33b6ecf519ae53ced15e25171d6a6e582a1c4ea0c" gracePeriod=600
Jul 24 17:13:03.253966 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.253669    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2d8f8104-653f-4c52-850b-56e496c30884" containerName="prometheus" containerID="cri-o://61aeae10049aa845078ce01582f6264a477edd96823ec4433c915634e71d1fea" gracePeriod=600
Jul 24 17:13:03.254086 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.253708    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2d8f8104-653f-4c52-850b-56e496c30884" containerName="thanos-sidecar" containerID="cri-o://058dd73dd5d71eba7411e7837e86e8f89ffdb4a84e36690223974e26aa1cdf7b" gracePeriod=600
Jul 24 17:13:03.254158 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.253723    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2d8f8104-653f-4c52-850b-56e496c30884" containerName="kube-rbac-proxy-web" containerID="cri-o://828371c79c36ed7658a3577fc6bd6996a79123f8c39d75b78f2c76fc8b6a799f" gracePeriod=600
Jul 24 17:13:03.254216 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.253738    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2d8f8104-653f-4c52-850b-56e496c30884" containerName="config-reloader" containerID="cri-o://06f613e9d037295d6cd807636b0bab5f9942d9ced65105df8c49c1c6dfad057d" gracePeriod=600
Jul 24 17:13:03.254270 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.253745    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2d8f8104-653f-4c52-850b-56e496c30884" containerName="kube-rbac-proxy-thanos" containerID="cri-o://65f8e05ab790aeded00fd275a7c7bf4d610ca5284044e60aea5b384d93e0556c" gracePeriod=600
Jul 24 17:13:03.386485 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.386456    2581 generic.go:356] "Generic (PLEG): container finished" podID="2d8f8104-653f-4c52-850b-56e496c30884" containerID="65f8e05ab790aeded00fd275a7c7bf4d610ca5284044e60aea5b384d93e0556c" exitCode=0
Jul 24 17:13:03.386485 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.386479    2581 generic.go:356] "Generic (PLEG): container finished" podID="2d8f8104-653f-4c52-850b-56e496c30884" containerID="e91d6684aac1046ff4df10d33b6ecf519ae53ced15e25171d6a6e582a1c4ea0c" exitCode=0
Jul 24 17:13:03.386485 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.386485    2581 generic.go:356] "Generic (PLEG): container finished" podID="2d8f8104-653f-4c52-850b-56e496c30884" containerID="828371c79c36ed7658a3577fc6bd6996a79123f8c39d75b78f2c76fc8b6a799f" exitCode=0
Jul 24 17:13:03.386485 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.386491    2581 generic.go:356] "Generic (PLEG): container finished" podID="2d8f8104-653f-4c52-850b-56e496c30884" containerID="058dd73dd5d71eba7411e7837e86e8f89ffdb4a84e36690223974e26aa1cdf7b" exitCode=0
Jul 24 17:13:03.387112 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.386496    2581 generic.go:356] "Generic (PLEG): container finished" podID="2d8f8104-653f-4c52-850b-56e496c30884" containerID="06f613e9d037295d6cd807636b0bab5f9942d9ced65105df8c49c1c6dfad057d" exitCode=0
Jul 24 17:13:03.387112 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.386508    2581 generic.go:356] "Generic (PLEG): container finished" podID="2d8f8104-653f-4c52-850b-56e496c30884" containerID="61aeae10049aa845078ce01582f6264a477edd96823ec4433c915634e71d1fea" exitCode=0
Jul 24 17:13:03.387112 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.386538    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d8f8104-653f-4c52-850b-56e496c30884","Type":"ContainerDied","Data":"65f8e05ab790aeded00fd275a7c7bf4d610ca5284044e60aea5b384d93e0556c"}
Jul 24 17:13:03.387112 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.386613    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d8f8104-653f-4c52-850b-56e496c30884","Type":"ContainerDied","Data":"e91d6684aac1046ff4df10d33b6ecf519ae53ced15e25171d6a6e582a1c4ea0c"}
Jul 24 17:13:03.387112 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.386643    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d8f8104-653f-4c52-850b-56e496c30884","Type":"ContainerDied","Data":"828371c79c36ed7658a3577fc6bd6996a79123f8c39d75b78f2c76fc8b6a799f"}
Jul 24 17:13:03.387112 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.386669    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d8f8104-653f-4c52-850b-56e496c30884","Type":"ContainerDied","Data":"058dd73dd5d71eba7411e7837e86e8f89ffdb4a84e36690223974e26aa1cdf7b"}
Jul 24 17:13:03.387112 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.386693    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d8f8104-653f-4c52-850b-56e496c30884","Type":"ContainerDied","Data":"06f613e9d037295d6cd807636b0bab5f9942d9ced65105df8c49c1c6dfad057d"}
Jul 24 17:13:03.387112 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.386716    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d8f8104-653f-4c52-850b-56e496c30884","Type":"ContainerDied","Data":"61aeae10049aa845078ce01582f6264a477edd96823ec4433c915634e71d1fea"}
Jul 24 17:13:03.521589 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.521568    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:03.661505 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.661472    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-secret-prometheus-k8s-tls\") pod \"2d8f8104-653f-4c52-850b-56e496c30884\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") "
Jul 24 17:13:03.661505 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.661506    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-secret-kube-rbac-proxy\") pod \"2d8f8104-653f-4c52-850b-56e496c30884\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") "
Jul 24 17:13:03.661759 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.661535    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-thanos-prometheus-http-client-file\") pod \"2d8f8104-653f-4c52-850b-56e496c30884\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") "
Jul 24 17:13:03.661759 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.661561    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-secret-metrics-client-certs\") pod \"2d8f8104-653f-4c52-850b-56e496c30884\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") "
Jul 24 17:13:03.661759 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.661601    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2d8f8104-653f-4c52-850b-56e496c30884-tls-assets\") pod \"2d8f8104-653f-4c52-850b-56e496c30884\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") "
Jul 24 17:13:03.661759 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.661632    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-configmap-metrics-client-ca\") pod \"2d8f8104-653f-4c52-850b-56e496c30884\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") "
Jul 24 17:13:03.661759 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.661653    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-web-config\") pod \"2d8f8104-653f-4c52-850b-56e496c30884\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") "
Jul 24 17:13:03.661759 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.661684    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pg7hv\" (UniqueName: \"kubernetes.io/projected/2d8f8104-653f-4c52-850b-56e496c30884-kube-api-access-pg7hv\") pod \"2d8f8104-653f-4c52-850b-56e496c30884\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") "
Jul 24 17:13:03.661759 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.661738    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-configmap-serving-certs-ca-bundle\") pod \"2d8f8104-653f-4c52-850b-56e496c30884\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") "
Jul 24 17:13:03.662134 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.661766    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-secret-grpc-tls\") pod \"2d8f8104-653f-4c52-850b-56e496c30884\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") "
Jul 24 17:13:03.662134 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.661800    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2d8f8104-653f-4c52-850b-56e496c30884-prometheus-k8s-db\") pod \"2d8f8104-653f-4c52-850b-56e496c30884\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") "
Jul 24 17:13:03.662134 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.661840    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-configmap-kubelet-serving-ca-bundle\") pod \"2d8f8104-653f-4c52-850b-56e496c30884\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") "
Jul 24 17:13:03.662134 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.661882    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"2d8f8104-653f-4c52-850b-56e496c30884\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") "
Jul 24 17:13:03.662134 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.661919    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2d8f8104-653f-4c52-850b-56e496c30884-config-out\") pod \"2d8f8104-653f-4c52-850b-56e496c30884\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") "
Jul 24 17:13:03.662134 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.661962    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-prometheus-k8s-rulefiles-2\") pod \"2d8f8104-653f-4c52-850b-56e496c30884\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") "
Jul 24 17:13:03.662134 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.661996    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"2d8f8104-653f-4c52-850b-56e496c30884\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") "
Jul 24 17:13:03.662134 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.662066    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-prometheus-k8s-rulefiles-1\") pod \"2d8f8104-653f-4c52-850b-56e496c30884\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") "
Jul 24 17:13:03.662134 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.662103    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-prometheus-k8s-rulefiles-0\") pod \"2d8f8104-653f-4c52-850b-56e496c30884\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") "
Jul 24 17:13:03.662134 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.662137    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-prometheus-trusted-ca-bundle\") pod \"2d8f8104-653f-4c52-850b-56e496c30884\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") "
Jul 24 17:13:03.662776 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.662162    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-config\") pod \"2d8f8104-653f-4c52-850b-56e496c30884\" (UID: \"2d8f8104-653f-4c52-850b-56e496c30884\") "
Jul 24 17:13:03.662776 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.662491    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "2d8f8104-653f-4c52-850b-56e496c30884" (UID: "2d8f8104-653f-4c52-850b-56e496c30884"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:13:03.662776 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.662655    2581 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-configmap-metrics-client-ca\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:03.664351 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.664318    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "2d8f8104-653f-4c52-850b-56e496c30884" (UID: "2d8f8104-653f-4c52-850b-56e496c30884"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:13:03.664633 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.664486    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "2d8f8104-653f-4c52-850b-56e496c30884" (UID: "2d8f8104-653f-4c52-850b-56e496c30884"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:13:03.664743 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.664721    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "2d8f8104-653f-4c52-850b-56e496c30884" (UID: "2d8f8104-653f-4c52-850b-56e496c30884"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:13:03.664802 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.664731    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "2d8f8104-653f-4c52-850b-56e496c30884" (UID: "2d8f8104-653f-4c52-850b-56e496c30884"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:13:03.664802 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.664777    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d8f8104-653f-4c52-850b-56e496c30884-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "2d8f8104-653f-4c52-850b-56e496c30884" (UID: "2d8f8104-653f-4c52-850b-56e496c30884"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:13:03.664908 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.664873    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "2d8f8104-653f-4c52-850b-56e496c30884" (UID: "2d8f8104-653f-4c52-850b-56e496c30884"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:13:03.664955 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.664922    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "2d8f8104-653f-4c52-850b-56e496c30884" (UID: "2d8f8104-653f-4c52-850b-56e496c30884"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:13:03.664990 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.664951    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-config" (OuterVolumeSpecName: "config") pod "2d8f8104-653f-4c52-850b-56e496c30884" (UID: "2d8f8104-653f-4c52-850b-56e496c30884"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:13:03.665182 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.665126    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "2d8f8104-653f-4c52-850b-56e496c30884" (UID: "2d8f8104-653f-4c52-850b-56e496c30884"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:13:03.665283 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.665195    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "2d8f8104-653f-4c52-850b-56e496c30884" (UID: "2d8f8104-653f-4c52-850b-56e496c30884"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:13:03.665692 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.665651    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "2d8f8104-653f-4c52-850b-56e496c30884" (UID: "2d8f8104-653f-4c52-850b-56e496c30884"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:13:03.665768 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.665748    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "2d8f8104-653f-4c52-850b-56e496c30884" (UID: "2d8f8104-653f-4c52-850b-56e496c30884"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:13:03.666149 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.666107    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d8f8104-653f-4c52-850b-56e496c30884-kube-api-access-pg7hv" (OuterVolumeSpecName: "kube-api-access-pg7hv") pod "2d8f8104-653f-4c52-850b-56e496c30884" (UID: "2d8f8104-653f-4c52-850b-56e496c30884"). InnerVolumeSpecName "kube-api-access-pg7hv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:13:03.666233 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.666196    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d8f8104-653f-4c52-850b-56e496c30884-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "2d8f8104-653f-4c52-850b-56e496c30884" (UID: "2d8f8104-653f-4c52-850b-56e496c30884"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:13:03.666663 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.666638    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "2d8f8104-653f-4c52-850b-56e496c30884" (UID: "2d8f8104-653f-4c52-850b-56e496c30884"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:13:03.667371 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.667346    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "2d8f8104-653f-4c52-850b-56e496c30884" (UID: "2d8f8104-653f-4c52-850b-56e496c30884"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:13:03.667570 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.667553    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "2d8f8104-653f-4c52-850b-56e496c30884" (UID: "2d8f8104-653f-4c52-850b-56e496c30884"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:13:03.667865 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.667842    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d8f8104-653f-4c52-850b-56e496c30884-config-out" (OuterVolumeSpecName: "config-out") pod "2d8f8104-653f-4c52-850b-56e496c30884" (UID: "2d8f8104-653f-4c52-850b-56e496c30884"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:13:03.682193 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.681849    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-web-config" (OuterVolumeSpecName: "web-config") pod "2d8f8104-653f-4c52-850b-56e496c30884" (UID: "2d8f8104-653f-4c52-850b-56e496c30884"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:13:03.763352 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.763276    2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:03.763352 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.763300    2581 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2d8f8104-653f-4c52-850b-56e496c30884-config-out\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:03.763352 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.763310    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:03.763352 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.763319    2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:03.763352 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.763328    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:03.763352 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.763336    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:03.763352 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.763345    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-prometheus-trusted-ca-bundle\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:03.763352 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.763354    2581 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-config\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:03.763702 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.763364    2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-secret-prometheus-k8s-tls\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:03.763702 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.763373    2581 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-secret-kube-rbac-proxy\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:03.763702 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.763381    2581 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-thanos-prometheus-http-client-file\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:03.763702 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.763389    2581 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-secret-metrics-client-certs\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:03.763702 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.763396    2581 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2d8f8104-653f-4c52-850b-56e496c30884-tls-assets\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:03.763702 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.763423    2581 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-web-config\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:03.763702 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.763431    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pg7hv\" (UniqueName: \"kubernetes.io/projected/2d8f8104-653f-4c52-850b-56e496c30884-kube-api-access-pg7hv\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:03.763702 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.763439    2581 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:03.763702 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.763447    2581 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2d8f8104-653f-4c52-850b-56e496c30884-secret-grpc-tls\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:03.763702 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.763455    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2d8f8104-653f-4c52-850b-56e496c30884-prometheus-k8s-db\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:03.763702 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:03.763464    2581 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d8f8104-653f-4c52-850b-56e496c30884-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:04.393929 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.393898    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d8f8104-653f-4c52-850b-56e496c30884","Type":"ContainerDied","Data":"b6995ad6a876c419645ebbd238422c58a963062cf5ff3f6004bf65a169e9c314"}
Jul 24 17:13:04.394344 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.393938    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.394344 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.393969    2581 scope.go:117] "RemoveContainer" containerID="65f8e05ab790aeded00fd275a7c7bf4d610ca5284044e60aea5b384d93e0556c"
Jul 24 17:13:04.402874 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.402858    2581 scope.go:117] "RemoveContainer" containerID="e91d6684aac1046ff4df10d33b6ecf519ae53ced15e25171d6a6e582a1c4ea0c"
Jul 24 17:13:04.410199 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.410185    2581 scope.go:117] "RemoveContainer" containerID="828371c79c36ed7658a3577fc6bd6996a79123f8c39d75b78f2c76fc8b6a799f"
Jul 24 17:13:04.417775 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.417754    2581 scope.go:117] "RemoveContainer" containerID="058dd73dd5d71eba7411e7837e86e8f89ffdb4a84e36690223974e26aa1cdf7b"
Jul 24 17:13:04.418639 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.418614    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 17:13:04.423256 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.423238    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 17:13:04.425593 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.425567    2581 scope.go:117] "RemoveContainer" containerID="06f613e9d037295d6cd807636b0bab5f9942d9ced65105df8c49c1c6dfad057d"
Jul 24 17:13:04.432397 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.432381    2581 scope.go:117] "RemoveContainer" containerID="61aeae10049aa845078ce01582f6264a477edd96823ec4433c915634e71d1fea"
Jul 24 17:13:04.440504 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.440487    2581 scope.go:117] "RemoveContainer" containerID="c16969c06171d5e3126c475a855fb259c859a2f8c35c6c4552ffe65284b20b95"
Jul 24 17:13:04.447412 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.447385    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 17:13:04.447880 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.447861    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d8f8104-653f-4c52-850b-56e496c30884" containerName="thanos-sidecar"
Jul 24 17:13:04.447917 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.447888    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d8f8104-653f-4c52-850b-56e496c30884" containerName="thanos-sidecar"
Jul 24 17:13:04.447917 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.447907    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d8f8104-653f-4c52-850b-56e496c30884" containerName="config-reloader"
Jul 24 17:13:04.447976 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.447916    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d8f8104-653f-4c52-850b-56e496c30884" containerName="config-reloader"
Jul 24 17:13:04.447976 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.447937    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d8f8104-653f-4c52-850b-56e496c30884" containerName="kube-rbac-proxy"
Jul 24 17:13:04.447976 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.447946    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d8f8104-653f-4c52-850b-56e496c30884" containerName="kube-rbac-proxy"
Jul 24 17:13:04.447976 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.447960    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d8f8104-653f-4c52-850b-56e496c30884" containerName="init-config-reloader"
Jul 24 17:13:04.447976 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.447968    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d8f8104-653f-4c52-850b-56e496c30884" containerName="init-config-reloader"
Jul 24 17:13:04.448177 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.447983    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d8f8104-653f-4c52-850b-56e496c30884" containerName="kube-rbac-proxy-thanos"
Jul 24 17:13:04.448177 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.447992    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d8f8104-653f-4c52-850b-56e496c30884" containerName="kube-rbac-proxy-thanos"
Jul 24 17:13:04.448177 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.448008    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d8f8104-653f-4c52-850b-56e496c30884" containerName="prometheus"
Jul 24 17:13:04.448177 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.448017    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d8f8104-653f-4c52-850b-56e496c30884" containerName="prometheus"
Jul 24 17:13:04.448177 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.448056    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d8f8104-653f-4c52-850b-56e496c30884" containerName="kube-rbac-proxy-web"
Jul 24 17:13:04.448177 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.448065    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d8f8104-653f-4c52-850b-56e496c30884" containerName="kube-rbac-proxy-web"
Jul 24 17:13:04.448177 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.448139    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d8f8104-653f-4c52-850b-56e496c30884" containerName="kube-rbac-proxy-thanos"
Jul 24 17:13:04.448177 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.448152    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d8f8104-653f-4c52-850b-56e496c30884" containerName="thanos-sidecar"
Jul 24 17:13:04.448177 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.448162    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d8f8104-653f-4c52-850b-56e496c30884" containerName="kube-rbac-proxy-web"
Jul 24 17:13:04.448177 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.448174    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d8f8104-653f-4c52-850b-56e496c30884" containerName="config-reloader"
Jul 24 17:13:04.448435 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.448183    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d8f8104-653f-4c52-850b-56e496c30884" containerName="prometheus"
Jul 24 17:13:04.448435 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.448194    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d8f8104-653f-4c52-850b-56e496c30884" containerName="kube-rbac-proxy"
Jul 24 17:13:04.453222 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.453200    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.456101 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.456079    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 24 17:13:04.456101 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.456094    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 24 17:13:04.456258 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.456133    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 24 17:13:04.456258 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.456186    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-8db87\""
Jul 24 17:13:04.456362 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.456351    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 24 17:13:04.456412 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.456364    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-28kkn82cat3cn\""
Jul 24 17:13:04.456412 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.456387    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 24 17:13:04.456697 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.456670    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 24 17:13:04.456812 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.456778    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 24 17:13:04.456896 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.456825    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 24 17:13:04.457501 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.457323    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 24 17:13:04.457501 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.457329    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 24 17:13:04.457501 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.457351    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 24 17:13:04.458054 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.458008    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 24 17:13:04.460237 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.460206    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 24 17:13:04.462136 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.462119    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 24 17:13:04.468707 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.468687    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 17:13:04.569427 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.569344    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f5ce6020-a3c4-45cb-ae9b-086df02dd950-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.569427 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.569380    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f5ce6020-a3c4-45cb-ae9b-086df02dd950-config\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.569627 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.569441    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f5ce6020-a3c4-45cb-ae9b-086df02dd950-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.569627 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.569466    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f5ce6020-a3c4-45cb-ae9b-086df02dd950-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.569627 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.569489    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f5ce6020-a3c4-45cb-ae9b-086df02dd950-web-config\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.569627 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.569519    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f5ce6020-a3c4-45cb-ae9b-086df02dd950-config-out\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.569627 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.569533    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f5ce6020-a3c4-45cb-ae9b-086df02dd950-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.569627 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.569563    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5ce6020-a3c4-45cb-ae9b-086df02dd950-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.569909 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.569637    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f5ce6020-a3c4-45cb-ae9b-086df02dd950-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.569909 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.569698    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f5ce6020-a3c4-45cb-ae9b-086df02dd950-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.569909 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.569731    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcrk6\" (UniqueName: \"kubernetes.io/projected/f5ce6020-a3c4-45cb-ae9b-086df02dd950-kube-api-access-gcrk6\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.569909 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.569763    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f5ce6020-a3c4-45cb-ae9b-086df02dd950-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.569909 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.569804    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f5ce6020-a3c4-45cb-ae9b-086df02dd950-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.569909 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.569834    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5ce6020-a3c4-45cb-ae9b-086df02dd950-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.569909 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.569880    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5ce6020-a3c4-45cb-ae9b-086df02dd950-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.569909 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.569907    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f5ce6020-a3c4-45cb-ae9b-086df02dd950-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.570242 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.569930    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f5ce6020-a3c4-45cb-ae9b-086df02dd950-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.570242 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.569958    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f5ce6020-a3c4-45cb-ae9b-086df02dd950-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.570242 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.569986    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f5ce6020-a3c4-45cb-ae9b-086df02dd950-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.570242 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.570008    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f5ce6020-a3c4-45cb-ae9b-086df02dd950-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.671375 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.671327    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f5ce6020-a3c4-45cb-ae9b-086df02dd950-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.671375 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.671376    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcrk6\" (UniqueName: \"kubernetes.io/projected/f5ce6020-a3c4-45cb-ae9b-086df02dd950-kube-api-access-gcrk6\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.671583 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.671488    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f5ce6020-a3c4-45cb-ae9b-086df02dd950-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.671583 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.671527    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f5ce6020-a3c4-45cb-ae9b-086df02dd950-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.671583 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.671552    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5ce6020-a3c4-45cb-ae9b-086df02dd950-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.671730 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.671584    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5ce6020-a3c4-45cb-ae9b-086df02dd950-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.671730 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.671608    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f5ce6020-a3c4-45cb-ae9b-086df02dd950-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.671730 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.671632    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f5ce6020-a3c4-45cb-ae9b-086df02dd950-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.671730 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.671658    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f5ce6020-a3c4-45cb-ae9b-086df02dd950-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.671730 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.671690    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f5ce6020-a3c4-45cb-ae9b-086df02dd950-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.671730 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.671711    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f5ce6020-a3c4-45cb-ae9b-086df02dd950-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.672021 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.671743    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f5ce6020-a3c4-45cb-ae9b-086df02dd950-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.672021 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.671772    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f5ce6020-a3c4-45cb-ae9b-086df02dd950-config\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.672021 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.671833    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f5ce6020-a3c4-45cb-ae9b-086df02dd950-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.672021 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.671862    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f5ce6020-a3c4-45cb-ae9b-086df02dd950-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.672021 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.671891    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f5ce6020-a3c4-45cb-ae9b-086df02dd950-web-config\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.672021 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.671913    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f5ce6020-a3c4-45cb-ae9b-086df02dd950-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.672021 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.671945    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f5ce6020-a3c4-45cb-ae9b-086df02dd950-config-out\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.672021 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.671970    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f5ce6020-a3c4-45cb-ae9b-086df02dd950-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.672021 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.672001    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5ce6020-a3c4-45cb-ae9b-086df02dd950-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.672472 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.672026    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f5ce6020-a3c4-45cb-ae9b-086df02dd950-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.672472 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.672362    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5ce6020-a3c4-45cb-ae9b-086df02dd950-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.672472 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.672391    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5ce6020-a3c4-45cb-ae9b-086df02dd950-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.672617 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.672543    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f5ce6020-a3c4-45cb-ae9b-086df02dd950-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.673524 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.673500    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f5ce6020-a3c4-45cb-ae9b-086df02dd950-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.675198 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.675009    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f5ce6020-a3c4-45cb-ae9b-086df02dd950-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.675198 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.675067    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f5ce6020-a3c4-45cb-ae9b-086df02dd950-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.675358 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.675297    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f5ce6020-a3c4-45cb-ae9b-086df02dd950-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.675657 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.675496    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f5ce6020-a3c4-45cb-ae9b-086df02dd950-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.675896 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.675763    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f5ce6020-a3c4-45cb-ae9b-086df02dd950-web-config\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.675896 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.675772    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f5ce6020-a3c4-45cb-ae9b-086df02dd950-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.675896 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.675792    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/f5ce6020-a3c4-45cb-ae9b-086df02dd950-config\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.676501 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.676477    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f5ce6020-a3c4-45cb-ae9b-086df02dd950-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.676820 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.676790    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5ce6020-a3c4-45cb-ae9b-086df02dd950-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.677439 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.677408    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f5ce6020-a3c4-45cb-ae9b-086df02dd950-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.677527 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.677471    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f5ce6020-a3c4-45cb-ae9b-086df02dd950-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.677840 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.677789    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f5ce6020-a3c4-45cb-ae9b-086df02dd950-config-out\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.678132 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.678114    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f5ce6020-a3c4-45cb-ae9b-086df02dd950-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.678212 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.678191    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f5ce6020-a3c4-45cb-ae9b-086df02dd950-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.679825 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.679807    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcrk6\" (UniqueName: \"kubernetes.io/projected/f5ce6020-a3c4-45cb-ae9b-086df02dd950-kube-api-access-gcrk6\") pod \"prometheus-k8s-0\" (UID: \"f5ce6020-a3c4-45cb-ae9b-086df02dd950\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.763724 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.763668    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:04.902097 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:04.901905    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 17:13:04.904170 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:13:04.904147    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5ce6020_a3c4_45cb_ae9b_086df02dd950.slice/crio-49c3eab7f870396efd9c1c983e736618bcf02d5fa7b357118920b3b669364141 WatchSource:0}: Error finding container 49c3eab7f870396efd9c1c983e736618bcf02d5fa7b357118920b3b669364141: Status 404 returned error can't find the container with id 49c3eab7f870396efd9c1c983e736618bcf02d5fa7b357118920b3b669364141
Jul 24 17:13:05.384874 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:05.384839    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d8f8104-653f-4c52-850b-56e496c30884" path="/var/lib/kubelet/pods/2d8f8104-653f-4c52-850b-56e496c30884/volumes"
Jul 24 17:13:05.399071 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:05.399027    2581 generic.go:356] "Generic (PLEG): container finished" podID="f5ce6020-a3c4-45cb-ae9b-086df02dd950" containerID="021aadff79e0c0639af980bfb4b5b5607a08f8a294c8fa7b65b61126bdec4349" exitCode=0
Jul 24 17:13:05.399416 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:05.399079    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5ce6020-a3c4-45cb-ae9b-086df02dd950","Type":"ContainerDied","Data":"021aadff79e0c0639af980bfb4b5b5607a08f8a294c8fa7b65b61126bdec4349"}
Jul 24 17:13:05.399416 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:05.399143    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5ce6020-a3c4-45cb-ae9b-086df02dd950","Type":"ContainerStarted","Data":"49c3eab7f870396efd9c1c983e736618bcf02d5fa7b357118920b3b669364141"}
Jul 24 17:13:06.407278 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:06.407245    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5ce6020-a3c4-45cb-ae9b-086df02dd950","Type":"ContainerStarted","Data":"844a11d142e00f6de144b76e9ae2df5aba535136fc43559b05ddc11eb2d24455"}
Jul 24 17:13:06.407672 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:06.407332    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5ce6020-a3c4-45cb-ae9b-086df02dd950","Type":"ContainerStarted","Data":"a8c04549b080cbfbd07eccaacdae8ac023b80ec006c59775380940a75b55e46b"}
Jul 24 17:13:06.407672 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:06.407361    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5ce6020-a3c4-45cb-ae9b-086df02dd950","Type":"ContainerStarted","Data":"830f25d1f07d3e9668c6c862f20c76d9b268d7f7a3c6ef8a2bb5eb8367494731"}
Jul 24 17:13:06.407672 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:06.407385    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5ce6020-a3c4-45cb-ae9b-086df02dd950","Type":"ContainerStarted","Data":"17482a3272e6da546d4d6dc7a65e648f97907cd7c7384e27af6ac02583a4202f"}
Jul 24 17:13:06.407672 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:06.407406    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5ce6020-a3c4-45cb-ae9b-086df02dd950","Type":"ContainerStarted","Data":"1cfb417b0c3e8381de7798b02f58fdd537f4e745da0e3ebe0236c4417f127600"}
Jul 24 17:13:06.407672 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:06.407428    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5ce6020-a3c4-45cb-ae9b-086df02dd950","Type":"ContainerStarted","Data":"ba5e0d169ae2292b333e7029c9edfcdac4472ed159cdb6e1bb101e40798274cf"}
Jul 24 17:13:06.435296 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:06.435233    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.4153422940000002 podStartE2EDuration="2.435214144s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.87185ms" isStatefulPod=false podCreationTimestamp="2026-07-24 17:13:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:13:06.433268875 +0000 UTC m=+153.635318164" watchObservedRunningTime="2026-07-24 17:13:06.435214144 +0000 UTC m=+153.637263419"
Jul 24 17:13:06.652211 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:06.652182    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:13:06.701692 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:06.701617    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:13:09.764249 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:09.764199    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:13:13.059049 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:13.059014    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c548b8c6d-drlkk"]
Jul 24 17:13:13.154997 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:13.154960    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:13:13.255870 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:13.255838    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:13:13.309597 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:13.309510    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:13:14.923629 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:14.923588    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:13:15.024568 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:15.024525    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:13:38.078789 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:38.078727    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6c548b8c6d-drlkk" podUID="5498695e-9a6a-4e67-9fca-ae3c027a48e6" containerName="console" containerID="cri-o://64f5452dbb5416dd80fadfecfb7b8f8b09a2fff068840f023015b894d079201c" gracePeriod=15
Jul 24 17:13:38.322752 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:38.322726    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c548b8c6d-drlkk"
Jul 24 17:13:38.378929 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:38.378858    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5498695e-9a6a-4e67-9fca-ae3c027a48e6-console-serving-cert\") pod \"5498695e-9a6a-4e67-9fca-ae3c027a48e6\" (UID: \"5498695e-9a6a-4e67-9fca-ae3c027a48e6\") "
Jul 24 17:13:38.378929 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:38.378890    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2fqvt\" (UniqueName: \"kubernetes.io/projected/5498695e-9a6a-4e67-9fca-ae3c027a48e6-kube-api-access-2fqvt\") pod \"5498695e-9a6a-4e67-9fca-ae3c027a48e6\" (UID: \"5498695e-9a6a-4e67-9fca-ae3c027a48e6\") "
Jul 24 17:13:38.379168 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:38.378933    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5498695e-9a6a-4e67-9fca-ae3c027a48e6-service-ca\") pod \"5498695e-9a6a-4e67-9fca-ae3c027a48e6\" (UID: \"5498695e-9a6a-4e67-9fca-ae3c027a48e6\") "
Jul 24 17:13:38.379168 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:38.378987    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5498695e-9a6a-4e67-9fca-ae3c027a48e6-console-oauth-config\") pod \"5498695e-9a6a-4e67-9fca-ae3c027a48e6\" (UID: \"5498695e-9a6a-4e67-9fca-ae3c027a48e6\") "
Jul 24 17:13:38.379168 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:38.379068    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5498695e-9a6a-4e67-9fca-ae3c027a48e6-console-config\") pod \"5498695e-9a6a-4e67-9fca-ae3c027a48e6\" (UID: \"5498695e-9a6a-4e67-9fca-ae3c027a48e6\") "
Jul 24 17:13:38.379168 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:38.379096    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5498695e-9a6a-4e67-9fca-ae3c027a48e6-trusted-ca-bundle\") pod \"5498695e-9a6a-4e67-9fca-ae3c027a48e6\" (UID: \"5498695e-9a6a-4e67-9fca-ae3c027a48e6\") "
Jul 24 17:13:38.379168 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:38.379132    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5498695e-9a6a-4e67-9fca-ae3c027a48e6-oauth-serving-cert\") pod \"5498695e-9a6a-4e67-9fca-ae3c027a48e6\" (UID: \"5498695e-9a6a-4e67-9fca-ae3c027a48e6\") "
Jul 24 17:13:38.379419 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:38.379374    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5498695e-9a6a-4e67-9fca-ae3c027a48e6-service-ca" (OuterVolumeSpecName: "service-ca") pod "5498695e-9a6a-4e67-9fca-ae3c027a48e6" (UID: "5498695e-9a6a-4e67-9fca-ae3c027a48e6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:13:38.379553 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:38.379528    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5498695e-9a6a-4e67-9fca-ae3c027a48e6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5498695e-9a6a-4e67-9fca-ae3c027a48e6" (UID: "5498695e-9a6a-4e67-9fca-ae3c027a48e6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:13:38.379882 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:38.379853    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5498695e-9a6a-4e67-9fca-ae3c027a48e6-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5498695e-9a6a-4e67-9fca-ae3c027a48e6" (UID: "5498695e-9a6a-4e67-9fca-ae3c027a48e6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:13:38.380005 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:38.379984    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5498695e-9a6a-4e67-9fca-ae3c027a48e6-console-config" (OuterVolumeSpecName: "console-config") pod "5498695e-9a6a-4e67-9fca-ae3c027a48e6" (UID: "5498695e-9a6a-4e67-9fca-ae3c027a48e6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:13:38.381191 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:38.381169    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5498695e-9a6a-4e67-9fca-ae3c027a48e6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5498695e-9a6a-4e67-9fca-ae3c027a48e6" (UID: "5498695e-9a6a-4e67-9fca-ae3c027a48e6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:13:38.381298 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:38.381207    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5498695e-9a6a-4e67-9fca-ae3c027a48e6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5498695e-9a6a-4e67-9fca-ae3c027a48e6" (UID: "5498695e-9a6a-4e67-9fca-ae3c027a48e6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:13:38.381298 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:38.381272    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5498695e-9a6a-4e67-9fca-ae3c027a48e6-kube-api-access-2fqvt" (OuterVolumeSpecName: "kube-api-access-2fqvt") pod "5498695e-9a6a-4e67-9fca-ae3c027a48e6" (UID: "5498695e-9a6a-4e67-9fca-ae3c027a48e6"). InnerVolumeSpecName "kube-api-access-2fqvt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:13:38.480102 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:38.480066    2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5498695e-9a6a-4e67-9fca-ae3c027a48e6-oauth-serving-cert\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:38.480102 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:38.480100    2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5498695e-9a6a-4e67-9fca-ae3c027a48e6-console-serving-cert\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:38.480355 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:38.480115    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2fqvt\" (UniqueName: \"kubernetes.io/projected/5498695e-9a6a-4e67-9fca-ae3c027a48e6-kube-api-access-2fqvt\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:38.480355 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:38.480128    2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5498695e-9a6a-4e67-9fca-ae3c027a48e6-service-ca\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:38.480355 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:38.480141    2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5498695e-9a6a-4e67-9fca-ae3c027a48e6-console-oauth-config\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:38.480355 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:38.480153    2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5498695e-9a6a-4e67-9fca-ae3c027a48e6-console-config\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:38.480355 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:38.480164    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5498695e-9a6a-4e67-9fca-ae3c027a48e6-trusted-ca-bundle\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:13:38.534962 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:38.534933    2581 generic.go:356] "Generic (PLEG): container finished" podID="5498695e-9a6a-4e67-9fca-ae3c027a48e6" containerID="64f5452dbb5416dd80fadfecfb7b8f8b09a2fff068840f023015b894d079201c" exitCode=0
Jul 24 17:13:38.535155 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:38.534979    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c548b8c6d-drlkk"
Jul 24 17:13:38.535155 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:38.535001    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c548b8c6d-drlkk" event={"ID":"5498695e-9a6a-4e67-9fca-ae3c027a48e6","Type":"ContainerDied","Data":"64f5452dbb5416dd80fadfecfb7b8f8b09a2fff068840f023015b894d079201c"}
Jul 24 17:13:38.535155 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:38.535076    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c548b8c6d-drlkk" event={"ID":"5498695e-9a6a-4e67-9fca-ae3c027a48e6","Type":"ContainerDied","Data":"a5715000e2522f22da357fedceeff383f79026c3aeb1238c262ba54343ac1a8e"}
Jul 24 17:13:38.535155 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:38.535097    2581 scope.go:117] "RemoveContainer" containerID="64f5452dbb5416dd80fadfecfb7b8f8b09a2fff068840f023015b894d079201c"
Jul 24 17:13:38.544163 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:38.544145    2581 scope.go:117] "RemoveContainer" containerID="64f5452dbb5416dd80fadfecfb7b8f8b09a2fff068840f023015b894d079201c"
Jul 24 17:13:38.544412 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:13:38.544390    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64f5452dbb5416dd80fadfecfb7b8f8b09a2fff068840f023015b894d079201c\": container with ID starting with 64f5452dbb5416dd80fadfecfb7b8f8b09a2fff068840f023015b894d079201c not found: ID does not exist" containerID="64f5452dbb5416dd80fadfecfb7b8f8b09a2fff068840f023015b894d079201c"
Jul 24 17:13:38.544501 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:38.544417    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64f5452dbb5416dd80fadfecfb7b8f8b09a2fff068840f023015b894d079201c"} err="failed to get container status \"64f5452dbb5416dd80fadfecfb7b8f8b09a2fff068840f023015b894d079201c\": rpc error: code = NotFound desc = could not find container \"64f5452dbb5416dd80fadfecfb7b8f8b09a2fff068840f023015b894d079201c\": container with ID starting with 64f5452dbb5416dd80fadfecfb7b8f8b09a2fff068840f023015b894d079201c not found: ID does not exist"
Jul 24 17:13:38.555753 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:38.555730    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c548b8c6d-drlkk"]
Jul 24 17:13:38.558816 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:38.558794    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6c548b8c6d-drlkk"]
Jul 24 17:13:39.382759 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:39.382728    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5498695e-9a6a-4e67-9fca-ae3c027a48e6" path="/var/lib/kubelet/pods/5498695e-9a6a-4e67-9fca-ae3c027a48e6/volumes"
Jul 24 17:13:44.540612 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:44.540579    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:13:44.596813 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:13:44.596769    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:14:04.314756 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:04.314719    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:14:04.764633 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:04.764587    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:14:04.780773 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:04.780747    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:14:05.659869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:05.659840    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:14:18.699280 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:18.699244    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:14:18.802138 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:18.802106    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:14:40.118121 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:40.118085    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:14:40.827032 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:40.826995    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:14:40.929271 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:40.929235    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:14:41.461092 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:41.461060    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:14:43.716713 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:43.716680    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:14:43.816821 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:43.816790    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:14:45.217585 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:45.217548    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:14:45.319640 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:45.319605    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:14:45.389257 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:45.389222    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-4dmhg"]
Jul 24 17:14:45.389559 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:45.389546    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5498695e-9a6a-4e67-9fca-ae3c027a48e6" containerName="console"
Jul 24 17:14:45.389601 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:45.389561    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5498695e-9a6a-4e67-9fca-ae3c027a48e6" containerName="console"
Jul 24 17:14:45.389639 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:45.389619    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5498695e-9a6a-4e67-9fca-ae3c027a48e6" containerName="console"
Jul 24 17:14:45.392365 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:45.392348    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-4dmhg"
Jul 24 17:14:45.395081 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:45.395062    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 24 17:14:45.395189 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:45.395087    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 24 17:14:45.396292 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:45.396278    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-rpvbh\""
Jul 24 17:14:45.402547 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:45.402527    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-4dmhg"]
Jul 24 17:14:45.478784 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:45.478700    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3e0760d3-6a40-4ceb-bc5d-c43741de5ad0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-4dmhg\" (UID: \"3e0760d3-6a40-4ceb-bc5d-c43741de5ad0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4dmhg"
Jul 24 17:14:45.478784 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:45.478752    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxj8l\" (UniqueName: \"kubernetes.io/projected/3e0760d3-6a40-4ceb-bc5d-c43741de5ad0-kube-api-access-nxj8l\") pod \"cert-manager-webhook-6486ffd997-4dmhg\" (UID: \"3e0760d3-6a40-4ceb-bc5d-c43741de5ad0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4dmhg"
Jul 24 17:14:45.579801 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:45.579766    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3e0760d3-6a40-4ceb-bc5d-c43741de5ad0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-4dmhg\" (UID: \"3e0760d3-6a40-4ceb-bc5d-c43741de5ad0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4dmhg"
Jul 24 17:14:45.579988 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:45.579828    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxj8l\" (UniqueName: \"kubernetes.io/projected/3e0760d3-6a40-4ceb-bc5d-c43741de5ad0-kube-api-access-nxj8l\") pod \"cert-manager-webhook-6486ffd997-4dmhg\" (UID: \"3e0760d3-6a40-4ceb-bc5d-c43741de5ad0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4dmhg"
Jul 24 17:14:45.588530 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:45.588505    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3e0760d3-6a40-4ceb-bc5d-c43741de5ad0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-4dmhg\" (UID: \"3e0760d3-6a40-4ceb-bc5d-c43741de5ad0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4dmhg"
Jul 24 17:14:45.588809 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:45.588790    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxj8l\" (UniqueName: \"kubernetes.io/projected/3e0760d3-6a40-4ceb-bc5d-c43741de5ad0-kube-api-access-nxj8l\") pod \"cert-manager-webhook-6486ffd997-4dmhg\" (UID: \"3e0760d3-6a40-4ceb-bc5d-c43741de5ad0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4dmhg"
Jul 24 17:14:45.709995 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:45.709958    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-4dmhg"
Jul 24 17:14:45.751558 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:45.751470    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:14:45.862089 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:45.861064    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-4dmhg"]
Jul 24 17:14:45.863195 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:14:45.863168    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3e0760d3_6a40_4ceb_bc5d_c43741de5ad0.slice/crio-326dc5a86df2e2a2c05dec615d7c1f9d82bdbf42b2b7db0714926f779f779ffb WatchSource:0}: Error finding container 326dc5a86df2e2a2c05dec615d7c1f9d82bdbf42b2b7db0714926f779f779ffb: Status 404 returned error can't find the container with id 326dc5a86df2e2a2c05dec615d7c1f9d82bdbf42b2b7db0714926f779f779ffb
Jul 24 17:14:45.873026 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:45.873001    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:14:46.823788 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:46.823753    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-4dmhg" event={"ID":"3e0760d3-6a40-4ceb-bc5d-c43741de5ad0","Type":"ContainerStarted","Data":"326dc5a86df2e2a2c05dec615d7c1f9d82bdbf42b2b7db0714926f779f779ffb"}
Jul 24 17:14:47.232015 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:47.231977    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:14:47.625010 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:47.624929    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:14:50.713065 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:50.713008    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:14:50.767599 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:50.767564    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:14:51.848430 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:51.848392    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-4dmhg" event={"ID":"3e0760d3-6a40-4ceb-bc5d-c43741de5ad0","Type":"ContainerStarted","Data":"8b3a691e44fa1efd77f6375850fe3ca1b19f77848fa22aba413864cb9d434adf"}
Jul 24 17:14:51.848902 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:51.848446    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-4dmhg"
Jul 24 17:14:51.865620 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:51.865563    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-4dmhg" podStartSLOduration=1.46461031 podStartE2EDuration="6.865548589s" totalImagesPullingTime="5.400938279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:14:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:14:51.864104676 +0000 UTC m=+259.066153950" watchObservedRunningTime="2026-07-24 17:14:51.865548589 +0000 UTC m=+259.067597864"
Jul 24 17:14:52.600182 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:52.600146    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-88wnr"]
Jul 24 17:14:52.602712 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:52.602695    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-88wnr"
Jul 24 17:14:52.605574 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:52.605557    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-66tmj\""
Jul 24 17:14:52.614237 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:52.614213    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-88wnr"]
Jul 24 17:14:52.637628 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:52.637590    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1402637d-9764-4eb0-86ac-e83906535c90-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-88wnr\" (UID: \"1402637d-9764-4eb0-86ac-e83906535c90\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-88wnr"
Jul 24 17:14:52.637766 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:52.637739    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsb62\" (UniqueName: \"kubernetes.io/projected/1402637d-9764-4eb0-86ac-e83906535c90-kube-api-access-nsb62\") pod \"cert-manager-cainjector-559cdd8c87-88wnr\" (UID: \"1402637d-9764-4eb0-86ac-e83906535c90\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-88wnr"
Jul 24 17:14:52.738904 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:52.738863    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1402637d-9764-4eb0-86ac-e83906535c90-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-88wnr\" (UID: \"1402637d-9764-4eb0-86ac-e83906535c90\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-88wnr"
Jul 24 17:14:52.739119 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:52.738960    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsb62\" (UniqueName: \"kubernetes.io/projected/1402637d-9764-4eb0-86ac-e83906535c90-kube-api-access-nsb62\") pod \"cert-manager-cainjector-559cdd8c87-88wnr\" (UID: \"1402637d-9764-4eb0-86ac-e83906535c90\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-88wnr"
Jul 24 17:14:52.748564 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:52.748532    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1402637d-9764-4eb0-86ac-e83906535c90-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-88wnr\" (UID: \"1402637d-9764-4eb0-86ac-e83906535c90\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-88wnr"
Jul 24 17:14:52.748666 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:52.748642    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsb62\" (UniqueName: \"kubernetes.io/projected/1402637d-9764-4eb0-86ac-e83906535c90-kube-api-access-nsb62\") pod \"cert-manager-cainjector-559cdd8c87-88wnr\" (UID: \"1402637d-9764-4eb0-86ac-e83906535c90\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-88wnr"
Jul 24 17:14:52.912363 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:52.912325    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-88wnr"
Jul 24 17:14:53.040094 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:53.040068    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-88wnr"]
Jul 24 17:14:53.042378 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:14:53.042354    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1402637d_9764_4eb0_86ac_e83906535c90.slice/crio-58a81e0bc1d8d21e7495c210e0288ccb42ca6c5b791a2a7ed41cb91aec08d336 WatchSource:0}: Error finding container 58a81e0bc1d8d21e7495c210e0288ccb42ca6c5b791a2a7ed41cb91aec08d336: Status 404 returned error can't find the container with id 58a81e0bc1d8d21e7495c210e0288ccb42ca6c5b791a2a7ed41cb91aec08d336
Jul 24 17:14:53.857164 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:53.857135    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-88wnr" event={"ID":"1402637d-9764-4eb0-86ac-e83906535c90","Type":"ContainerStarted","Data":"4e4466e01b5c9e093eed5f8821d1dc08adfc39ac9cf3a3f15cfd3d7d2fca208b"}
Jul 24 17:14:53.857164 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:53.857172    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-88wnr" event={"ID":"1402637d-9764-4eb0-86ac-e83906535c90","Type":"ContainerStarted","Data":"58a81e0bc1d8d21e7495c210e0288ccb42ca6c5b791a2a7ed41cb91aec08d336"}
Jul 24 17:14:53.895127 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:53.895075    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-88wnr" podStartSLOduration=1.895056561 podStartE2EDuration="1.895056561s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:14:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:14:53.893781934 +0000 UTC m=+261.095831208" watchObservedRunningTime="2026-07-24 17:14:53.895056561 +0000 UTC m=+261.097105829"
Jul 24 17:14:53.948866 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:53.948838    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:14:54.049733 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:54.049699    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:14:55.226700 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:55.226667    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:14:57.123655 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:57.123627    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:14:57.221206 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:57.221172    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:14:57.853938 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:14:57.853905    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-4dmhg"
Jul 24 17:15:02.176555 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:02.176517    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:15:02.278284 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:02.278252    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:15:05.192651 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:05.192617    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:15:18.401710 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:18.401672    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:15:19.401456 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:19.401421    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5bcd74455c-r4pqr"]
Jul 24 17:15:19.405230 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:19.405214    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-r4pqr"
Jul 24 17:15:19.408624 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:19.408600    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 24 17:15:19.408740 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:19.408606    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 24 17:15:19.408882 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:19.408865    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4tjsr\""
Jul 24 17:15:19.409012 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:19.408997    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 24 17:15:19.409567 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:19.409549    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 24 17:15:19.438985 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:19.438947    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5bcd74455c-r4pqr"]
Jul 24 17:15:19.474101 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:19.474066    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ad445777-f665-482c-9f35-2eb21a836bb2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5bcd74455c-r4pqr\" (UID: \"ad445777-f665-482c-9f35-2eb21a836bb2\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-r4pqr"
Jul 24 17:15:19.474259 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:19.474111    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ad445777-f665-482c-9f35-2eb21a836bb2-manifests\") pod \"opendatahub-operator-controller-manager-5bcd74455c-r4pqr\" (UID: \"ad445777-f665-482c-9f35-2eb21a836bb2\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-r4pqr"
Jul 24 17:15:19.474259 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:19.474228    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfsj2\" (UniqueName: \"kubernetes.io/projected/ad445777-f665-482c-9f35-2eb21a836bb2-kube-api-access-qfsj2\") pod \"opendatahub-operator-controller-manager-5bcd74455c-r4pqr\" (UID: \"ad445777-f665-482c-9f35-2eb21a836bb2\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-r4pqr"
Jul 24 17:15:19.474338 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:19.474280    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad445777-f665-482c-9f35-2eb21a836bb2-webhook-cert\") pod \"opendatahub-operator-controller-manager-5bcd74455c-r4pqr\" (UID: \"ad445777-f665-482c-9f35-2eb21a836bb2\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-r4pqr"
Jul 24 17:15:19.474373 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:19.474344    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad445777-f665-482c-9f35-2eb21a836bb2-tmp\") pod \"opendatahub-operator-controller-manager-5bcd74455c-r4pqr\" (UID: \"ad445777-f665-482c-9f35-2eb21a836bb2\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-r4pqr"
Jul 24 17:15:19.575135 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:19.575098    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfsj2\" (UniqueName: \"kubernetes.io/projected/ad445777-f665-482c-9f35-2eb21a836bb2-kube-api-access-qfsj2\") pod \"opendatahub-operator-controller-manager-5bcd74455c-r4pqr\" (UID: \"ad445777-f665-482c-9f35-2eb21a836bb2\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-r4pqr"
Jul 24 17:15:19.575319 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:19.575148    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad445777-f665-482c-9f35-2eb21a836bb2-webhook-cert\") pod \"opendatahub-operator-controller-manager-5bcd74455c-r4pqr\" (UID: \"ad445777-f665-482c-9f35-2eb21a836bb2\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-r4pqr"
Jul 24 17:15:19.575319 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:19.575198    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad445777-f665-482c-9f35-2eb21a836bb2-tmp\") pod \"opendatahub-operator-controller-manager-5bcd74455c-r4pqr\" (UID: \"ad445777-f665-482c-9f35-2eb21a836bb2\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-r4pqr"
Jul 24 17:15:19.575319 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:19.575222    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ad445777-f665-482c-9f35-2eb21a836bb2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5bcd74455c-r4pqr\" (UID: \"ad445777-f665-482c-9f35-2eb21a836bb2\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-r4pqr"
Jul 24 17:15:19.575319 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:19.575259    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ad445777-f665-482c-9f35-2eb21a836bb2-manifests\") pod \"opendatahub-operator-controller-manager-5bcd74455c-r4pqr\" (UID: \"ad445777-f665-482c-9f35-2eb21a836bb2\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-r4pqr"
Jul 24 17:15:19.575696 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:19.575672    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ad445777-f665-482c-9f35-2eb21a836bb2-manifests\") pod \"opendatahub-operator-controller-manager-5bcd74455c-r4pqr\" (UID: \"ad445777-f665-482c-9f35-2eb21a836bb2\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-r4pqr"
Jul 24 17:15:19.577598 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:19.577571    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad445777-f665-482c-9f35-2eb21a836bb2-tmp\") pod \"opendatahub-operator-controller-manager-5bcd74455c-r4pqr\" (UID: \"ad445777-f665-482c-9f35-2eb21a836bb2\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-r4pqr"
Jul 24 17:15:19.577790 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:19.577769    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad445777-f665-482c-9f35-2eb21a836bb2-webhook-cert\") pod \"opendatahub-operator-controller-manager-5bcd74455c-r4pqr\" (UID: \"ad445777-f665-482c-9f35-2eb21a836bb2\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-r4pqr"
Jul 24 17:15:19.577879 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:19.577859    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ad445777-f665-482c-9f35-2eb21a836bb2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5bcd74455c-r4pqr\" (UID: \"ad445777-f665-482c-9f35-2eb21a836bb2\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-r4pqr"
Jul 24 17:15:19.587527 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:19.587503    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfsj2\" (UniqueName: \"kubernetes.io/projected/ad445777-f665-482c-9f35-2eb21a836bb2-kube-api-access-qfsj2\") pod \"opendatahub-operator-controller-manager-5bcd74455c-r4pqr\" (UID: \"ad445777-f665-482c-9f35-2eb21a836bb2\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-r4pqr"
Jul 24 17:15:19.716317 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:19.716237    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-r4pqr"
Jul 24 17:15:19.857778 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:19.857751    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5bcd74455c-r4pqr"]
Jul 24 17:15:19.860008 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:15:19.859975    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad445777_f665_482c_9f35_2eb21a836bb2.slice/crio-74eb077b0df4257b4d153d34ed5a1cc86e612b607dabd0fed5e5e8ba559e9f8b WatchSource:0}: Error finding container 74eb077b0df4257b4d153d34ed5a1cc86e612b607dabd0fed5e5e8ba559e9f8b: Status 404 returned error can't find the container with id 74eb077b0df4257b4d153d34ed5a1cc86e612b607dabd0fed5e5e8ba559e9f8b
Jul 24 17:15:19.876877 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:19.876837    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:15:19.964162 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:19.964127    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-r4pqr" event={"ID":"ad445777-f665-482c-9f35-2eb21a836bb2","Type":"ContainerStarted","Data":"74eb077b0df4257b4d153d34ed5a1cc86e612b607dabd0fed5e5e8ba559e9f8b"}
Jul 24 17:15:19.978509 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:19.978422    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:15:20.083381 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:20.083317    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:15:23.987722 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:23.987685    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-r4pqr" event={"ID":"ad445777-f665-482c-9f35-2eb21a836bb2","Type":"ContainerStarted","Data":"b0bbc0c91f27b584cbd8c7d2d938ea165dba69c16aebea3a4e40848e6c6716b2"}
Jul 24 17:15:24.993684 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:24.993649    2581 generic.go:356] "Generic (PLEG): container finished" podID="ad445777-f665-482c-9f35-2eb21a836bb2" containerID="b0bbc0c91f27b584cbd8c7d2d938ea165dba69c16aebea3a4e40848e6c6716b2" exitCode=0
Jul 24 17:15:24.994195 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:24.993697    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-r4pqr" event={"ID":"ad445777-f665-482c-9f35-2eb21a836bb2","Type":"ContainerDied","Data":"b0bbc0c91f27b584cbd8c7d2d938ea165dba69c16aebea3a4e40848e6c6716b2"}
Jul 24 17:15:25.999755 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:25.999722    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-r4pqr" event={"ID":"ad445777-f665-482c-9f35-2eb21a836bb2","Type":"ContainerStarted","Data":"e6fdc81e14299ef337a94534d261cc0e5e16d59ed9a7f6578b68c8012613f33b"}
Jul 24 17:15:26.000130 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:25.999973    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-r4pqr"
Jul 24 17:15:26.023281 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:26.023236    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-r4pqr" podStartSLOduration=2.329677644 podStartE2EDuration="7.023215348s" totalImagesPullingTime="3.644177842s" totalInitContainerRuntime="1.049359862s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:15:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:15:26.020188869 +0000 UTC m=+293.222238142" watchObservedRunningTime="2026-07-24 17:15:26.023215348 +0000 UTC m=+293.225264622"
Jul 24 17:15:33.260985 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:33.260959    2581 kubelet.go:1648] "Image garbage collection succeeded"
Jul 24 17:15:35.606796 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:35.606758    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:15:35.707058 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:35.706998    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:15:37.005603 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:37.005579    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-r4pqr"
Jul 24 17:15:37.638475 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:37.638439    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-57c7c45669-pf7jm"]
Jul 24 17:15:37.641620 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:37.641600    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-57c7c45669-pf7jm"
Jul 24 17:15:37.644697 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:37.644673    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 24 17:15:37.644827 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:37.644673    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 24 17:15:37.644827 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:37.644673    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 24 17:15:37.644978 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:37.644956    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 24 17:15:37.646180 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:37.646166    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 24 17:15:37.646255 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:37.646188    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-6snd8\""
Jul 24 17:15:37.653579 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:37.653559    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-57c7c45669-pf7jm"]
Jul 24 17:15:37.735119 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:37.735085    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b295a22b-ef6d-4c71-9925-ca8919ab0be6-manager-config\") pod \"lws-controller-manager-57c7c45669-pf7jm\" (UID: \"b295a22b-ef6d-4c71-9925-ca8919ab0be6\") " pod="openshift-lws-operator/lws-controller-manager-57c7c45669-pf7jm"
Jul 24 17:15:37.735307 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:37.735143    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b295a22b-ef6d-4c71-9925-ca8919ab0be6-cert\") pod \"lws-controller-manager-57c7c45669-pf7jm\" (UID: \"b295a22b-ef6d-4c71-9925-ca8919ab0be6\") " pod="openshift-lws-operator/lws-controller-manager-57c7c45669-pf7jm"
Jul 24 17:15:37.735307 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:37.735189    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b295a22b-ef6d-4c71-9925-ca8919ab0be6-metrics-cert\") pod \"lws-controller-manager-57c7c45669-pf7jm\" (UID: \"b295a22b-ef6d-4c71-9925-ca8919ab0be6\") " pod="openshift-lws-operator/lws-controller-manager-57c7c45669-pf7jm"
Jul 24 17:15:37.735307 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:37.735239    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55pql\" (UniqueName: \"kubernetes.io/projected/b295a22b-ef6d-4c71-9925-ca8919ab0be6-kube-api-access-55pql\") pod \"lws-controller-manager-57c7c45669-pf7jm\" (UID: \"b295a22b-ef6d-4c71-9925-ca8919ab0be6\") " pod="openshift-lws-operator/lws-controller-manager-57c7c45669-pf7jm"
Jul 24 17:15:37.836630 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:37.836590    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55pql\" (UniqueName: \"kubernetes.io/projected/b295a22b-ef6d-4c71-9925-ca8919ab0be6-kube-api-access-55pql\") pod \"lws-controller-manager-57c7c45669-pf7jm\" (UID: \"b295a22b-ef6d-4c71-9925-ca8919ab0be6\") " pod="openshift-lws-operator/lws-controller-manager-57c7c45669-pf7jm"
Jul 24 17:15:37.836788 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:37.836669    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b295a22b-ef6d-4c71-9925-ca8919ab0be6-manager-config\") pod \"lws-controller-manager-57c7c45669-pf7jm\" (UID: \"b295a22b-ef6d-4c71-9925-ca8919ab0be6\") " pod="openshift-lws-operator/lws-controller-manager-57c7c45669-pf7jm"
Jul 24 17:15:37.836788 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:37.836712    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b295a22b-ef6d-4c71-9925-ca8919ab0be6-cert\") pod \"lws-controller-manager-57c7c45669-pf7jm\" (UID: \"b295a22b-ef6d-4c71-9925-ca8919ab0be6\") " pod="openshift-lws-operator/lws-controller-manager-57c7c45669-pf7jm"
Jul 24 17:15:37.836899 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:37.836858    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b295a22b-ef6d-4c71-9925-ca8919ab0be6-metrics-cert\") pod \"lws-controller-manager-57c7c45669-pf7jm\" (UID: \"b295a22b-ef6d-4c71-9925-ca8919ab0be6\") " pod="openshift-lws-operator/lws-controller-manager-57c7c45669-pf7jm"
Jul 24 17:15:37.837310 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:37.837285    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b295a22b-ef6d-4c71-9925-ca8919ab0be6-manager-config\") pod \"lws-controller-manager-57c7c45669-pf7jm\" (UID: \"b295a22b-ef6d-4c71-9925-ca8919ab0be6\") " pod="openshift-lws-operator/lws-controller-manager-57c7c45669-pf7jm"
Jul 24 17:15:37.839362 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:37.839338    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b295a22b-ef6d-4c71-9925-ca8919ab0be6-cert\") pod \"lws-controller-manager-57c7c45669-pf7jm\" (UID: \"b295a22b-ef6d-4c71-9925-ca8919ab0be6\") " pod="openshift-lws-operator/lws-controller-manager-57c7c45669-pf7jm"
Jul 24 17:15:37.839469 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:37.839381    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b295a22b-ef6d-4c71-9925-ca8919ab0be6-metrics-cert\") pod \"lws-controller-manager-57c7c45669-pf7jm\" (UID: \"b295a22b-ef6d-4c71-9925-ca8919ab0be6\") " pod="openshift-lws-operator/lws-controller-manager-57c7c45669-pf7jm"
Jul 24 17:15:37.845504 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:37.845483    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55pql\" (UniqueName: \"kubernetes.io/projected/b295a22b-ef6d-4c71-9925-ca8919ab0be6-kube-api-access-55pql\") pod \"lws-controller-manager-57c7c45669-pf7jm\" (UID: \"b295a22b-ef6d-4c71-9925-ca8919ab0be6\") " pod="openshift-lws-operator/lws-controller-manager-57c7c45669-pf7jm"
Jul 24 17:15:37.951597 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:37.951489    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-57c7c45669-pf7jm"
Jul 24 17:15:38.081346 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:38.081321    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-57c7c45669-pf7jm"]
Jul 24 17:15:38.083854 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:15:38.083829    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb295a22b_ef6d_4c71_9925_ca8919ab0be6.slice/crio-e1ac7142d8fd0c4b56a946030eb5611f7ada23ce2bf70a8e25270fda5782c48c WatchSource:0}: Error finding container e1ac7142d8fd0c4b56a946030eb5611f7ada23ce2bf70a8e25270fda5782c48c: Status 404 returned error can't find the container with id e1ac7142d8fd0c4b56a946030eb5611f7ada23ce2bf70a8e25270fda5782c48c
Jul 24 17:15:38.086056 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:38.086016    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 17:15:38.783675 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:38.783639    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:15:38.909689 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:38.909622    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:15:39.063289 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:39.063200    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-57c7c45669-pf7jm" event={"ID":"b295a22b-ef6d-4c71-9925-ca8919ab0be6","Type":"ContainerStarted","Data":"e1ac7142d8fd0c4b56a946030eb5611f7ada23ce2bf70a8e25270fda5782c48c"}
Jul 24 17:15:39.275478 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:39.275443    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:15:39.410555 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:39.410511    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-84b496c558-987vc"]
Jul 24 17:15:39.414363 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:39.414342    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-84b496c558-987vc"
Jul 24 17:15:39.417699 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:39.417664    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 24 17:15:39.417699 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:39.417677    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 24 17:15:39.417899 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:39.417708    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 24 17:15:39.417899 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:39.417707    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 24 17:15:39.418529 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:39.418505    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-22trv\""
Jul 24 17:15:39.432923 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:39.432899    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-84b496c558-987vc"]
Jul 24 17:15:39.451742 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:39.451705    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:15:39.552650 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:39.552609    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe8593b3-ba31-4b15-9485-281e108a027f-tls-certs\") pod \"kube-auth-proxy-84b496c558-987vc\" (UID: \"fe8593b3-ba31-4b15-9485-281e108a027f\") " pod="openshift-ingress/kube-auth-proxy-84b496c558-987vc"
Jul 24 17:15:39.552840 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:39.552684    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqqxk\" (UniqueName: \"kubernetes.io/projected/fe8593b3-ba31-4b15-9485-281e108a027f-kube-api-access-lqqxk\") pod \"kube-auth-proxy-84b496c558-987vc\" (UID: \"fe8593b3-ba31-4b15-9485-281e108a027f\") " pod="openshift-ingress/kube-auth-proxy-84b496c558-987vc"
Jul 24 17:15:39.552840 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:39.552721    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fe8593b3-ba31-4b15-9485-281e108a027f-tmp\") pod \"kube-auth-proxy-84b496c558-987vc\" (UID: \"fe8593b3-ba31-4b15-9485-281e108a027f\") " pod="openshift-ingress/kube-auth-proxy-84b496c558-987vc"
Jul 24 17:15:39.640465 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:39.640432    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:15:39.653680 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:39.653645    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe8593b3-ba31-4b15-9485-281e108a027f-tls-certs\") pod \"kube-auth-proxy-84b496c558-987vc\" (UID: \"fe8593b3-ba31-4b15-9485-281e108a027f\") " pod="openshift-ingress/kube-auth-proxy-84b496c558-987vc"
Jul 24 17:15:39.653868 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:39.653698    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqqxk\" (UniqueName: \"kubernetes.io/projected/fe8593b3-ba31-4b15-9485-281e108a027f-kube-api-access-lqqxk\") pod \"kube-auth-proxy-84b496c558-987vc\" (UID: \"fe8593b3-ba31-4b15-9485-281e108a027f\") " pod="openshift-ingress/kube-auth-proxy-84b496c558-987vc"
Jul 24 17:15:39.653868 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:39.653740    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fe8593b3-ba31-4b15-9485-281e108a027f-tmp\") pod \"kube-auth-proxy-84b496c558-987vc\" (UID: \"fe8593b3-ba31-4b15-9485-281e108a027f\") " pod="openshift-ingress/kube-auth-proxy-84b496c558-987vc"
Jul 24 17:15:39.656486 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:39.656458    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fe8593b3-ba31-4b15-9485-281e108a027f-tmp\") pod \"kube-auth-proxy-84b496c558-987vc\" (UID: \"fe8593b3-ba31-4b15-9485-281e108a027f\") " pod="openshift-ingress/kube-auth-proxy-84b496c558-987vc"
Jul 24 17:15:39.656657 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:39.656622    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe8593b3-ba31-4b15-9485-281e108a027f-tls-certs\") pod \"kube-auth-proxy-84b496c558-987vc\" (UID: \"fe8593b3-ba31-4b15-9485-281e108a027f\") " pod="openshift-ingress/kube-auth-proxy-84b496c558-987vc"
Jul 24 17:15:39.670029 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:39.669963    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqqxk\" (UniqueName: \"kubernetes.io/projected/fe8593b3-ba31-4b15-9485-281e108a027f-kube-api-access-lqqxk\") pod \"kube-auth-proxy-84b496c558-987vc\" (UID: \"fe8593b3-ba31-4b15-9485-281e108a027f\") " pod="openshift-ingress/kube-auth-proxy-84b496c558-987vc"
Jul 24 17:15:39.725242 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:39.725206    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-84b496c558-987vc"
Jul 24 17:15:39.732311 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:39.732288    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:15:39.832259 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:39.832210    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:15:40.309584 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:40.309545    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:15:40.501177 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:40.501146    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:15:40.534231 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:40.534209    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-84b496c558-987vc"]
Jul 24 17:15:40.535994 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:15:40.535965    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe8593b3_ba31_4b15_9485_281e108a027f.slice/crio-87d1ffc7367a3a1eb071e510a7afd2139b2aa8d2aaad7564536b16b32eb30953 WatchSource:0}: Error finding container 87d1ffc7367a3a1eb071e510a7afd2139b2aa8d2aaad7564536b16b32eb30953: Status 404 returned error can't find the container with id 87d1ffc7367a3a1eb071e510a7afd2139b2aa8d2aaad7564536b16b32eb30953
Jul 24 17:15:40.670240 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:40.670215    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:15:40.763047 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:40.763003    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:15:40.862990 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:40.862963    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:15:41.074843 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:41.074747    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-84b496c558-987vc" event={"ID":"fe8593b3-ba31-4b15-9485-281e108a027f","Type":"ContainerStarted","Data":"87d1ffc7367a3a1eb071e510a7afd2139b2aa8d2aaad7564536b16b32eb30953"}
Jul 24 17:15:41.076548 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:41.076518    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-57c7c45669-pf7jm" event={"ID":"b295a22b-ef6d-4c71-9925-ca8919ab0be6","Type":"ContainerStarted","Data":"08df6751bf1967f0a6842a193cf5ac31aa69fd6f2d2e105e5f80eeadf54d140b"}
Jul 24 17:15:41.076701 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:41.076634    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-57c7c45669-pf7jm"
Jul 24 17:15:41.104475 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:41.098721    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-57c7c45669-pf7jm" podStartSLOduration=1.7096083530000001 podStartE2EDuration="4.098699602s" totalImagesPullingTime="2.389091249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:15:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:15:41.095031949 +0000 UTC m=+308.297081224" watchObservedRunningTime="2026-07-24 17:15:41.098699602 +0000 UTC m=+308.300748877"
Jul 24 17:15:41.767828 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:41.767791    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:15:43.089529 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:43.089499    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-84b496c558-987vc" event={"ID":"fe8593b3-ba31-4b15-9485-281e108a027f","Type":"ContainerStarted","Data":"157c12165cb5789a5fb4acef3efa9971a9a8cd55ccc7c2d404566569f75e2d35"}
Jul 24 17:15:43.107942 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:43.107886    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-84b496c558-987vc" podStartSLOduration=2.065251629 podStartE2EDuration="4.107866533s" totalImagesPullingTime="2.042614904s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:15:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:15:43.107799933 +0000 UTC m=+310.309849207" watchObservedRunningTime="2026-07-24 17:15:43.107866533 +0000 UTC m=+310.309915804"
Jul 24 17:15:46.225058 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:46.225004    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:15:46.326243 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:46.326208    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:15:46.479715 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:46.479631    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:15:46.579572 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:46.579537    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:15:46.827188 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:46.827096    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:15:50.933546 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:50.933439    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:15:51.024668 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:51.024610    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:15:51.151372 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:51.151344    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:15:51.295278 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:51.295180    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:15:51.389334 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:51.389293    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:15:51.860556 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:51.860526    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:15:52.085713 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:15:52.085686    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-57c7c45669-pf7jm"
Jul 24 17:16:36.748645 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:16:36.748609    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:16:38.696590 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:16:38.696555    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:16:38.755699 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:16:38.755660    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:16:38.812155 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:16:38.812114    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:16:40.302887 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:16:40.302852    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:16:40.404875 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:16:40.404838    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:16:40.501692 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:16:40.501656    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:16:41.211243 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:16:41.211217    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:16:41.410978 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:16:41.410924    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:16:53.329540 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:16:53.329506    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:16:53.394835 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:16:53.394777    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:09.644567 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:09.644506    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:11.346322 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:11.346263    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:12.077154 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:12.077119    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:12.136835 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:12.136796    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:12.228102 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:12.228068    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:12.831937 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:12.831907    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:12.932878 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:12.932849    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:13.834911 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:13.834882    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:16.264057 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:16.263998    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:16.482851 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:16.482822    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:17.671156 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:17.671101    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:20.825100 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:20.825067    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:21.230743 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:21.230703    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:21.553917 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:21.553840    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:21.635518 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:21.635483    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:23.444651 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:23.444616    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:23.542544 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:23.542510    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:24.298545 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:24.298512    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:25.637216 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:25.637175    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:25.737478 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:25.737450    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:28.523089 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:28.523030    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-cswnr"]
Jul 24 17:17:28.526454 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:28.526436    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cswnr"
Jul 24 17:17:28.529060 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:28.529003    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 24 17:17:28.529180 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:28.529119    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 24 17:17:28.529180 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:28.529169    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 24 17:17:28.530248 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:28.530231    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-gwqqm\""
Jul 24 17:17:28.530355 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:28.530232    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 24 17:17:28.533218 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:28.533194    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-cswnr"]
Jul 24 17:17:28.535841 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:28.535816    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:28.717609 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:28.717566    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ch5dk\" (UniqueName: \"kubernetes.io/projected/68a9959d-3d81-4dbc-9bd9-b447935635bf-kube-api-access-ch5dk\") pod \"kuadrant-console-plugin-66b5774985-cswnr\" (UID: \"68a9959d-3d81-4dbc-9bd9-b447935635bf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cswnr"
Jul 24 17:17:28.717796 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:28.717699    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/68a9959d-3d81-4dbc-9bd9-b447935635bf-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-cswnr\" (UID: \"68a9959d-3d81-4dbc-9bd9-b447935635bf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cswnr"
Jul 24 17:17:28.717796 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:28.717725    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/68a9959d-3d81-4dbc-9bd9-b447935635bf-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-cswnr\" (UID: \"68a9959d-3d81-4dbc-9bd9-b447935635bf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cswnr"
Jul 24 17:17:28.819169 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:28.819069    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/68a9959d-3d81-4dbc-9bd9-b447935635bf-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-cswnr\" (UID: \"68a9959d-3d81-4dbc-9bd9-b447935635bf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cswnr"
Jul 24 17:17:28.819169 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:28.819119    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/68a9959d-3d81-4dbc-9bd9-b447935635bf-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-cswnr\" (UID: \"68a9959d-3d81-4dbc-9bd9-b447935635bf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cswnr"
Jul 24 17:17:28.819403 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:17:28.819181    2581 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 24 17:17:28.819403 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:28.819212    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ch5dk\" (UniqueName: \"kubernetes.io/projected/68a9959d-3d81-4dbc-9bd9-b447935635bf-kube-api-access-ch5dk\") pod \"kuadrant-console-plugin-66b5774985-cswnr\" (UID: \"68a9959d-3d81-4dbc-9bd9-b447935635bf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cswnr"
Jul 24 17:17:28.819403 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:17:28.819244    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68a9959d-3d81-4dbc-9bd9-b447935635bf-plugin-serving-cert podName:68a9959d-3d81-4dbc-9bd9-b447935635bf nodeName:}" failed. No retries permitted until 2026-07-24 17:17:29.319226446 +0000 UTC m=+416.521275698 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/68a9959d-3d81-4dbc-9bd9-b447935635bf-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-cswnr" (UID: "68a9959d-3d81-4dbc-9bd9-b447935635bf") : secret "plugin-serving-cert" not found
Jul 24 17:17:28.819713 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:28.819694    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/68a9959d-3d81-4dbc-9bd9-b447935635bf-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-cswnr\" (UID: \"68a9959d-3d81-4dbc-9bd9-b447935635bf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cswnr"
Jul 24 17:17:28.829874 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:28.829853    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ch5dk\" (UniqueName: \"kubernetes.io/projected/68a9959d-3d81-4dbc-9bd9-b447935635bf-kube-api-access-ch5dk\") pod \"kuadrant-console-plugin-66b5774985-cswnr\" (UID: \"68a9959d-3d81-4dbc-9bd9-b447935635bf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cswnr"
Jul 24 17:17:29.002057 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:29.001999    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:29.324013 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:29.323966    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/68a9959d-3d81-4dbc-9bd9-b447935635bf-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-cswnr\" (UID: \"68a9959d-3d81-4dbc-9bd9-b447935635bf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cswnr"
Jul 24 17:17:29.326663 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:29.326636    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/68a9959d-3d81-4dbc-9bd9-b447935635bf-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-cswnr\" (UID: \"68a9959d-3d81-4dbc-9bd9-b447935635bf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cswnr"
Jul 24 17:17:29.436541 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:29.436505    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cswnr"
Jul 24 17:17:29.444316 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:29.444283    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:29.566066 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:29.566012    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:29.581454 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:29.581434    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-cswnr"]
Jul 24 17:17:29.583850 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:17:29.583825    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod68a9959d_3d81_4dbc_9bd9_b447935635bf.slice/crio-99f268a15a43e1bd7cacd86b16c08f4a23eb71e1bcc2504e8379597be72941da WatchSource:0}: Error finding container 99f268a15a43e1bd7cacd86b16c08f4a23eb71e1bcc2504e8379597be72941da: Status 404 returned error can't find the container with id 99f268a15a43e1bd7cacd86b16c08f4a23eb71e1bcc2504e8379597be72941da
Jul 24 17:17:30.567142 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:30.567109    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cswnr" event={"ID":"68a9959d-3d81-4dbc-9bd9-b447935635bf","Type":"ContainerStarted","Data":"99f268a15a43e1bd7cacd86b16c08f4a23eb71e1bcc2504e8379597be72941da"}
Jul 24 17:17:32.846198 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:32.846128    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:32.964632 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:32.964582    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:41.279405 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:41.279374    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:41.385418 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:41.385388    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:43.793179 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:43.793147    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:43.893421 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:43.893386    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:46.611557 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:46.611516    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:46.711949 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:46.711917    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:50.160631 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:50.160593    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:50.261128 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:50.261096    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:53.694296 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:53.694217    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cswnr" event={"ID":"68a9959d-3d81-4dbc-9bd9-b447935635bf","Type":"ContainerStarted","Data":"c1634c29b8557f22dbd76d7cbb445d6e0f0500198613f1d7ec41efe8e92ebbfb"}
Jul 24 17:17:53.710258 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:53.710213    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cswnr" podStartSLOduration=2.706605137 podStartE2EDuration="25.71019876s" totalImagesPullingTime="23.003593623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:17:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:17:53.708999147 +0000 UTC m=+440.911048440" watchObservedRunningTime="2026-07-24 17:17:53.71019876 +0000 UTC m=+440.912248034"
Jul 24 17:17:58.980912 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:58.980880    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:59.079684 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:17:59.079653    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:18:01.599381 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:01.599350    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:18:01.699981 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:01.699945    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:18:04.787906 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:04.787876    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:18:04.888185 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:04.888120    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:18:07.835861 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:07.835827    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:18:07.936541 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:07.936505    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:18:08.385300 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:08.385271    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:18:08.485726 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:08.485694    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:18:17.387921 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:17.387879    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:18:17.487859 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:17.487829    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:18:23.248246 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:23.248206    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:18:23.355991 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:23.355953    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:18:23.414471 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:23.414427    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:18:23.636071 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:23.636021    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:18:23.641849 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:23.641824    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"
Jul 24 17:18:23.644874 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:23.644847    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 24 17:18:23.651494 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:23.651469    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:18:23.706480 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:23.706430    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:18:23.733734 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:23.733690    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47fqq\" (UniqueName: \"kubernetes.io/projected/10336757-378d-4415-ad40-56386dd911bb-kube-api-access-47fqq\") pod \"limitador-limitador-54d6d988d5-5dbvm\" (UID: \"10336757-378d-4415-ad40-56386dd911bb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"
Jul 24 17:18:23.733901 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:23.733747    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/10336757-378d-4415-ad40-56386dd911bb-config-file\") pod \"limitador-limitador-54d6d988d5-5dbvm\" (UID: \"10336757-378d-4415-ad40-56386dd911bb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"
Jul 24 17:18:23.834370 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:23.834330    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47fqq\" (UniqueName: \"kubernetes.io/projected/10336757-378d-4415-ad40-56386dd911bb-kube-api-access-47fqq\") pod \"limitador-limitador-54d6d988d5-5dbvm\" (UID: \"10336757-378d-4415-ad40-56386dd911bb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"
Jul 24 17:18:23.834579 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:23.834397    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/10336757-378d-4415-ad40-56386dd911bb-config-file\") pod \"limitador-limitador-54d6d988d5-5dbvm\" (UID: \"10336757-378d-4415-ad40-56386dd911bb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"
Jul 24 17:18:23.835096 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:23.835076    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/10336757-378d-4415-ad40-56386dd911bb-config-file\") pod \"limitador-limitador-54d6d988d5-5dbvm\" (UID: \"10336757-378d-4415-ad40-56386dd911bb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"
Jul 24 17:18:23.842625 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:23.842598    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47fqq\" (UniqueName: \"kubernetes.io/projected/10336757-378d-4415-ad40-56386dd911bb-kube-api-access-47fqq\") pod \"limitador-limitador-54d6d988d5-5dbvm\" (UID: \"10336757-378d-4415-ad40-56386dd911bb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"
Jul 24 17:18:23.953246 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:23.953153    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"
Jul 24 17:18:24.087442 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:24.087405    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:18:24.089879 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:18:24.089847    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10336757_378d_4415_ad40_56386dd911bb.slice/crio-e9354c927af127ca217e971848fd076a7da495c87365c6bf1b9ae4cdc3285535 WatchSource:0}: Error finding container e9354c927af127ca217e971848fd076a7da495c87365c6bf1b9ae4cdc3285535: Status 404 returned error can't find the container with id e9354c927af127ca217e971848fd076a7da495c87365c6bf1b9ae4cdc3285535
Jul 24 17:18:24.091237 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:24.091208    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:18:24.146984 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:24.146930    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:18:24.233539 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:24.233454    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-n6nj7"]
Jul 24 17:18:24.236271 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:24.236253    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-n6nj7"
Jul 24 17:18:24.238822 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:24.238801    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-jjkvj\""
Jul 24 17:18:24.242905 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:24.242883    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-n6nj7"]
Jul 24 17:18:24.339885 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:24.339844    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6zdm\" (UniqueName: \"kubernetes.io/projected/18b37214-804a-4c4d-ac17-7b3c4a7eff93-kube-api-access-v6zdm\") pod \"authorino-75d556bd77-n6nj7\" (UID: \"18b37214-804a-4c4d-ac17-7b3c4a7eff93\") " pod="kuadrant-system/authorino-75d556bd77-n6nj7"
Jul 24 17:18:24.441159 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:24.441124    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6zdm\" (UniqueName: \"kubernetes.io/projected/18b37214-804a-4c4d-ac17-7b3c4a7eff93-kube-api-access-v6zdm\") pod \"authorino-75d556bd77-n6nj7\" (UID: \"18b37214-804a-4c4d-ac17-7b3c4a7eff93\") " pod="kuadrant-system/authorino-75d556bd77-n6nj7"
Jul 24 17:18:24.449358 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:24.449332    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6zdm\" (UniqueName: \"kubernetes.io/projected/18b37214-804a-4c4d-ac17-7b3c4a7eff93-kube-api-access-v6zdm\") pod \"authorino-75d556bd77-n6nj7\" (UID: \"18b37214-804a-4c4d-ac17-7b3c4a7eff93\") " pod="kuadrant-system/authorino-75d556bd77-n6nj7"
Jul 24 17:18:24.546651 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:24.546463    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-n6nj7"
Jul 24 17:18:24.732739 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:24.732577    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-n6nj7"]
Jul 24 17:18:24.846754 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:24.846665    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-n6nj7" event={"ID":"18b37214-804a-4c4d-ac17-7b3c4a7eff93","Type":"ContainerStarted","Data":"c8aebb6b3cf2dbc770c01fb3f7565d975dc2d91030774d94203bfe8521747452"}
Jul 24 17:18:24.848836 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:24.848805    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-5dbvm" event={"ID":"10336757-378d-4415-ad40-56386dd911bb","Type":"ContainerStarted","Data":"e9354c927af127ca217e971848fd076a7da495c87365c6bf1b9ae4cdc3285535"}
Jul 24 17:18:28.534921 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:28.534896    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:18:28.635495 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:28.635457    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:18:28.872820 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:28.872728    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-n6nj7" event={"ID":"18b37214-804a-4c4d-ac17-7b3c4a7eff93","Type":"ContainerStarted","Data":"7f37d05ecb7b5ced4f1969331a0552c4b8f4b5ca180cb8f2360fe278876cf3e9"}
Jul 24 17:18:28.874261 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:28.874238    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-5dbvm" event={"ID":"10336757-378d-4415-ad40-56386dd911bb","Type":"ContainerStarted","Data":"66d40a96138e782860938635f168522d3c68b176251a0decc7b958f865978f15"}
Jul 24 17:18:28.874380 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:28.874309    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"
Jul 24 17:18:28.887813 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:28.887770    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-n6nj7" podStartSLOduration=1.148109558 podStartE2EDuration="4.887732353s" totalImagesPullingTime="3.739622795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:18:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:18:28.886871933 +0000 UTC m=+476.088921189" watchObservedRunningTime="2026-07-24 17:18:28.887732353 +0000 UTC m=+476.089781627"
Jul 24 17:18:28.904068 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:28.904005    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-5dbvm" podStartSLOduration=1.480823893 podStartE2EDuration="5.903989543s" totalImagesPullingTime="4.42316565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:18:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:18:28.90233553 +0000 UTC m=+476.104384806" watchObservedRunningTime="2026-07-24 17:18:28.903989543 +0000 UTC m=+476.106038836"
Jul 24 17:18:29.564312 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:29.564284    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:18:39.880614 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:39.880582    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"
Jul 24 17:18:45.980033 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:45.979980    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:18:46.074107 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:46.074068    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:18:46.520471 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:46.520439    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:18:58.130323 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:58.130293    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:18:58.331723 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:58.331690    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:18:59.872073 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:59.872019    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-n6nj7"]
Jul 24 17:18:59.872491 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:18:59.872260    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-n6nj7" podUID="18b37214-804a-4c4d-ac17-7b3c4a7eff93" containerName="authorino" containerID="cri-o://7f37d05ecb7b5ced4f1969331a0552c4b8f4b5ca180cb8f2360fe278876cf3e9" gracePeriod=30
Jul 24 17:19:00.022570 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:19:00.022534    2581 generic.go:356] "Generic (PLEG): container finished" podID="18b37214-804a-4c4d-ac17-7b3c4a7eff93" containerID="7f37d05ecb7b5ced4f1969331a0552c4b8f4b5ca180cb8f2360fe278876cf3e9" exitCode=0
Jul 24 17:19:00.022738 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:19:00.022625    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-n6nj7" event={"ID":"18b37214-804a-4c4d-ac17-7b3c4a7eff93","Type":"ContainerDied","Data":"7f37d05ecb7b5ced4f1969331a0552c4b8f4b5ca180cb8f2360fe278876cf3e9"}
Jul 24 17:19:00.126968 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:19:00.126909    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-n6nj7"
Jul 24 17:19:00.159794 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:19:00.159764    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6zdm\" (UniqueName: \"kubernetes.io/projected/18b37214-804a-4c4d-ac17-7b3c4a7eff93-kube-api-access-v6zdm\") pod \"18b37214-804a-4c4d-ac17-7b3c4a7eff93\" (UID: \"18b37214-804a-4c4d-ac17-7b3c4a7eff93\") "
Jul 24 17:19:00.162098 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:19:00.162072    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18b37214-804a-4c4d-ac17-7b3c4a7eff93-kube-api-access-v6zdm" (OuterVolumeSpecName: "kube-api-access-v6zdm") pod "18b37214-804a-4c4d-ac17-7b3c4a7eff93" (UID: "18b37214-804a-4c4d-ac17-7b3c4a7eff93"). InnerVolumeSpecName "kube-api-access-v6zdm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:19:00.260435 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:19:00.260395    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6zdm\" (UniqueName: \"kubernetes.io/projected/18b37214-804a-4c4d-ac17-7b3c4a7eff93-kube-api-access-v6zdm\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:19:01.028725 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:19:01.028697    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-n6nj7"
Jul 24 17:19:01.028725 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:19:01.028706    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-n6nj7" event={"ID":"18b37214-804a-4c4d-ac17-7b3c4a7eff93","Type":"ContainerDied","Data":"c8aebb6b3cf2dbc770c01fb3f7565d975dc2d91030774d94203bfe8521747452"}
Jul 24 17:19:01.029215 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:19:01.028761    2581 scope.go:117] "RemoveContainer" containerID="7f37d05ecb7b5ced4f1969331a0552c4b8f4b5ca180cb8f2360fe278876cf3e9"
Jul 24 17:19:01.049670 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:19:01.049643    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-n6nj7"]
Jul 24 17:19:01.053581 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:19:01.053556    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-n6nj7"]
Jul 24 17:19:01.382997 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:19:01.382968    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18b37214-804a-4c4d-ac17-7b3c4a7eff93" path="/var/lib/kubelet/pods/18b37214-804a-4c4d-ac17-7b3c4a7eff93/volumes"
Jul 24 17:19:04.521849 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:19:04.521810    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:19:04.691354 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:19:04.691318    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:19:04.767815 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:19:04.767782    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:19:04.832113 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:19:04.831984    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:19:07.579247 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:19:07.579191    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:19:07.680215 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:19:07.680183    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:19:39.354878 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:19:39.354843    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:19:39.454217 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:19:39.454122    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:19:39.855796 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:19:39.855768    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:19:40.018830 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:19:40.018767    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:19:40.599382 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:19:40.599347    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:19:40.699332 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:19:40.699293    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:19:40.716235 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:19:40.716202    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:19:45.046776 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:19:45.046734    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:19:45.130398 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:19:45.130349    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:19:48.765713 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:19:48.765680    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:19:48.845924 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:19:48.845893    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:19:50.121821 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:19:50.121789    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:19:50.234487 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:19:50.234450    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:20:10.673289 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:10.673259    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd"]
Jul 24 17:20:10.673774 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:10.673658    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18b37214-804a-4c4d-ac17-7b3c4a7eff93" containerName="authorino"
Jul 24 17:20:10.673774 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:10.673671    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="18b37214-804a-4c4d-ac17-7b3c4a7eff93" containerName="authorino"
Jul 24 17:20:10.673774 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:10.673738    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="18b37214-804a-4c4d-ac17-7b3c4a7eff93" containerName="authorino"
Jul 24 17:20:10.675958 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:10.675943    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd"
Jul 24 17:20:10.678877 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:10.678852    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 24 17:20:10.678992 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:10.678892    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 24 17:20:10.680012 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:10.679997    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-7nbv9\""
Jul 24 17:20:10.680079 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:10.680024    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 24 17:20:10.686471 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:10.686443    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd"]
Jul 24 17:20:10.763435 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:10.763406    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:20:10.777672 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:10.777645    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57141de1-829e-4358-9620-6427e97dcab1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd\" (UID: \"57141de1-829e-4358-9620-6427e97dcab1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd"
Jul 24 17:20:10.777809 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:10.777702    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57141de1-829e-4358-9620-6427e97dcab1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd\" (UID: \"57141de1-829e-4358-9620-6427e97dcab1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd"
Jul 24 17:20:10.777809 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:10.777752    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8kkj\" (UniqueName: \"kubernetes.io/projected/57141de1-829e-4358-9620-6427e97dcab1-kube-api-access-q8kkj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd\" (UID: \"57141de1-829e-4358-9620-6427e97dcab1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd"
Jul 24 17:20:10.777911 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:10.777810    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57141de1-829e-4358-9620-6427e97dcab1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd\" (UID: \"57141de1-829e-4358-9620-6427e97dcab1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd"
Jul 24 17:20:10.777911 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:10.777837    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57141de1-829e-4358-9620-6427e97dcab1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd\" (UID: \"57141de1-829e-4358-9620-6427e97dcab1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd"
Jul 24 17:20:10.777911 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:10.777879    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57141de1-829e-4358-9620-6427e97dcab1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd\" (UID: \"57141de1-829e-4358-9620-6427e97dcab1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd"
Jul 24 17:20:10.878361 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:10.878328    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57141de1-829e-4358-9620-6427e97dcab1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd\" (UID: \"57141de1-829e-4358-9620-6427e97dcab1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd"
Jul 24 17:20:10.878545 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:10.878384    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57141de1-829e-4358-9620-6427e97dcab1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd\" (UID: \"57141de1-829e-4358-9620-6427e97dcab1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd"
Jul 24 17:20:10.878545 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:10.878404    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8kkj\" (UniqueName: \"kubernetes.io/projected/57141de1-829e-4358-9620-6427e97dcab1-kube-api-access-q8kkj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd\" (UID: \"57141de1-829e-4358-9620-6427e97dcab1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd"
Jul 24 17:20:10.878545 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:10.878428    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57141de1-829e-4358-9620-6427e97dcab1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd\" (UID: \"57141de1-829e-4358-9620-6427e97dcab1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd"
Jul 24 17:20:10.878545 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:10.878443    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57141de1-829e-4358-9620-6427e97dcab1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd\" (UID: \"57141de1-829e-4358-9620-6427e97dcab1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd"
Jul 24 17:20:10.878545 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:10.878464    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57141de1-829e-4358-9620-6427e97dcab1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd\" (UID: \"57141de1-829e-4358-9620-6427e97dcab1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd"
Jul 24 17:20:10.878809 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:10.878781    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57141de1-829e-4358-9620-6427e97dcab1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd\" (UID: \"57141de1-829e-4358-9620-6427e97dcab1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd"
Jul 24 17:20:10.878870 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:10.878812    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57141de1-829e-4358-9620-6427e97dcab1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd\" (UID: \"57141de1-829e-4358-9620-6427e97dcab1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd"
Jul 24 17:20:10.878870 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:10.878844    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57141de1-829e-4358-9620-6427e97dcab1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd\" (UID: \"57141de1-829e-4358-9620-6427e97dcab1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd"
Jul 24 17:20:10.880886 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:10.880865    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57141de1-829e-4358-9620-6427e97dcab1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd\" (UID: \"57141de1-829e-4358-9620-6427e97dcab1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd"
Jul 24 17:20:10.881075 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:10.881058    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57141de1-829e-4358-9620-6427e97dcab1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd\" (UID: \"57141de1-829e-4358-9620-6427e97dcab1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd"
Jul 24 17:20:10.886449 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:10.886432    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8kkj\" (UniqueName: \"kubernetes.io/projected/57141de1-829e-4358-9620-6427e97dcab1-kube-api-access-q8kkj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd\" (UID: \"57141de1-829e-4358-9620-6427e97dcab1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd"
Jul 24 17:20:10.987401 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:10.987312    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd"
Jul 24 17:20:11.085991 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:11.085960    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:20:11.122524 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:11.122503    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd"]
Jul 24 17:20:11.124886 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:20:11.124859    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57141de1_829e_4358_9620_6427e97dcab1.slice/crio-0ce01fc0588eed3b8f46767592a28eae7b2e386837dae8bbc28282ac237ac66e WatchSource:0}: Error finding container 0ce01fc0588eed3b8f46767592a28eae7b2e386837dae8bbc28282ac237ac66e: Status 404 returned error can't find the container with id 0ce01fc0588eed3b8f46767592a28eae7b2e386837dae8bbc28282ac237ac66e
Jul 24 17:20:11.359720 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:11.359637    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd" event={"ID":"57141de1-829e-4358-9620-6427e97dcab1","Type":"ContainerStarted","Data":"0ce01fc0588eed3b8f46767592a28eae7b2e386837dae8bbc28282ac237ac66e"}
Jul 24 17:20:12.366144 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:12.366102    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd" event={"ID":"57141de1-829e-4358-9620-6427e97dcab1","Type":"ContainerStarted","Data":"0a3f62d608922b0de5b910a32be1f56aee485d9b9f7517e382ecb98aebef0045"}
Jul 24 17:20:12.385349 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:12.385300    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd" podStartSLOduration=1.259641913 podStartE2EDuration="2.385284547s" totalImagesPullingTime="1.125642634s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:20:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:20:12.384133583 +0000 UTC m=+579.586182867" watchObservedRunningTime="2026-07-24 17:20:12.385284547 +0000 UTC m=+579.587333826"
Jul 24 17:20:18.138755 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:18.138709    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:20:19.067642 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:19.067608    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:20:20.988223 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:20.988188    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd"
Jul 24 17:20:20.988641 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:20.988267    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd"
Jul 24 17:20:21.000516 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:21.000485    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd"
Jul 24 17:20:21.420271 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:21.420241    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd"
Jul 24 17:20:25.977686 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:25.977647    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:20:27.252684 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:27.252652    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:20:33.842391 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:33.842349    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv"]
Jul 24 17:20:33.846077 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:33.846056    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv"
Jul 24 17:20:33.849061 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:33.849020    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 24 17:20:33.868886 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:33.868860    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv"]
Jul 24 17:20:33.884597 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:33.884568    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:20:33.889981 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:33.889962    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73d0f656-1ccb-4fd7-9639-6a515a9284e9-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv\" (UID: \"73d0f656-1ccb-4fd7-9639-6a515a9284e9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv"
Jul 24 17:20:33.890139 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:33.889991    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjv8h\" (UniqueName: \"kubernetes.io/projected/73d0f656-1ccb-4fd7-9639-6a515a9284e9-kube-api-access-kjv8h\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv\" (UID: \"73d0f656-1ccb-4fd7-9639-6a515a9284e9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv"
Jul 24 17:20:33.890139 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:33.890079    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73d0f656-1ccb-4fd7-9639-6a515a9284e9-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv\" (UID: \"73d0f656-1ccb-4fd7-9639-6a515a9284e9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv"
Jul 24 17:20:33.890139 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:33.890107    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73d0f656-1ccb-4fd7-9639-6a515a9284e9-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv\" (UID: \"73d0f656-1ccb-4fd7-9639-6a515a9284e9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv"
Jul 24 17:20:33.890244 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:33.890168    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73d0f656-1ccb-4fd7-9639-6a515a9284e9-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv\" (UID: \"73d0f656-1ccb-4fd7-9639-6a515a9284e9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv"
Jul 24 17:20:33.890244 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:33.890199    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73d0f656-1ccb-4fd7-9639-6a515a9284e9-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv\" (UID: \"73d0f656-1ccb-4fd7-9639-6a515a9284e9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv"
Jul 24 17:20:33.991400 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:33.991363    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73d0f656-1ccb-4fd7-9639-6a515a9284e9-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv\" (UID: \"73d0f656-1ccb-4fd7-9639-6a515a9284e9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv"
Jul 24 17:20:33.991400 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:33.991399    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73d0f656-1ccb-4fd7-9639-6a515a9284e9-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv\" (UID: \"73d0f656-1ccb-4fd7-9639-6a515a9284e9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv"
Jul 24 17:20:33.991643 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:33.991430    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73d0f656-1ccb-4fd7-9639-6a515a9284e9-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv\" (UID: \"73d0f656-1ccb-4fd7-9639-6a515a9284e9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv"
Jul 24 17:20:33.991643 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:33.991457    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73d0f656-1ccb-4fd7-9639-6a515a9284e9-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv\" (UID: \"73d0f656-1ccb-4fd7-9639-6a515a9284e9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv"
Jul 24 17:20:33.991643 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:33.991499    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73d0f656-1ccb-4fd7-9639-6a515a9284e9-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv\" (UID: \"73d0f656-1ccb-4fd7-9639-6a515a9284e9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv"
Jul 24 17:20:33.991643 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:33.991526    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjv8h\" (UniqueName: \"kubernetes.io/projected/73d0f656-1ccb-4fd7-9639-6a515a9284e9-kube-api-access-kjv8h\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv\" (UID: \"73d0f656-1ccb-4fd7-9639-6a515a9284e9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv"
Jul 24 17:20:33.991870 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:33.991799    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73d0f656-1ccb-4fd7-9639-6a515a9284e9-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv\" (UID: \"73d0f656-1ccb-4fd7-9639-6a515a9284e9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv"
Jul 24 17:20:33.991870 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:33.991847    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73d0f656-1ccb-4fd7-9639-6a515a9284e9-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv\" (UID: \"73d0f656-1ccb-4fd7-9639-6a515a9284e9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv"
Jul 24 17:20:33.991958 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:33.991913    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73d0f656-1ccb-4fd7-9639-6a515a9284e9-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv\" (UID: \"73d0f656-1ccb-4fd7-9639-6a515a9284e9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv"
Jul 24 17:20:33.993941 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:33.993914    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73d0f656-1ccb-4fd7-9639-6a515a9284e9-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv\" (UID: \"73d0f656-1ccb-4fd7-9639-6a515a9284e9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv"
Jul 24 17:20:33.994165 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:33.994148    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73d0f656-1ccb-4fd7-9639-6a515a9284e9-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv\" (UID: \"73d0f656-1ccb-4fd7-9639-6a515a9284e9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv"
Jul 24 17:20:34.000367 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:34.000344    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjv8h\" (UniqueName: \"kubernetes.io/projected/73d0f656-1ccb-4fd7-9639-6a515a9284e9-kube-api-access-kjv8h\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv\" (UID: \"73d0f656-1ccb-4fd7-9639-6a515a9284e9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv"
Jul 24 17:20:34.157148 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:34.157101    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv"
Jul 24 17:20:34.291454 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:34.291428    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv"]
Jul 24 17:20:34.293821 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:20:34.293793    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73d0f656_1ccb_4fd7_9639_6a515a9284e9.slice/crio-a6b3b1bdc06a25391a846bc3b142cd285a08e16edf2b3477922b66b9ad79e7d9 WatchSource:0}: Error finding container a6b3b1bdc06a25391a846bc3b142cd285a08e16edf2b3477922b66b9ad79e7d9: Status 404 returned error can't find the container with id a6b3b1bdc06a25391a846bc3b142cd285a08e16edf2b3477922b66b9ad79e7d9
Jul 24 17:20:34.471535 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:34.471454    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv" event={"ID":"73d0f656-1ccb-4fd7-9639-6a515a9284e9","Type":"ContainerStarted","Data":"a6b3b1bdc06a25391a846bc3b142cd285a08e16edf2b3477922b66b9ad79e7d9"}
Jul 24 17:20:35.477839 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:35.477801    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv" event={"ID":"73d0f656-1ccb-4fd7-9639-6a515a9284e9","Type":"ContainerStarted","Data":"3a98d7d3d46d7f9a19a29ae949510f910b1cdcd72837d6fcfa1007bae6707a7a"}
Jul 24 17:20:35.498275 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:35.498224    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv" podStartSLOduration=2.310272314 podStartE2EDuration="2.498204472s" totalImagesPullingTime="187.932158ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:20:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:20:35.497561865 +0000 UTC m=+602.699611141" watchObservedRunningTime="2026-07-24 17:20:35.498204472 +0000 UTC m=+602.700253747"
Jul 24 17:20:35.664087 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:35.664057    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:20:44.158116 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:44.158061    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv"
Jul 24 17:20:44.158645 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:44.158152    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv"
Jul 24 17:20:44.171078 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:44.171030    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv"
Jul 24 17:20:44.529716 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:44.529634    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv"
Jul 24 17:20:45.847479 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:45.847446    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:20:45.904907 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:45.904873    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:20:47.884527 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:47.884487    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:20:50.461303 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:20:50.461272    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:21:02.367216 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:21:02.367185    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv"]
Jul 24 17:21:02.371083 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:21:02.371060    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv"
Jul 24 17:21:02.374523 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:21:02.374496    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 24 17:21:02.382738 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:21:02.382715    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv"]
Jul 24 17:21:02.410600 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:21:02.410561    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:21:02.465942 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:21:02.465900    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21472c42-aa48-4fda-8e8d-b30e8ec5def9-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv\" (UID: \"21472c42-aa48-4fda-8e8d-b30e8ec5def9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv"
Jul 24 17:21:02.466144 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:21:02.466076    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21472c42-aa48-4fda-8e8d-b30e8ec5def9-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv\" (UID: \"21472c42-aa48-4fda-8e8d-b30e8ec5def9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv"
Jul 24 17:21:02.466144 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:21:02.466105    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21472c42-aa48-4fda-8e8d-b30e8ec5def9-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv\" (UID: \"21472c42-aa48-4fda-8e8d-b30e8ec5def9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv"
Jul 24 17:21:02.466144 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:21:02.466121    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wt7jm\" (UniqueName: \"kubernetes.io/projected/21472c42-aa48-4fda-8e8d-b30e8ec5def9-kube-api-access-wt7jm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv\" (UID: \"21472c42-aa48-4fda-8e8d-b30e8ec5def9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv"
Jul 24 17:21:02.466278 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:21:02.466259    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21472c42-aa48-4fda-8e8d-b30e8ec5def9-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv\" (UID: \"21472c42-aa48-4fda-8e8d-b30e8ec5def9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv"
Jul 24 17:21:02.466316 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:21:02.466288    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21472c42-aa48-4fda-8e8d-b30e8ec5def9-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv\" (UID: \"21472c42-aa48-4fda-8e8d-b30e8ec5def9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv"
Jul 24 17:21:02.567095 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:21:02.567063    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21472c42-aa48-4fda-8e8d-b30e8ec5def9-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv\" (UID: \"21472c42-aa48-4fda-8e8d-b30e8ec5def9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv"
Jul 24 17:21:02.567095 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:21:02.567103    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21472c42-aa48-4fda-8e8d-b30e8ec5def9-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv\" (UID: \"21472c42-aa48-4fda-8e8d-b30e8ec5def9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv"
Jul 24 17:21:02.567351 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:21:02.567124    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21472c42-aa48-4fda-8e8d-b30e8ec5def9-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv\" (UID: \"21472c42-aa48-4fda-8e8d-b30e8ec5def9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv"
Jul 24 17:21:02.567351 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:21:02.567198    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21472c42-aa48-4fda-8e8d-b30e8ec5def9-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv\" (UID: \"21472c42-aa48-4fda-8e8d-b30e8ec5def9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv"
Jul 24 17:21:02.567351 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:21:02.567214    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21472c42-aa48-4fda-8e8d-b30e8ec5def9-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv\" (UID: \"21472c42-aa48-4fda-8e8d-b30e8ec5def9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv"
Jul 24 17:21:02.567351 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:21:02.567228    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wt7jm\" (UniqueName: \"kubernetes.io/projected/21472c42-aa48-4fda-8e8d-b30e8ec5def9-kube-api-access-wt7jm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv\" (UID: \"21472c42-aa48-4fda-8e8d-b30e8ec5def9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv"
Jul 24 17:21:02.567565 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:21:02.567449    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21472c42-aa48-4fda-8e8d-b30e8ec5def9-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv\" (UID: \"21472c42-aa48-4fda-8e8d-b30e8ec5def9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv"
Jul 24 17:21:02.567618 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:21:02.567568    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21472c42-aa48-4fda-8e8d-b30e8ec5def9-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv\" (UID: \"21472c42-aa48-4fda-8e8d-b30e8ec5def9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv"
Jul 24 17:21:02.567618 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:21:02.567608    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21472c42-aa48-4fda-8e8d-b30e8ec5def9-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv\" (UID: \"21472c42-aa48-4fda-8e8d-b30e8ec5def9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv"
Jul 24 17:21:02.569566 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:21:02.569543    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21472c42-aa48-4fda-8e8d-b30e8ec5def9-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv\" (UID: \"21472c42-aa48-4fda-8e8d-b30e8ec5def9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv"
Jul 24 17:21:02.569703 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:21:02.569687    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21472c42-aa48-4fda-8e8d-b30e8ec5def9-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv\" (UID: \"21472c42-aa48-4fda-8e8d-b30e8ec5def9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv"
Jul 24 17:21:02.575703 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:21:02.575683    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wt7jm\" (UniqueName: \"kubernetes.io/projected/21472c42-aa48-4fda-8e8d-b30e8ec5def9-kube-api-access-wt7jm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv\" (UID: \"21472c42-aa48-4fda-8e8d-b30e8ec5def9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv"
Jul 24 17:21:02.682269 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:21:02.682238    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv"
Jul 24 17:21:02.814829 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:21:02.814799    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv"]
Jul 24 17:21:02.818931 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:21:02.817229    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod21472c42_aa48_4fda_8e8d_b30e8ec5def9.slice/crio-0d4b3d617b665b806f28751e715aeb9313ac1454366faee815319170c7727f36 WatchSource:0}: Error finding container 0d4b3d617b665b806f28751e715aeb9313ac1454366faee815319170c7727f36: Status 404 returned error can't find the container with id 0d4b3d617b665b806f28751e715aeb9313ac1454366faee815319170c7727f36
Jul 24 17:21:02.821388 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:21:02.821365    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 17:21:03.610400 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:21:03.610361    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv" event={"ID":"21472c42-aa48-4fda-8e8d-b30e8ec5def9","Type":"ContainerStarted","Data":"52385b419a2a5099fddbf76559f1e60daa9ad9349ea1749c1324cff367943ca1"}
Jul 24 17:21:03.610400 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:21:03.610410    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv" event={"ID":"21472c42-aa48-4fda-8e8d-b30e8ec5def9","Type":"ContainerStarted","Data":"0d4b3d617b665b806f28751e715aeb9313ac1454366faee815319170c7727f36"}
Jul 24 17:21:03.629053 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:21:03.628983    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv" podStartSLOduration=1.440454882 podStartE2EDuration="1.628967812s" totalImagesPullingTime="188.51293ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:21:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:21:03.627539422 +0000 UTC m=+630.829588696" watchObservedRunningTime="2026-07-24 17:21:03.628967812 +0000 UTC m=+630.831017087"
Jul 24 17:21:04.952882 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:21:04.952844    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:21:11.500176 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:21:11.500143    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:21:11.600949 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:21:11.600914    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:21:12.682364 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:21:12.682321    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv"
Jul 24 17:21:12.682876 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:21:12.682381    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv"
Jul 24 17:21:12.694964 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:21:12.694939    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv"
Jul 24 17:21:13.674085 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:21:13.674058    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv"
Jul 24 17:22:06.452082 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:22:06.452024    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:22:06.545677 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:22:06.545641    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:22:07.587720 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:22:07.587665    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:22:07.737712 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:22:07.737672    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:22:37.061442 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:22:37.061397    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:22:46.159229 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:22:46.159193    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:22:57.253894 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:22:57.253859    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:23:37.847638 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:23:37.847607    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:23:37.900059 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:23:37.900004    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:24:12.359233 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:24:12.359193    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:24:13.201281 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:24:13.201243    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:24:13.568502 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:24:13.568424    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:24:36.462496 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:24:36.462460    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:24:46.984082 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:24:46.984017    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:24:47.084221 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:24:47.084173    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:24:47.205949 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:24:47.205912    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:24:51.288374 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:24:51.288338    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:24:52.873377 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:24:52.873341    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:24:52.950306 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:24:52.950269    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:24:53.046053 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:24:53.046005    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:24:53.230261 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:24:53.230229    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:24:58.828929 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:24:58.828889    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:24:58.897753 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:24:58.897717    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:25:15.548341 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:25:15.548303    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:25:25.450139 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:25:25.450034    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:25:35.858278 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:25:35.858159    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:25:41.462089 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:25:41.462033    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:25:52.162506 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:25:52.161431    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:26:03.863857 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:26:03.858724    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:26:14.855733 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:26:14.855694    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:26:22.404642 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:26:22.404603    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:26:30.456379 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:26:30.456345    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:26:38.254049 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:26:38.254001    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:26:50.356016 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:26:50.355979    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:27:02.154823 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:27:02.154786    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:27:25.850646 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:27:25.850608    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:27:36.362569 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:27:36.362534    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:27:51.057267 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:27:51.057233    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:28:00.551183 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:28:00.551146    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:28:10.962593 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:28:10.962552    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:28:20.551133 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:28:20.551090    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:28:29.950154 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:28:29.950073    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:29:34.525006 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:29:34.524952    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:29:35.811816 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:29:35.811770    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:29:35.969910 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:29:35.969873    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:30:19.060820 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:30:19.060764    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:30:29.658828 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:30:29.658789    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:30:40.066228 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:30:40.066188    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:30:51.955142 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:30:51.955105    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:31:01.455234 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:31:01.455196    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:31:12.262214 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:31:12.262180    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:31:22.350883 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:31:22.350840    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:31:31.864265 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:31:31.864184    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:31:43.350583 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:31:43.350539    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:31:53.555027 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:31:53.554992    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:32:07.362909 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:07.362871    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:32:25.161676 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:25.161639    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:32:34.354297 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:34.354262    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:32:36.425147 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:36.425097    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:32:41.661155 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:41.661117    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:32:45.366989 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:45.366955    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck"]
Jul 24 17:32:45.370616 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:45.370600    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck"
Jul 24 17:32:45.373636 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:45.373615    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-36c07d19-kserve-self-signed-certs\""
Jul 24 17:32:45.384747 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:45.384722    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck"]
Jul 24 17:32:45.392344 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:45.392323    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:32:45.450781 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:45.450749    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7eccb9c-0150-4438-849d-fa498dd9ac41-tmp-dir\") pod \"e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck\" (UID: \"f7eccb9c-0150-4438-849d-fa498dd9ac41\") " pod="llm/e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck"
Jul 24 17:32:45.450985 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:45.450801    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7eccb9c-0150-4438-849d-fa498dd9ac41-tls-certs\") pod \"e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck\" (UID: \"f7eccb9c-0150-4438-849d-fa498dd9ac41\") " pod="llm/e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck"
Jul 24 17:32:45.450985 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:45.450953    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7eccb9c-0150-4438-849d-fa498dd9ac41-dshm\") pod \"e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck\" (UID: \"f7eccb9c-0150-4438-849d-fa498dd9ac41\") " pod="llm/e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck"
Jul 24 17:32:45.451101 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:45.451069    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7eccb9c-0150-4438-849d-fa498dd9ac41-model-cache\") pod \"e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck\" (UID: \"f7eccb9c-0150-4438-849d-fa498dd9ac41\") " pod="llm/e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck"
Jul 24 17:32:45.451242 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:45.451219    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqjcm\" (UniqueName: \"kubernetes.io/projected/f7eccb9c-0150-4438-849d-fa498dd9ac41-kube-api-access-hqjcm\") pod \"e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck\" (UID: \"f7eccb9c-0150-4438-849d-fa498dd9ac41\") " pod="llm/e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck"
Jul 24 17:32:45.451311 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:45.451262    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7eccb9c-0150-4438-849d-fa498dd9ac41-home\") pod \"e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck\" (UID: \"f7eccb9c-0150-4438-849d-fa498dd9ac41\") " pod="llm/e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck"
Jul 24 17:32:45.552068 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:45.552007    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7eccb9c-0150-4438-849d-fa498dd9ac41-home\") pod \"e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck\" (UID: \"f7eccb9c-0150-4438-849d-fa498dd9ac41\") " pod="llm/e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck"
Jul 24 17:32:45.552246 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:45.552089    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7eccb9c-0150-4438-849d-fa498dd9ac41-tmp-dir\") pod \"e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck\" (UID: \"f7eccb9c-0150-4438-849d-fa498dd9ac41\") " pod="llm/e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck"
Jul 24 17:32:45.552246 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:45.552143    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7eccb9c-0150-4438-849d-fa498dd9ac41-tls-certs\") pod \"e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck\" (UID: \"f7eccb9c-0150-4438-849d-fa498dd9ac41\") " pod="llm/e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck"
Jul 24 17:32:45.552365 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:45.552280    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7eccb9c-0150-4438-849d-fa498dd9ac41-dshm\") pod \"e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck\" (UID: \"f7eccb9c-0150-4438-849d-fa498dd9ac41\") " pod="llm/e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck"
Jul 24 17:32:45.552424 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:45.552376    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7eccb9c-0150-4438-849d-fa498dd9ac41-model-cache\") pod \"e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck\" (UID: \"f7eccb9c-0150-4438-849d-fa498dd9ac41\") " pod="llm/e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck"
Jul 24 17:32:45.552515 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:45.552487    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7eccb9c-0150-4438-849d-fa498dd9ac41-home\") pod \"e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck\" (UID: \"f7eccb9c-0150-4438-849d-fa498dd9ac41\") " pod="llm/e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck"
Jul 24 17:32:45.552689 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:45.552504    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqjcm\" (UniqueName: \"kubernetes.io/projected/f7eccb9c-0150-4438-849d-fa498dd9ac41-kube-api-access-hqjcm\") pod \"e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck\" (UID: \"f7eccb9c-0150-4438-849d-fa498dd9ac41\") " pod="llm/e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck"
Jul 24 17:32:45.552792 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:45.552531    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7eccb9c-0150-4438-849d-fa498dd9ac41-tmp-dir\") pod \"e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck\" (UID: \"f7eccb9c-0150-4438-849d-fa498dd9ac41\") " pod="llm/e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck"
Jul 24 17:32:45.552792 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:45.552665    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7eccb9c-0150-4438-849d-fa498dd9ac41-model-cache\") pod \"e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck\" (UID: \"f7eccb9c-0150-4438-849d-fa498dd9ac41\") " pod="llm/e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck"
Jul 24 17:32:45.554589 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:45.554570    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7eccb9c-0150-4438-849d-fa498dd9ac41-dshm\") pod \"e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck\" (UID: \"f7eccb9c-0150-4438-849d-fa498dd9ac41\") " pod="llm/e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck"
Jul 24 17:32:45.554835 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:45.554820    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7eccb9c-0150-4438-849d-fa498dd9ac41-tls-certs\") pod \"e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck\" (UID: \"f7eccb9c-0150-4438-849d-fa498dd9ac41\") " pod="llm/e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck"
Jul 24 17:32:45.561573 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:45.561549    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqjcm\" (UniqueName: \"kubernetes.io/projected/f7eccb9c-0150-4438-849d-fa498dd9ac41-kube-api-access-hqjcm\") pod \"e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck\" (UID: \"f7eccb9c-0150-4438-849d-fa498dd9ac41\") " pod="llm/e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck"
Jul 24 17:32:45.680129 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:45.680097    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck"
Jul 24 17:32:45.784490 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:45.784448    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:32:45.814824 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:45.814800    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck"]
Jul 24 17:32:45.816663 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:32:45.816633    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf7eccb9c_0150_4438_849d_fa498dd9ac41.slice/crio-b8fe232f65fe83882951da6bdb1fa449e7b8adc8e7a84b31fb5dad731ee4edc8 WatchSource:0}: Error finding container b8fe232f65fe83882951da6bdb1fa449e7b8adc8e7a84b31fb5dad731ee4edc8: Status 404 returned error can't find the container with id b8fe232f65fe83882951da6bdb1fa449e7b8adc8e7a84b31fb5dad731ee4edc8
Jul 24 17:32:45.988762 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:45.988672    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck" event={"ID":"f7eccb9c-0150-4438-849d-fa498dd9ac41","Type":"ContainerStarted","Data":"163b08d066682940390501a192030d80977c260744c688b59622e4078f023c6d"}
Jul 24 17:32:45.988762 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:45.988719    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck" event={"ID":"f7eccb9c-0150-4438-849d-fa498dd9ac41","Type":"ContainerStarted","Data":"b8fe232f65fe83882951da6bdb1fa449e7b8adc8e7a84b31fb5dad731ee4edc8"}
Jul 24 17:32:46.016701 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:46.016649    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck" podStartSLOduration=1.016632523 podStartE2EDuration="1.016632523s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:32:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:32:46.014871898 +0000 UTC m=+1333.216921173" watchObservedRunningTime="2026-07-24 17:32:46.016632523 +0000 UTC m=+1333.218681797"
Jul 24 17:32:50.083585 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:50.083549    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:32:55.681183 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:55.681104    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck"
Jul 24 17:32:55.681183 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:55.681148    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck"
Jul 24 17:32:55.693716 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:55.693690    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck"
Jul 24 17:32:56.046251 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:32:56.046164    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck"
Jul 24 17:33:24.313160 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:33:24.313121    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:33:25.040452 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:33:25.040419    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck"]
Jul 24 17:33:25.055691 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:33:25.055658    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:33:27.961887 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:33:27.961850    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:33:40.060152 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:33:40.060113    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck" podUID="f7eccb9c-0150-4438-849d-fa498dd9ac41" containerName="main" containerID="cri-o://163b08d066682940390501a192030d80977c260744c688b59622e4078f023c6d" gracePeriod=45
Jul 24 17:33:40.245529 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:33:40.245479    2581 generic.go:356] "Generic (PLEG): container finished" podID="f7eccb9c-0150-4438-849d-fa498dd9ac41" containerID="163b08d066682940390501a192030d80977c260744c688b59622e4078f023c6d" exitCode=0
Jul 24 17:33:40.245708 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:33:40.245544    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck" event={"ID":"f7eccb9c-0150-4438-849d-fa498dd9ac41","Type":"ContainerDied","Data":"163b08d066682940390501a192030d80977c260744c688b59622e4078f023c6d"}
Jul 24 17:33:40.314538 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:33:40.314472    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck"
Jul 24 17:33:40.379983 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:33:40.379954    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hqjcm\" (UniqueName: \"kubernetes.io/projected/f7eccb9c-0150-4438-849d-fa498dd9ac41-kube-api-access-hqjcm\") pod \"f7eccb9c-0150-4438-849d-fa498dd9ac41\" (UID: \"f7eccb9c-0150-4438-849d-fa498dd9ac41\") "
Jul 24 17:33:40.380171 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:33:40.380019    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7eccb9c-0150-4438-849d-fa498dd9ac41-tls-certs\") pod \"f7eccb9c-0150-4438-849d-fa498dd9ac41\" (UID: \"f7eccb9c-0150-4438-849d-fa498dd9ac41\") "
Jul 24 17:33:40.380171 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:33:40.380091    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7eccb9c-0150-4438-849d-fa498dd9ac41-dshm\") pod \"f7eccb9c-0150-4438-849d-fa498dd9ac41\" (UID: \"f7eccb9c-0150-4438-849d-fa498dd9ac41\") "
Jul 24 17:33:40.380171 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:33:40.380145    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7eccb9c-0150-4438-849d-fa498dd9ac41-tmp-dir\") pod \"f7eccb9c-0150-4438-849d-fa498dd9ac41\" (UID: \"f7eccb9c-0150-4438-849d-fa498dd9ac41\") "
Jul 24 17:33:40.380311 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:33:40.380187    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7eccb9c-0150-4438-849d-fa498dd9ac41-model-cache\") pod \"f7eccb9c-0150-4438-849d-fa498dd9ac41\" (UID: \"f7eccb9c-0150-4438-849d-fa498dd9ac41\") "
Jul 24 17:33:40.380311 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:33:40.380221    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7eccb9c-0150-4438-849d-fa498dd9ac41-home\") pod \"f7eccb9c-0150-4438-849d-fa498dd9ac41\" (UID: \"f7eccb9c-0150-4438-849d-fa498dd9ac41\") "
Jul 24 17:33:40.380516 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:33:40.380487    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7eccb9c-0150-4438-849d-fa498dd9ac41-model-cache" (OuterVolumeSpecName: "model-cache") pod "f7eccb9c-0150-4438-849d-fa498dd9ac41" (UID: "f7eccb9c-0150-4438-849d-fa498dd9ac41"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:33:40.380591 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:33:40.380502    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7eccb9c-0150-4438-849d-fa498dd9ac41-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f7eccb9c-0150-4438-849d-fa498dd9ac41" (UID: "f7eccb9c-0150-4438-849d-fa498dd9ac41"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:33:40.380634 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:33:40.380619    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7eccb9c-0150-4438-849d-fa498dd9ac41-home" (OuterVolumeSpecName: "home") pod "f7eccb9c-0150-4438-849d-fa498dd9ac41" (UID: "f7eccb9c-0150-4438-849d-fa498dd9ac41"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:33:40.382384 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:33:40.382367    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7eccb9c-0150-4438-849d-fa498dd9ac41-dshm" (OuterVolumeSpecName: "dshm") pod "f7eccb9c-0150-4438-849d-fa498dd9ac41" (UID: "f7eccb9c-0150-4438-849d-fa498dd9ac41"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:33:40.382776 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:33:40.382748    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7eccb9c-0150-4438-849d-fa498dd9ac41-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f7eccb9c-0150-4438-849d-fa498dd9ac41" (UID: "f7eccb9c-0150-4438-849d-fa498dd9ac41"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:33:40.382869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:33:40.382751    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7eccb9c-0150-4438-849d-fa498dd9ac41-kube-api-access-hqjcm" (OuterVolumeSpecName: "kube-api-access-hqjcm") pod "f7eccb9c-0150-4438-849d-fa498dd9ac41" (UID: "f7eccb9c-0150-4438-849d-fa498dd9ac41"). InnerVolumeSpecName "kube-api-access-hqjcm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:33:40.480988 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:33:40.480948    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7eccb9c-0150-4438-849d-fa498dd9ac41-tls-certs\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:33:40.480988 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:33:40.480981    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7eccb9c-0150-4438-849d-fa498dd9ac41-dshm\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:33:40.480988 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:33:40.480992    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7eccb9c-0150-4438-849d-fa498dd9ac41-tmp-dir\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:33:40.481248 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:33:40.481004    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7eccb9c-0150-4438-849d-fa498dd9ac41-model-cache\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:33:40.481248 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:33:40.481014    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7eccb9c-0150-4438-849d-fa498dd9ac41-home\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:33:40.481248 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:33:40.481024    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqjcm\" (UniqueName: \"kubernetes.io/projected/f7eccb9c-0150-4438-849d-fa498dd9ac41-kube-api-access-hqjcm\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:33:41.251929 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:33:41.251892    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck" event={"ID":"f7eccb9c-0150-4438-849d-fa498dd9ac41","Type":"ContainerDied","Data":"b8fe232f65fe83882951da6bdb1fa449e7b8adc8e7a84b31fb5dad731ee4edc8"}
Jul 24 17:33:41.252377 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:33:41.251942    2581 scope.go:117] "RemoveContainer" containerID="163b08d066682940390501a192030d80977c260744c688b59622e4078f023c6d"
Jul 24 17:33:41.252377 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:33:41.251903    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck"
Jul 24 17:33:41.274512 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:33:41.274485    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck"]
Jul 24 17:33:41.278445 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:33:41.278422    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-36c07d19-kserve-765dfd5d67-lb7ck"]
Jul 24 17:33:41.384801 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:33:41.384766    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7eccb9c-0150-4438-849d-fa498dd9ac41" path="/var/lib/kubelet/pods/f7eccb9c-0150-4438-849d-fa498dd9ac41/volumes"
Jul 24 17:33:41.666982 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:33:41.666942    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:33:48.746476 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:33:48.746446    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:34:04.661838 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:34:04.661803    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:34:18.853277 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:34:18.853240    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:34:28.655726 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:34:28.655633    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:34:41.473117 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:34:41.473084    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:34:44.620474 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:34:44.620410    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:34:45.814799 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:34:45.814742    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:34:51.055320 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:34:51.055278    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:35:03.163788 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:35:03.163755    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:35:15.356827 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:35:15.356776    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:35:30.664898 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:35:30.664856    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:35:38.652092 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:35:38.652027    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:35:52.653440 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:35:52.653404    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:36:04.753921 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:36:04.753882    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:36:16.550862 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:36:16.550830    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:36:26.055414 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:36:26.055380    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:36:28.607683 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:36:28.607642    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:36:41.095681 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:36:41.095648    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:36:49.926630 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:36:49.926595    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:36:53.058985 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:36:53.058946    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:36:53.154917 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:36:53.154884    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:36:53.365020 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:36:53.364937    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:36:53.493016 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:36:53.492989    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:36:59.140067 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:36:59.139988    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:36:59.240381 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:36:59.240348    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:36:59.377473 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:36:59.377416    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:36:59.674462 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:36:59.674423    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:36:59.774550 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:36:59.774522    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:37:02.389677 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:37:02.389650    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:37:02.486390 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:37:02.486360    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:37:02.579869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:37:02.579843    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:37:02.770402 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:37:02.770322    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:37:02.945887 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:37:02.945853    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:37:03.041892 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:37:03.041808    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:37:09.457750 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:37:09.457713    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:37:38.480403 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:37:38.480290    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:37:38.578996 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:37:38.578957    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:37:41.650372 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:37:41.650332    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:37:43.074831 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:37:43.074793    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:37:43.234956 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:37:43.234927    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:37:43.531494 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:37:43.531454    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:37:43.752014 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:37:43.751964    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:37:43.849388 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:37:43.849304    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:37:43.951912 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:37:43.951877    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:37:44.298432 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:37:44.298396    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:37:44.477293 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:37:44.477260    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:37:46.737388 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:37:46.737357    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:37:46.846797 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:37:46.846761    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:37:47.036813 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:37:47.036735    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:37:47.338620 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:37:47.338521    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:37:47.429458 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:37:47.429430    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:37:52.456401 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:37:52.456367    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:38:22.183385 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:38:22.183349    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:38:22.244699 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:38:22.244650    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:38:22.304331 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:38:22.304292    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:38:25.456190 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:38:25.456153    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:38:26.881154 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:38:26.881118    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:38:27.033221 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:38:27.033187    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:38:27.278932 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:38:27.278899    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:38:27.528298 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:38:27.528266    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:38:27.841928 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:38:27.841897    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:38:28.538908 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:38:28.538856    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:38:29.964856 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:38:29.964825    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:38:30.064501 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:38:30.064469    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:38:30.169364 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:38:30.169330    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:38:30.630432 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:38:30.630399    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:38:30.753003 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:38:30.752959    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:38:36.759280 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:38:36.759245    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:39:06.164999 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:39:06.164962    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:39:06.230918 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:39:06.230872    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:39:06.301979 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:39:06.301948    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:39:09.155824 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:39:09.155776    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:39:10.688222 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:39:10.688194    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:39:11.881451 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:39:11.881416    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:39:12.101759 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:39:12.101728    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:39:12.348839 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:39:12.348807    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:39:12.653714 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:39:12.653681    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:39:12.972845 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:39:12.972765    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:39:15.040277 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:39:15.040244    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:39:15.121006 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:39:15.120967    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:39:15.190114 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:39:15.190082    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:39:15.463224 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:39:15.463187    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:39:15.590593 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:39:15.590562    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:39:15.691526 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:39:15.691497    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:39:20.158563 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:39:20.158525    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:39:26.366686 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:39:26.366647    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:39:26.466721 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:39:26.466691    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:39:50.316947 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:39:50.316912    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:39:51.426455 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:39:51.426420    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:39:53.454112 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:39:53.454068    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:39:54.995150 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:39:54.995114    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:39:55.188700 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:39:55.188666    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:39:55.390009 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:39:55.389963    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:39:55.897287 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:39:55.897256    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:39:56.000754 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:39:56.000714    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:39:56.097295 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:39:56.097266    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:39:57.466332 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:39:57.466295    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:40:03.096534 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:40:03.096489    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:40:03.203302 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:40:03.203266    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:40:03.297075 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:40:03.297012    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:40:03.396789 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:40:03.396757    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:40:03.816605 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:40:03.816521    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:40:29.783185 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:40:29.783107    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:40:36.181264 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:40:36.181225    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:40:36.285878 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:40:36.285843    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:40:36.411511 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:40:36.411475    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:40:36.511291 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:40:36.511206    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:40:44.812986 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:40:44.812955    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:40:52.302953 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:40:52.302916    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:40:57.924718 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:40:57.924682    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:40:58.028199 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:40:58.028154    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:40:58.125976 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:40:58.125946    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:41:08.651824 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:41:08.651791    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:41:09.455448 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:41:09.455392    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:41:15.632542 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:41:15.632509    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:41:15.766576 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:41:15.766532    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:41:15.872356 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:41:15.872320    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:41:15.964786 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:41:15.964705    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:41:16.667002 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:41:16.666962    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:41:16.751935 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:41:16.751894    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:41:16.852095 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:41:16.852063    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:41:20.159077 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:41:20.159046    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:41:23.561446 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:41:23.561396    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:41:26.861989 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:41:26.861952    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:41:30.593847 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:41:30.593804    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:41:30.694594 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:41:30.694558    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:41:31.287532 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:41:31.287492    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:41:36.659688 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:41:36.659650    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:41:37.245985 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:41:37.245952    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:41:37.345782 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:41:37.345747    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:41:37.476861 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:41:37.476827    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:41:37.787588 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:41:37.787553    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:41:38.002141 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:41:38.002102    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:41:40.477761 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:41:40.477725    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:41:40.579495 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:41:40.579467    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:41:40.745153 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:41:40.745070    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:41:41.169986 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:41:41.169957    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:41:41.289521 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:41:41.289485    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:41:41.377352 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:41:41.377310    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:41:44.993500 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:41:44.993465    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:41:45.631986 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:41:45.631951    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:41:45.728423 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:41:45.728390    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:41:46.061318 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:41:46.061232    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:41:47.748830 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:41:47.748793    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:41:49.849429 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:41:49.849395    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:41:51.770710 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:41:51.770676    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:41:51.871667 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:41:51.871634    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:42:02.271870 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:42:02.271834    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:42:02.343749 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:42:02.343712    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:42:02.407923 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:42:02.407891    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:42:05.650609 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:42:05.650579    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:42:07.234245 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:42:07.234207    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:42:07.608240 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:42:07.608156    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:42:13.459820 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:42:13.459784    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:42:13.538631 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:42:13.538598    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:42:13.683388 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:42:13.683352    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:42:13.903420 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:42:13.903388    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:42:14.037829 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:42:14.037798    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:42:14.130448 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:42:14.130418    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:42:14.248973 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:42:14.248888    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:42:19.855944 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:42:19.855905    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:42:19.957683 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:42:19.957653    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:42:20.056454 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:42:20.056422    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:42:20.198556 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:42:20.198518    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:42:25.841551 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:42:25.841513    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:42:25.897555 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:42:25.897517    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:42:32.592179 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:42:32.592133    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:42:32.885436 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:42:32.885400    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:42:33.288867 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:42:33.288777    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:42:35.493457 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:42:35.493424    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:42:35.592268 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:42:35.592233    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:42:35.754458 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:42:35.754374    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:42:36.202550 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:42:36.202498    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:42:36.302089 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:42:36.302058    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:42:38.156411 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:42:38.156375    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:42:39.965563 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:42:39.965531    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:43:00.767653 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:00.767619    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-a5726951/e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr"]
Jul 24 17:43:00.768080 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:00.768006    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7eccb9c-0150-4438-849d-fa498dd9ac41" containerName="main"
Jul 24 17:43:00.768080 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:00.768017    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7eccb9c-0150-4438-849d-fa498dd9ac41" containerName="main"
Jul 24 17:43:00.768159 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:00.768095    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7eccb9c-0150-4438-849d-fa498dd9ac41" containerName="main"
Jul 24 17:43:00.771294 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:00.771278    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-a5726951/e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr"
Jul 24 17:43:00.774229 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:00.774205    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-a5726951\"/\"openshift-service-ca.crt\""
Jul 24 17:43:00.774373 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:00.774244    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-a5726951\"/\"kube-root-ca.crt\""
Jul 24 17:43:00.775444 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:00.775408    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-a5726951\"/\"default-dockercfg-hgmf5\""
Jul 24 17:43:00.775444 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:00.775434    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-a5726951\"/\"e2e-lifecycle-model-a5726951-kserve-self-signed-certs\""
Jul 24 17:43:00.781909 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:00.781886    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-a5726951/e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr"]
Jul 24 17:43:00.789864 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:00.789841    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:43:00.879358 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:00.879308    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a319ae66-cafa-4ff4-810f-f453730865dc-tmp-dir\") pod \"e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr\" (UID: \"a319ae66-cafa-4ff4-810f-f453730865dc\") " pod="ai-tenant-e2e-mt-a5726951/e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr"
Jul 24 17:43:00.879592 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:00.879389    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a319ae66-cafa-4ff4-810f-f453730865dc-home\") pod \"e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr\" (UID: \"a319ae66-cafa-4ff4-810f-f453730865dc\") " pod="ai-tenant-e2e-mt-a5726951/e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr"
Jul 24 17:43:00.879592 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:00.879480    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a319ae66-cafa-4ff4-810f-f453730865dc-model-cache\") pod \"e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr\" (UID: \"a319ae66-cafa-4ff4-810f-f453730865dc\") " pod="ai-tenant-e2e-mt-a5726951/e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr"
Jul 24 17:43:00.879592 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:00.879522    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a319ae66-cafa-4ff4-810f-f453730865dc-dshm\") pod \"e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr\" (UID: \"a319ae66-cafa-4ff4-810f-f453730865dc\") " pod="ai-tenant-e2e-mt-a5726951/e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr"
Jul 24 17:43:00.879716 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:00.879590    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsf28\" (UniqueName: \"kubernetes.io/projected/a319ae66-cafa-4ff4-810f-f453730865dc-kube-api-access-wsf28\") pod \"e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr\" (UID: \"a319ae66-cafa-4ff4-810f-f453730865dc\") " pod="ai-tenant-e2e-mt-a5726951/e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr"
Jul 24 17:43:00.879716 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:00.879625    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a319ae66-cafa-4ff4-810f-f453730865dc-tls-certs\") pod \"e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr\" (UID: \"a319ae66-cafa-4ff4-810f-f453730865dc\") " pod="ai-tenant-e2e-mt-a5726951/e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr"
Jul 24 17:43:00.980923 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:00.980872    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsf28\" (UniqueName: \"kubernetes.io/projected/a319ae66-cafa-4ff4-810f-f453730865dc-kube-api-access-wsf28\") pod \"e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr\" (UID: \"a319ae66-cafa-4ff4-810f-f453730865dc\") " pod="ai-tenant-e2e-mt-a5726951/e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr"
Jul 24 17:43:00.980923 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:00.980931    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a319ae66-cafa-4ff4-810f-f453730865dc-tls-certs\") pod \"e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr\" (UID: \"a319ae66-cafa-4ff4-810f-f453730865dc\") " pod="ai-tenant-e2e-mt-a5726951/e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr"
Jul 24 17:43:00.981226 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:00.980972    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a319ae66-cafa-4ff4-810f-f453730865dc-tmp-dir\") pod \"e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr\" (UID: \"a319ae66-cafa-4ff4-810f-f453730865dc\") " pod="ai-tenant-e2e-mt-a5726951/e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr"
Jul 24 17:43:00.981226 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:00.981020    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a319ae66-cafa-4ff4-810f-f453730865dc-home\") pod \"e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr\" (UID: \"a319ae66-cafa-4ff4-810f-f453730865dc\") " pod="ai-tenant-e2e-mt-a5726951/e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr"
Jul 24 17:43:00.981226 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:00.981093    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a319ae66-cafa-4ff4-810f-f453730865dc-model-cache\") pod \"e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr\" (UID: \"a319ae66-cafa-4ff4-810f-f453730865dc\") " pod="ai-tenant-e2e-mt-a5726951/e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr"
Jul 24 17:43:00.981226 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:00.981117    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a319ae66-cafa-4ff4-810f-f453730865dc-dshm\") pod \"e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr\" (UID: \"a319ae66-cafa-4ff4-810f-f453730865dc\") " pod="ai-tenant-e2e-mt-a5726951/e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr"
Jul 24 17:43:00.981523 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:00.981488    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a319ae66-cafa-4ff4-810f-f453730865dc-tmp-dir\") pod \"e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr\" (UID: \"a319ae66-cafa-4ff4-810f-f453730865dc\") " pod="ai-tenant-e2e-mt-a5726951/e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr"
Jul 24 17:43:00.981642 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:00.981529    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a319ae66-cafa-4ff4-810f-f453730865dc-model-cache\") pod \"e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr\" (UID: \"a319ae66-cafa-4ff4-810f-f453730865dc\") " pod="ai-tenant-e2e-mt-a5726951/e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr"
Jul 24 17:43:00.981642 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:00.981558    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a319ae66-cafa-4ff4-810f-f453730865dc-home\") pod \"e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr\" (UID: \"a319ae66-cafa-4ff4-810f-f453730865dc\") " pod="ai-tenant-e2e-mt-a5726951/e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr"
Jul 24 17:43:00.983510 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:00.983487    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a319ae66-cafa-4ff4-810f-f453730865dc-dshm\") pod \"e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr\" (UID: \"a319ae66-cafa-4ff4-810f-f453730865dc\") " pod="ai-tenant-e2e-mt-a5726951/e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr"
Jul 24 17:43:00.983771 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:00.983755    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a319ae66-cafa-4ff4-810f-f453730865dc-tls-certs\") pod \"e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr\" (UID: \"a319ae66-cafa-4ff4-810f-f453730865dc\") " pod="ai-tenant-e2e-mt-a5726951/e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr"
Jul 24 17:43:00.989874 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:00.989854    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsf28\" (UniqueName: \"kubernetes.io/projected/a319ae66-cafa-4ff4-810f-f453730865dc-kube-api-access-wsf28\") pod \"e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr\" (UID: \"a319ae66-cafa-4ff4-810f-f453730865dc\") " pod="ai-tenant-e2e-mt-a5726951/e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr"
Jul 24 17:43:01.081023 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:01.080904    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-a5726951/e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr"
Jul 24 17:43:01.213852 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:01.213827    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-a5726951/e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr"]
Jul 24 17:43:01.215657 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:43:01.215621    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda319ae66_cafa_4ff4_810f_f453730865dc.slice/crio-1d8c7bba3b1aa1eec80653c604b3ace2c9ff2f6c2e74ae13350c96e281618054 WatchSource:0}: Error finding container 1d8c7bba3b1aa1eec80653c604b3ace2c9ff2f6c2e74ae13350c96e281618054: Status 404 returned error can't find the container with id 1d8c7bba3b1aa1eec80653c604b3ace2c9ff2f6c2e74ae13350c96e281618054
Jul 24 17:43:01.969590 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:01.969553    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-a5726951/e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr" event={"ID":"a319ae66-cafa-4ff4-810f-f453730865dc","Type":"ContainerStarted","Data":"101a728dc6c9670b12563e9f9c38ecd455d09869f2dbc363b0f1fd40a6aabda8"}
Jul 24 17:43:01.969590 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:01.969592    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-a5726951/e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr" event={"ID":"a319ae66-cafa-4ff4-810f-f453730865dc","Type":"ContainerStarted","Data":"1d8c7bba3b1aa1eec80653c604b3ace2c9ff2f6c2e74ae13350c96e281618054"}
Jul 24 17:43:01.987203 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:01.987149    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-a5726951/e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr" podStartSLOduration=1.987127726 podStartE2EDuration="1.987127726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:43:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:43:01.985999983 +0000 UTC m=+1949.188049282" watchObservedRunningTime="2026-07-24 17:43:01.987127726 +0000 UTC m=+1949.189177003"
Jul 24 17:43:04.349535 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:04.349499    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:43:11.081116 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:11.081073    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-a5726951/e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr"
Jul 24 17:43:11.081532 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:11.081136    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-a5726951/e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr"
Jul 24 17:43:11.094226 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:11.094196    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-a5726951/e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr"
Jul 24 17:43:12.030286 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:12.030253    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-a5726951/e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr"
Jul 24 17:43:20.263920 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:20.263887    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:43:38.563676 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:38.563600    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:43:39.997211 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:39.997178    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:43:40.097475 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:40.097443    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:43:40.164198 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:40.164166    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:43:44.587206 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:44.587171    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:43:47.458308 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:47.458276    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:43:47.864597 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:47.864553    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:43:52.781619 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:52.781582    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-a5726951/e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr"]
Jul 24 17:43:52.836333 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:43:52.836293    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:44:07.801669 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:07.801622    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-a5726951/e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr" podUID="a319ae66-cafa-4ff4-810f-f453730865dc" containerName="main" containerID="cri-o://101a728dc6c9670b12563e9f9c38ecd455d09869f2dbc363b0f1fd40a6aabda8" gracePeriod=45
Jul 24 17:44:08.057055 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:08.056983    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-a5726951/e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr"
Jul 24 17:44:08.128882 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:08.128853    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wsf28\" (UniqueName: \"kubernetes.io/projected/a319ae66-cafa-4ff4-810f-f453730865dc-kube-api-access-wsf28\") pod \"a319ae66-cafa-4ff4-810f-f453730865dc\" (UID: \"a319ae66-cafa-4ff4-810f-f453730865dc\") "
Jul 24 17:44:08.129087 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:08.128898    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a319ae66-cafa-4ff4-810f-f453730865dc-dshm\") pod \"a319ae66-cafa-4ff4-810f-f453730865dc\" (UID: \"a319ae66-cafa-4ff4-810f-f453730865dc\") "
Jul 24 17:44:08.129087 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:08.128944    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a319ae66-cafa-4ff4-810f-f453730865dc-tmp-dir\") pod \"a319ae66-cafa-4ff4-810f-f453730865dc\" (UID: \"a319ae66-cafa-4ff4-810f-f453730865dc\") "
Jul 24 17:44:08.129087 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:08.128966    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a319ae66-cafa-4ff4-810f-f453730865dc-tls-certs\") pod \"a319ae66-cafa-4ff4-810f-f453730865dc\" (UID: \"a319ae66-cafa-4ff4-810f-f453730865dc\") "
Jul 24 17:44:08.129087 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:08.129072    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a319ae66-cafa-4ff4-810f-f453730865dc-model-cache\") pod \"a319ae66-cafa-4ff4-810f-f453730865dc\" (UID: \"a319ae66-cafa-4ff4-810f-f453730865dc\") "
Jul 24 17:44:08.129314 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:08.129142    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a319ae66-cafa-4ff4-810f-f453730865dc-home\") pod \"a319ae66-cafa-4ff4-810f-f453730865dc\" (UID: \"a319ae66-cafa-4ff4-810f-f453730865dc\") "
Jul 24 17:44:08.129314 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:08.129227    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a319ae66-cafa-4ff4-810f-f453730865dc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a319ae66-cafa-4ff4-810f-f453730865dc" (UID: "a319ae66-cafa-4ff4-810f-f453730865dc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:44:08.129434 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:08.129338    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a319ae66-cafa-4ff4-810f-f453730865dc-model-cache" (OuterVolumeSpecName: "model-cache") pod "a319ae66-cafa-4ff4-810f-f453730865dc" (UID: "a319ae66-cafa-4ff4-810f-f453730865dc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:44:08.129474 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:08.129426    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a319ae66-cafa-4ff4-810f-f453730865dc-home" (OuterVolumeSpecName: "home") pod "a319ae66-cafa-4ff4-810f-f453730865dc" (UID: "a319ae66-cafa-4ff4-810f-f453730865dc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:44:08.129766 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:08.129746    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a319ae66-cafa-4ff4-810f-f453730865dc-tmp-dir\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:44:08.129839 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:08.129772    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a319ae66-cafa-4ff4-810f-f453730865dc-model-cache\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:44:08.129839 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:08.129787    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a319ae66-cafa-4ff4-810f-f453730865dc-home\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:44:08.131268 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:08.131232    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a319ae66-cafa-4ff4-810f-f453730865dc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a319ae66-cafa-4ff4-810f-f453730865dc" (UID: "a319ae66-cafa-4ff4-810f-f453730865dc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:44:08.131391 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:08.131297    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a319ae66-cafa-4ff4-810f-f453730865dc-dshm" (OuterVolumeSpecName: "dshm") pod "a319ae66-cafa-4ff4-810f-f453730865dc" (UID: "a319ae66-cafa-4ff4-810f-f453730865dc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:44:08.131391 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:08.131316    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a319ae66-cafa-4ff4-810f-f453730865dc-kube-api-access-wsf28" (OuterVolumeSpecName: "kube-api-access-wsf28") pod "a319ae66-cafa-4ff4-810f-f453730865dc" (UID: "a319ae66-cafa-4ff4-810f-f453730865dc"). InnerVolumeSpecName "kube-api-access-wsf28". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:44:08.230479 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:08.230431    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsf28\" (UniqueName: \"kubernetes.io/projected/a319ae66-cafa-4ff4-810f-f453730865dc-kube-api-access-wsf28\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:44:08.230479 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:08.230474    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a319ae66-cafa-4ff4-810f-f453730865dc-dshm\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:44:08.230479 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:08.230483    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a319ae66-cafa-4ff4-810f-f453730865dc-tls-certs\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:44:08.299774 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:08.299740    2581 generic.go:356] "Generic (PLEG): container finished" podID="a319ae66-cafa-4ff4-810f-f453730865dc" containerID="101a728dc6c9670b12563e9f9c38ecd455d09869f2dbc363b0f1fd40a6aabda8" exitCode=0
Jul 24 17:44:08.299946 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:08.299784    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-a5726951/e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr"
Jul 24 17:44:08.299946 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:08.299783    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-a5726951/e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr" event={"ID":"a319ae66-cafa-4ff4-810f-f453730865dc","Type":"ContainerDied","Data":"101a728dc6c9670b12563e9f9c38ecd455d09869f2dbc363b0f1fd40a6aabda8"}
Jul 24 17:44:08.299946 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:08.299817    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-a5726951/e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr" event={"ID":"a319ae66-cafa-4ff4-810f-f453730865dc","Type":"ContainerDied","Data":"1d8c7bba3b1aa1eec80653c604b3ace2c9ff2f6c2e74ae13350c96e281618054"}
Jul 24 17:44:08.299946 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:08.299837    2581 scope.go:117] "RemoveContainer" containerID="101a728dc6c9670b12563e9f9c38ecd455d09869f2dbc363b0f1fd40a6aabda8"
Jul 24 17:44:08.309362 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:08.309344    2581 scope.go:117] "RemoveContainer" containerID="101a728dc6c9670b12563e9f9c38ecd455d09869f2dbc363b0f1fd40a6aabda8"
Jul 24 17:44:08.309622 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:44:08.309598    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"101a728dc6c9670b12563e9f9c38ecd455d09869f2dbc363b0f1fd40a6aabda8\": container with ID starting with 101a728dc6c9670b12563e9f9c38ecd455d09869f2dbc363b0f1fd40a6aabda8 not found: ID does not exist" containerID="101a728dc6c9670b12563e9f9c38ecd455d09869f2dbc363b0f1fd40a6aabda8"
Jul 24 17:44:08.309695 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:08.309631    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"101a728dc6c9670b12563e9f9c38ecd455d09869f2dbc363b0f1fd40a6aabda8"} err="failed to get container status \"101a728dc6c9670b12563e9f9c38ecd455d09869f2dbc363b0f1fd40a6aabda8\": rpc error: code = NotFound desc = could not find container \"101a728dc6c9670b12563e9f9c38ecd455d09869f2dbc363b0f1fd40a6aabda8\": container with ID starting with 101a728dc6c9670b12563e9f9c38ecd455d09869f2dbc363b0f1fd40a6aabda8 not found: ID does not exist"
Jul 24 17:44:08.321111 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:08.321090    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-a5726951/e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr"]
Jul 24 17:44:08.330267 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:08.326503    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-a5726951/e2e-lifecycle-model-a5726951-kserve-85dd8dd4c5-vrdrr"]
Jul 24 17:44:09.385153 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:09.385118    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a319ae66-cafa-4ff4-810f-f453730865dc" path="/var/lib/kubelet/pods/a319ae66-cafa-4ff4-810f-f453730865dc/volumes"
Jul 24 17:44:29.033857 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:29.033820    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:44:29.873249 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:29.873218    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:44:35.328279 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:35.328238    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:44:35.432241 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:35.432202    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:44:35.613278 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:35.613184    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:44:35.858699 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:35.858649    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:44:36.940587 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:36.940537    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:44:37.041626 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:37.041581    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:44:37.146432 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:37.146396    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:44:37.856343 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:37.856309    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:44:37.952028 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:37.951991    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:44:38.052880 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:38.052849    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:44:39.065319 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:39.065287    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:44:39.422152 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:39.422112    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:44:45.207118 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:45.207083    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:44:45.284719 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:45.284677    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:44:45.869836 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:45.869803    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:44:51.611995 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:51.611958    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:44:51.776351 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:51.776318    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:44:52.999772 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:44:52.999739    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:45:02.300698 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:02.300661    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:45:02.648295 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:02.648259    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:45:23.607417 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:23.607383    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:45:30.294537 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:30.294503    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:45:30.361017 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:30.360973    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:45:30.631670 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:30.631584    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:45:30.931830 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:30.931793    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:45:31.136189 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:31.136137    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:45:34.294573 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:34.294524    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:45:34.416480 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:34.416449    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:45:34.537132 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:34.537100    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:45:34.962086 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:34.962029    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:45:35.060399 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:35.060362    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:45:35.142204 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:35.142163    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:45:38.789829 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:38.789789    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:45:39.130276 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:39.130167    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:45:39.414550 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:39.414504    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:45:39.684984 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:39.684879    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:45:39.756973 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:39.756936    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:45:42.312226 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:42.312189    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:45:42.388860 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:42.388830    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:45:42.459318 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:42.459275    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:45:42.621445 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:42.621340    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:45:42.906981 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:42.906952    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:45:43.006889 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:43.006856    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:45:46.824960 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:46.824920    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:45:46.931772 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:46.931741    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:45:46.991553 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:46.991500    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:45:52.154584 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:52.154540    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:45:56.557241 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:56.557207    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-387e6e/test-model-387e6e-kserve-668ccd9fb9-x76bx"]
Jul 24 17:45:56.557772 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:56.557751    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a319ae66-cafa-4ff4-810f-f453730865dc" containerName="main"
Jul 24 17:45:56.557772 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:56.557775    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a319ae66-cafa-4ff4-810f-f453730865dc" containerName="main"
Jul 24 17:45:56.557891 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:56.557849    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a319ae66-cafa-4ff4-810f-f453730865dc" containerName="main"
Jul 24 17:45:56.561005 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:56.560990    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-387e6e/test-model-387e6e-kserve-668ccd9fb9-x76bx"
Jul 24 17:45:56.565205 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:56.565176    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-387e6e\"/\"test-model-387e6e-kserve-self-signed-certs\""
Jul 24 17:45:56.565330 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:56.565208    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-387e6e\"/\"kube-root-ca.crt\""
Jul 24 17:45:56.565330 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:56.565255    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-387e6e\"/\"default-dockercfg-rfdwb\""
Jul 24 17:45:56.565330 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:56.565275    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-387e6e\"/\"openshift-service-ca.crt\""
Jul 24 17:45:56.572579 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:56.572560    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-387e6e/test-model-387e6e-kserve-668ccd9fb9-x76bx"]
Jul 24 17:45:56.586341 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:56.586311    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:45:56.653667 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:56.653639    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xksjq\" (UniqueName: \"kubernetes.io/projected/306628fc-85be-4c4b-98dd-25cfdc8371a7-kube-api-access-xksjq\") pod \"test-model-387e6e-kserve-668ccd9fb9-x76bx\" (UID: \"306628fc-85be-4c4b-98dd-25cfdc8371a7\") " pod="ai-tenant-e2e-inf-a-387e6e/test-model-387e6e-kserve-668ccd9fb9-x76bx"
Jul 24 17:45:56.653826 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:56.653687    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/306628fc-85be-4c4b-98dd-25cfdc8371a7-home\") pod \"test-model-387e6e-kserve-668ccd9fb9-x76bx\" (UID: \"306628fc-85be-4c4b-98dd-25cfdc8371a7\") " pod="ai-tenant-e2e-inf-a-387e6e/test-model-387e6e-kserve-668ccd9fb9-x76bx"
Jul 24 17:45:56.653826 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:56.653704    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/306628fc-85be-4c4b-98dd-25cfdc8371a7-tls-certs\") pod \"test-model-387e6e-kserve-668ccd9fb9-x76bx\" (UID: \"306628fc-85be-4c4b-98dd-25cfdc8371a7\") " pod="ai-tenant-e2e-inf-a-387e6e/test-model-387e6e-kserve-668ccd9fb9-x76bx"
Jul 24 17:45:56.653826 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:56.653791    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/306628fc-85be-4c4b-98dd-25cfdc8371a7-tmp-dir\") pod \"test-model-387e6e-kserve-668ccd9fb9-x76bx\" (UID: \"306628fc-85be-4c4b-98dd-25cfdc8371a7\") " pod="ai-tenant-e2e-inf-a-387e6e/test-model-387e6e-kserve-668ccd9fb9-x76bx"
Jul 24 17:45:56.653935 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:56.653855    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/306628fc-85be-4c4b-98dd-25cfdc8371a7-model-cache\") pod \"test-model-387e6e-kserve-668ccd9fb9-x76bx\" (UID: \"306628fc-85be-4c4b-98dd-25cfdc8371a7\") " pod="ai-tenant-e2e-inf-a-387e6e/test-model-387e6e-kserve-668ccd9fb9-x76bx"
Jul 24 17:45:56.653935 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:56.653913    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/306628fc-85be-4c4b-98dd-25cfdc8371a7-dshm\") pod \"test-model-387e6e-kserve-668ccd9fb9-x76bx\" (UID: \"306628fc-85be-4c4b-98dd-25cfdc8371a7\") " pod="ai-tenant-e2e-inf-a-387e6e/test-model-387e6e-kserve-668ccd9fb9-x76bx"
Jul 24 17:45:56.755188 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:56.755151    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/306628fc-85be-4c4b-98dd-25cfdc8371a7-model-cache\") pod \"test-model-387e6e-kserve-668ccd9fb9-x76bx\" (UID: \"306628fc-85be-4c4b-98dd-25cfdc8371a7\") " pod="ai-tenant-e2e-inf-a-387e6e/test-model-387e6e-kserve-668ccd9fb9-x76bx"
Jul 24 17:45:56.755188 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:56.755194    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/306628fc-85be-4c4b-98dd-25cfdc8371a7-dshm\") pod \"test-model-387e6e-kserve-668ccd9fb9-x76bx\" (UID: \"306628fc-85be-4c4b-98dd-25cfdc8371a7\") " pod="ai-tenant-e2e-inf-a-387e6e/test-model-387e6e-kserve-668ccd9fb9-x76bx"
Jul 24 17:45:56.755455 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:56.755218    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xksjq\" (UniqueName: \"kubernetes.io/projected/306628fc-85be-4c4b-98dd-25cfdc8371a7-kube-api-access-xksjq\") pod \"test-model-387e6e-kserve-668ccd9fb9-x76bx\" (UID: \"306628fc-85be-4c4b-98dd-25cfdc8371a7\") " pod="ai-tenant-e2e-inf-a-387e6e/test-model-387e6e-kserve-668ccd9fb9-x76bx"
Jul 24 17:45:56.755455 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:56.755255    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/306628fc-85be-4c4b-98dd-25cfdc8371a7-home\") pod \"test-model-387e6e-kserve-668ccd9fb9-x76bx\" (UID: \"306628fc-85be-4c4b-98dd-25cfdc8371a7\") " pod="ai-tenant-e2e-inf-a-387e6e/test-model-387e6e-kserve-668ccd9fb9-x76bx"
Jul 24 17:45:56.755455 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:56.755272    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/306628fc-85be-4c4b-98dd-25cfdc8371a7-tls-certs\") pod \"test-model-387e6e-kserve-668ccd9fb9-x76bx\" (UID: \"306628fc-85be-4c4b-98dd-25cfdc8371a7\") " pod="ai-tenant-e2e-inf-a-387e6e/test-model-387e6e-kserve-668ccd9fb9-x76bx"
Jul 24 17:45:56.755455 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:56.755313    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/306628fc-85be-4c4b-98dd-25cfdc8371a7-tmp-dir\") pod \"test-model-387e6e-kserve-668ccd9fb9-x76bx\" (UID: \"306628fc-85be-4c4b-98dd-25cfdc8371a7\") " pod="ai-tenant-e2e-inf-a-387e6e/test-model-387e6e-kserve-668ccd9fb9-x76bx"
Jul 24 17:45:56.755667 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:56.755619    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/306628fc-85be-4c4b-98dd-25cfdc8371a7-model-cache\") pod \"test-model-387e6e-kserve-668ccd9fb9-x76bx\" (UID: \"306628fc-85be-4c4b-98dd-25cfdc8371a7\") " pod="ai-tenant-e2e-inf-a-387e6e/test-model-387e6e-kserve-668ccd9fb9-x76bx"
Jul 24 17:45:56.755723 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:56.755704    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/306628fc-85be-4c4b-98dd-25cfdc8371a7-home\") pod \"test-model-387e6e-kserve-668ccd9fb9-x76bx\" (UID: \"306628fc-85be-4c4b-98dd-25cfdc8371a7\") " pod="ai-tenant-e2e-inf-a-387e6e/test-model-387e6e-kserve-668ccd9fb9-x76bx"
Jul 24 17:45:56.755762 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:56.755729    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/306628fc-85be-4c4b-98dd-25cfdc8371a7-tmp-dir\") pod \"test-model-387e6e-kserve-668ccd9fb9-x76bx\" (UID: \"306628fc-85be-4c4b-98dd-25cfdc8371a7\") " pod="ai-tenant-e2e-inf-a-387e6e/test-model-387e6e-kserve-668ccd9fb9-x76bx"
Jul 24 17:45:56.757647 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:56.757625    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/306628fc-85be-4c4b-98dd-25cfdc8371a7-dshm\") pod \"test-model-387e6e-kserve-668ccd9fb9-x76bx\" (UID: \"306628fc-85be-4c4b-98dd-25cfdc8371a7\") " pod="ai-tenant-e2e-inf-a-387e6e/test-model-387e6e-kserve-668ccd9fb9-x76bx"
Jul 24 17:45:56.757906 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:56.757889    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/306628fc-85be-4c4b-98dd-25cfdc8371a7-tls-certs\") pod \"test-model-387e6e-kserve-668ccd9fb9-x76bx\" (UID: \"306628fc-85be-4c4b-98dd-25cfdc8371a7\") " pod="ai-tenant-e2e-inf-a-387e6e/test-model-387e6e-kserve-668ccd9fb9-x76bx"
Jul 24 17:45:56.763707 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:56.763689    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xksjq\" (UniqueName: \"kubernetes.io/projected/306628fc-85be-4c4b-98dd-25cfdc8371a7-kube-api-access-xksjq\") pod \"test-model-387e6e-kserve-668ccd9fb9-x76bx\" (UID: \"306628fc-85be-4c4b-98dd-25cfdc8371a7\") " pod="ai-tenant-e2e-inf-a-387e6e/test-model-387e6e-kserve-668ccd9fb9-x76bx"
Jul 24 17:45:56.870278 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:56.870181    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-387e6e/test-model-387e6e-kserve-668ccd9fb9-x76bx"
Jul 24 17:45:57.002487 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:57.002453    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-387e6e/test-model-387e6e-kserve-668ccd9fb9-x76bx"]
Jul 24 17:45:57.004495 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:45:57.004462    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod306628fc_85be_4c4b_98dd_25cfdc8371a7.slice/crio-b2b9089200006ab4d5ad338d446d705af25fdee41e81bdcd42ab2dbb53cf7fc8 WatchSource:0}: Error finding container b2b9089200006ab4d5ad338d446d705af25fdee41e81bdcd42ab2dbb53cf7fc8: Status 404 returned error can't find the container with id b2b9089200006ab4d5ad338d446d705af25fdee41e81bdcd42ab2dbb53cf7fc8
Jul 24 17:45:57.822030 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:57.821993    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-387e6e/test-model-387e6e-kserve-668ccd9fb9-x76bx" event={"ID":"306628fc-85be-4c4b-98dd-25cfdc8371a7","Type":"ContainerStarted","Data":"b4e5a6a6713e170246006981da355cc38c13b9168be9f3ac69afdbdce6d9ac9c"}
Jul 24 17:45:57.822030 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:57.822048    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-387e6e/test-model-387e6e-kserve-668ccd9fb9-x76bx" event={"ID":"306628fc-85be-4c4b-98dd-25cfdc8371a7","Type":"ContainerStarted","Data":"b2b9089200006ab4d5ad338d446d705af25fdee41e81bdcd42ab2dbb53cf7fc8"}
Jul 24 17:45:57.837793 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:45:57.837746    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-387e6e/test-model-387e6e-kserve-668ccd9fb9-x76bx" podStartSLOduration=1.837727165 podStartE2EDuration="1.837727165s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:45:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:45:57.837232624 +0000 UTC m=+2125.039281898" watchObservedRunningTime="2026-07-24 17:45:57.837727165 +0000 UTC m=+2125.039776440"
Jul 24 17:46:01.350531 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:01.350485    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:46:06.870390 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:06.870353    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-387e6e/test-model-387e6e-kserve-668ccd9fb9-x76bx"
Jul 24 17:46:06.870898 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:06.870434    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-387e6e/test-model-387e6e-kserve-668ccd9fb9-x76bx"
Jul 24 17:46:06.882716 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:06.882690    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-387e6e/test-model-387e6e-kserve-668ccd9fb9-x76bx"
Jul 24 17:46:07.880630 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:07.880597    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-387e6e/test-model-387e6e-kserve-668ccd9fb9-x76bx"
Jul 24 17:46:35.264551 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:35.264462    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-22bf5f/test-model-22bf5f-kserve-6ff4f45f5-dzd97"]
Jul 24 17:46:35.269787 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:35.269761    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-22bf5f/test-model-22bf5f-kserve-6ff4f45f5-dzd97"
Jul 24 17:46:35.272715 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:35.272690    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-22bf5f\"/\"kube-root-ca.crt\""
Jul 24 17:46:35.272889 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:35.272719    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-22bf5f\"/\"openshift-service-ca.crt\""
Jul 24 17:46:35.273709 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:35.273682    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-22bf5f/test-model-22bf5f-kserve-6ff4f45f5-dzd97"]
Jul 24 17:46:35.274175 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:35.274154    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-22bf5f\"/\"default-dockercfg-4lzd2\""
Jul 24 17:46:35.274304 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:35.274181    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-22bf5f\"/\"test-model-22bf5f-kserve-self-signed-certs\""
Jul 24 17:46:35.280009 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:35.279981    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:46:35.331306 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:35.331276    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67daf14a-ba12-464a-acfb-9f4640f382f3-tmp-dir\") pod \"test-model-22bf5f-kserve-6ff4f45f5-dzd97\" (UID: \"67daf14a-ba12-464a-acfb-9f4640f382f3\") " pod="ai-tenant-e2e-inf-b-22bf5f/test-model-22bf5f-kserve-6ff4f45f5-dzd97"
Jul 24 17:46:35.331474 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:35.331326    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67daf14a-ba12-464a-acfb-9f4640f382f3-dshm\") pod \"test-model-22bf5f-kserve-6ff4f45f5-dzd97\" (UID: \"67daf14a-ba12-464a-acfb-9f4640f382f3\") " pod="ai-tenant-e2e-inf-b-22bf5f/test-model-22bf5f-kserve-6ff4f45f5-dzd97"
Jul 24 17:46:35.331474 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:35.331345    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67daf14a-ba12-464a-acfb-9f4640f382f3-model-cache\") pod \"test-model-22bf5f-kserve-6ff4f45f5-dzd97\" (UID: \"67daf14a-ba12-464a-acfb-9f4640f382f3\") " pod="ai-tenant-e2e-inf-b-22bf5f/test-model-22bf5f-kserve-6ff4f45f5-dzd97"
Jul 24 17:46:35.331474 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:35.331425    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67daf14a-ba12-464a-acfb-9f4640f382f3-home\") pod \"test-model-22bf5f-kserve-6ff4f45f5-dzd97\" (UID: \"67daf14a-ba12-464a-acfb-9f4640f382f3\") " pod="ai-tenant-e2e-inf-b-22bf5f/test-model-22bf5f-kserve-6ff4f45f5-dzd97"
Jul 24 17:46:35.331474 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:35.331468    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdkpl\" (UniqueName: \"kubernetes.io/projected/67daf14a-ba12-464a-acfb-9f4640f382f3-kube-api-access-mdkpl\") pod \"test-model-22bf5f-kserve-6ff4f45f5-dzd97\" (UID: \"67daf14a-ba12-464a-acfb-9f4640f382f3\") " pod="ai-tenant-e2e-inf-b-22bf5f/test-model-22bf5f-kserve-6ff4f45f5-dzd97"
Jul 24 17:46:35.331622 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:35.331573    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67daf14a-ba12-464a-acfb-9f4640f382f3-tls-certs\") pod \"test-model-22bf5f-kserve-6ff4f45f5-dzd97\" (UID: \"67daf14a-ba12-464a-acfb-9f4640f382f3\") " pod="ai-tenant-e2e-inf-b-22bf5f/test-model-22bf5f-kserve-6ff4f45f5-dzd97"
Jul 24 17:46:35.432701 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:35.432666    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67daf14a-ba12-464a-acfb-9f4640f382f3-dshm\") pod \"test-model-22bf5f-kserve-6ff4f45f5-dzd97\" (UID: \"67daf14a-ba12-464a-acfb-9f4640f382f3\") " pod="ai-tenant-e2e-inf-b-22bf5f/test-model-22bf5f-kserve-6ff4f45f5-dzd97"
Jul 24 17:46:35.432701 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:35.432698    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67daf14a-ba12-464a-acfb-9f4640f382f3-model-cache\") pod \"test-model-22bf5f-kserve-6ff4f45f5-dzd97\" (UID: \"67daf14a-ba12-464a-acfb-9f4640f382f3\") " pod="ai-tenant-e2e-inf-b-22bf5f/test-model-22bf5f-kserve-6ff4f45f5-dzd97"
Jul 24 17:46:35.432921 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:35.432746    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67daf14a-ba12-464a-acfb-9f4640f382f3-home\") pod \"test-model-22bf5f-kserve-6ff4f45f5-dzd97\" (UID: \"67daf14a-ba12-464a-acfb-9f4640f382f3\") " pod="ai-tenant-e2e-inf-b-22bf5f/test-model-22bf5f-kserve-6ff4f45f5-dzd97"
Jul 24 17:46:35.432921 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:35.432784    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdkpl\" (UniqueName: \"kubernetes.io/projected/67daf14a-ba12-464a-acfb-9f4640f382f3-kube-api-access-mdkpl\") pod \"test-model-22bf5f-kserve-6ff4f45f5-dzd97\" (UID: \"67daf14a-ba12-464a-acfb-9f4640f382f3\") " pod="ai-tenant-e2e-inf-b-22bf5f/test-model-22bf5f-kserve-6ff4f45f5-dzd97"
Jul 24 17:46:35.432921 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:35.432866    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67daf14a-ba12-464a-acfb-9f4640f382f3-tls-certs\") pod \"test-model-22bf5f-kserve-6ff4f45f5-dzd97\" (UID: \"67daf14a-ba12-464a-acfb-9f4640f382f3\") " pod="ai-tenant-e2e-inf-b-22bf5f/test-model-22bf5f-kserve-6ff4f45f5-dzd97"
Jul 24 17:46:35.432921 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:35.432913    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67daf14a-ba12-464a-acfb-9f4640f382f3-tmp-dir\") pod \"test-model-22bf5f-kserve-6ff4f45f5-dzd97\" (UID: \"67daf14a-ba12-464a-acfb-9f4640f382f3\") " pod="ai-tenant-e2e-inf-b-22bf5f/test-model-22bf5f-kserve-6ff4f45f5-dzd97"
Jul 24 17:46:35.433184 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:35.433158    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67daf14a-ba12-464a-acfb-9f4640f382f3-model-cache\") pod \"test-model-22bf5f-kserve-6ff4f45f5-dzd97\" (UID: \"67daf14a-ba12-464a-acfb-9f4640f382f3\") " pod="ai-tenant-e2e-inf-b-22bf5f/test-model-22bf5f-kserve-6ff4f45f5-dzd97"
Jul 24 17:46:35.433260 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:35.433218    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67daf14a-ba12-464a-acfb-9f4640f382f3-home\") pod \"test-model-22bf5f-kserve-6ff4f45f5-dzd97\" (UID: \"67daf14a-ba12-464a-acfb-9f4640f382f3\") " pod="ai-tenant-e2e-inf-b-22bf5f/test-model-22bf5f-kserve-6ff4f45f5-dzd97"
Jul 24 17:46:35.433339 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:35.433319    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67daf14a-ba12-464a-acfb-9f4640f382f3-tmp-dir\") pod \"test-model-22bf5f-kserve-6ff4f45f5-dzd97\" (UID: \"67daf14a-ba12-464a-acfb-9f4640f382f3\") " pod="ai-tenant-e2e-inf-b-22bf5f/test-model-22bf5f-kserve-6ff4f45f5-dzd97"
Jul 24 17:46:35.435243 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:35.435226    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67daf14a-ba12-464a-acfb-9f4640f382f3-dshm\") pod \"test-model-22bf5f-kserve-6ff4f45f5-dzd97\" (UID: \"67daf14a-ba12-464a-acfb-9f4640f382f3\") " pod="ai-tenant-e2e-inf-b-22bf5f/test-model-22bf5f-kserve-6ff4f45f5-dzd97"
Jul 24 17:46:35.435486 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:35.435468    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67daf14a-ba12-464a-acfb-9f4640f382f3-tls-certs\") pod \"test-model-22bf5f-kserve-6ff4f45f5-dzd97\" (UID: \"67daf14a-ba12-464a-acfb-9f4640f382f3\") " pod="ai-tenant-e2e-inf-b-22bf5f/test-model-22bf5f-kserve-6ff4f45f5-dzd97"
Jul 24 17:46:35.440914 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:35.440897    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdkpl\" (UniqueName: \"kubernetes.io/projected/67daf14a-ba12-464a-acfb-9f4640f382f3-kube-api-access-mdkpl\") pod \"test-model-22bf5f-kserve-6ff4f45f5-dzd97\" (UID: \"67daf14a-ba12-464a-acfb-9f4640f382f3\") " pod="ai-tenant-e2e-inf-b-22bf5f/test-model-22bf5f-kserve-6ff4f45f5-dzd97"
Jul 24 17:46:35.581823 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:35.581722    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-22bf5f/test-model-22bf5f-kserve-6ff4f45f5-dzd97"
Jul 24 17:46:35.719652 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:35.719625    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-22bf5f/test-model-22bf5f-kserve-6ff4f45f5-dzd97"]
Jul 24 17:46:35.721500 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:46:35.721478    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod67daf14a_ba12_464a_acfb_9f4640f382f3.slice/crio-aae72e031bb403ed11caf16f720fa9cfa2cec2077a1f721d86b9d0698e909c80 WatchSource:0}: Error finding container aae72e031bb403ed11caf16f720fa9cfa2cec2077a1f721d86b9d0698e909c80: Status 404 returned error can't find the container with id aae72e031bb403ed11caf16f720fa9cfa2cec2077a1f721d86b9d0698e909c80
Jul 24 17:46:36.009451 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:36.009413    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-22bf5f/test-model-22bf5f-kserve-6ff4f45f5-dzd97" event={"ID":"67daf14a-ba12-464a-acfb-9f4640f382f3","Type":"ContainerStarted","Data":"85b18dc06b3ee70e44a111d324109246e01fdc867ab24447ec075651cc63fa87"}
Jul 24 17:46:36.009451 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:36.009455    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-22bf5f/test-model-22bf5f-kserve-6ff4f45f5-dzd97" event={"ID":"67daf14a-ba12-464a-acfb-9f4640f382f3","Type":"ContainerStarted","Data":"aae72e031bb403ed11caf16f720fa9cfa2cec2077a1f721d86b9d0698e909c80"}
Jul 24 17:46:36.029343 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:36.029298    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-22bf5f/test-model-22bf5f-kserve-6ff4f45f5-dzd97" podStartSLOduration=1.029283602 podStartE2EDuration="1.029283602s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:46:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:46:36.026310414 +0000 UTC m=+2163.228359687" watchObservedRunningTime="2026-07-24 17:46:36.029283602 +0000 UTC m=+2163.231332876"
Jul 24 17:46:39.557956 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:39.557924    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:46:45.582657 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:45.582610    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-22bf5f/test-model-22bf5f-kserve-6ff4f45f5-dzd97"
Jul 24 17:46:45.582657 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:45.582669    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-22bf5f/test-model-22bf5f-kserve-6ff4f45f5-dzd97"
Jul 24 17:46:45.595384 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:45.595359    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-22bf5f/test-model-22bf5f-kserve-6ff4f45f5-dzd97"
Jul 24 17:46:46.074012 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:46.073975    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-22bf5f/test-model-22bf5f-kserve-6ff4f45f5-dzd97"
Jul 24 17:46:52.973444 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:46:52.973401    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:47:16.537870 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:16.537835    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-387e6e/test-model-387e6e-kserve-668ccd9fb9-x76bx"]
Jul 24 17:47:16.562166 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:16.562137    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:47:17.413707 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:17.413674    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-22bf5f/test-model-22bf5f-kserve-6ff4f45f5-dzd97"]
Jul 24 17:47:17.434193 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:17.434162    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:47:17.761174 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:17.761079    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:47:21.353676 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:21.353639    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:47:27.920689 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:27.920657    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:47:28.034653 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:28.034619    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:47:28.135566 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:28.135528    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:47:31.557748 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:31.557709    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-387e6e/test-model-387e6e-kserve-668ccd9fb9-x76bx" podUID="306628fc-85be-4c4b-98dd-25cfdc8371a7" containerName="main" containerID="cri-o://b4e5a6a6713e170246006981da355cc38c13b9168be9f3ac69afdbdce6d9ac9c" gracePeriod=45
Jul 24 17:47:31.807558 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:31.807526    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-387e6e/test-model-387e6e-kserve-668ccd9fb9-x76bx"
Jul 24 17:47:31.890352 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:31.890321    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/306628fc-85be-4c4b-98dd-25cfdc8371a7-tls-certs\") pod \"306628fc-85be-4c4b-98dd-25cfdc8371a7\" (UID: \"306628fc-85be-4c4b-98dd-25cfdc8371a7\") "
Jul 24 17:47:31.890542 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:31.890362    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/306628fc-85be-4c4b-98dd-25cfdc8371a7-dshm\") pod \"306628fc-85be-4c4b-98dd-25cfdc8371a7\" (UID: \"306628fc-85be-4c4b-98dd-25cfdc8371a7\") "
Jul 24 17:47:31.890542 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:31.890405    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xksjq\" (UniqueName: \"kubernetes.io/projected/306628fc-85be-4c4b-98dd-25cfdc8371a7-kube-api-access-xksjq\") pod \"306628fc-85be-4c4b-98dd-25cfdc8371a7\" (UID: \"306628fc-85be-4c4b-98dd-25cfdc8371a7\") "
Jul 24 17:47:31.890542 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:31.890438    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/306628fc-85be-4c4b-98dd-25cfdc8371a7-home\") pod \"306628fc-85be-4c4b-98dd-25cfdc8371a7\" (UID: \"306628fc-85be-4c4b-98dd-25cfdc8371a7\") "
Jul 24 17:47:31.890542 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:31.890487    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/306628fc-85be-4c4b-98dd-25cfdc8371a7-tmp-dir\") pod \"306628fc-85be-4c4b-98dd-25cfdc8371a7\" (UID: \"306628fc-85be-4c4b-98dd-25cfdc8371a7\") "
Jul 24 17:47:31.890758 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:31.890550    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/306628fc-85be-4c4b-98dd-25cfdc8371a7-model-cache\") pod \"306628fc-85be-4c4b-98dd-25cfdc8371a7\" (UID: \"306628fc-85be-4c4b-98dd-25cfdc8371a7\") "
Jul 24 17:47:31.890758 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:31.890710    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/306628fc-85be-4c4b-98dd-25cfdc8371a7-home" (OuterVolumeSpecName: "home") pod "306628fc-85be-4c4b-98dd-25cfdc8371a7" (UID: "306628fc-85be-4c4b-98dd-25cfdc8371a7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:47:31.890882 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:31.890784    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/306628fc-85be-4c4b-98dd-25cfdc8371a7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "306628fc-85be-4c4b-98dd-25cfdc8371a7" (UID: "306628fc-85be-4c4b-98dd-25cfdc8371a7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:47:31.890882 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:31.890856    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/306628fc-85be-4c4b-98dd-25cfdc8371a7-model-cache" (OuterVolumeSpecName: "model-cache") pod "306628fc-85be-4c4b-98dd-25cfdc8371a7" (UID: "306628fc-85be-4c4b-98dd-25cfdc8371a7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:47:31.890987 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:31.890905    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/306628fc-85be-4c4b-98dd-25cfdc8371a7-home\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:47:31.890987 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:31.890940    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/306628fc-85be-4c4b-98dd-25cfdc8371a7-tmp-dir\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:47:31.892801 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:31.892772    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/306628fc-85be-4c4b-98dd-25cfdc8371a7-dshm" (OuterVolumeSpecName: "dshm") pod "306628fc-85be-4c4b-98dd-25cfdc8371a7" (UID: "306628fc-85be-4c4b-98dd-25cfdc8371a7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:47:31.892912 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:31.892804    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/306628fc-85be-4c4b-98dd-25cfdc8371a7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "306628fc-85be-4c4b-98dd-25cfdc8371a7" (UID: "306628fc-85be-4c4b-98dd-25cfdc8371a7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:47:31.892912 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:31.892815    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/306628fc-85be-4c4b-98dd-25cfdc8371a7-kube-api-access-xksjq" (OuterVolumeSpecName: "kube-api-access-xksjq") pod "306628fc-85be-4c4b-98dd-25cfdc8371a7" (UID: "306628fc-85be-4c4b-98dd-25cfdc8371a7"). InnerVolumeSpecName "kube-api-access-xksjq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:47:31.992330 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:31.992296    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/306628fc-85be-4c4b-98dd-25cfdc8371a7-model-cache\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:47:31.992330 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:31.992322    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/306628fc-85be-4c4b-98dd-25cfdc8371a7-tls-certs\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:47:31.992330 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:31.992330    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/306628fc-85be-4c4b-98dd-25cfdc8371a7-dshm\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:47:31.992330 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:31.992338    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xksjq\" (UniqueName: \"kubernetes.io/projected/306628fc-85be-4c4b-98dd-25cfdc8371a7-kube-api-access-xksjq\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:47:32.288887 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:32.288792    2581 generic.go:356] "Generic (PLEG): container finished" podID="306628fc-85be-4c4b-98dd-25cfdc8371a7" containerID="b4e5a6a6713e170246006981da355cc38c13b9168be9f3ac69afdbdce6d9ac9c" exitCode=0
Jul 24 17:47:32.288887 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:32.288839    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-387e6e/test-model-387e6e-kserve-668ccd9fb9-x76bx"
Jul 24 17:47:32.289157 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:32.288874    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-387e6e/test-model-387e6e-kserve-668ccd9fb9-x76bx" event={"ID":"306628fc-85be-4c4b-98dd-25cfdc8371a7","Type":"ContainerDied","Data":"b4e5a6a6713e170246006981da355cc38c13b9168be9f3ac69afdbdce6d9ac9c"}
Jul 24 17:47:32.289157 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:32.288928    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-387e6e/test-model-387e6e-kserve-668ccd9fb9-x76bx" event={"ID":"306628fc-85be-4c4b-98dd-25cfdc8371a7","Type":"ContainerDied","Data":"b2b9089200006ab4d5ad338d446d705af25fdee41e81bdcd42ab2dbb53cf7fc8"}
Jul 24 17:47:32.289157 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:32.288950    2581 scope.go:117] "RemoveContainer" containerID="b4e5a6a6713e170246006981da355cc38c13b9168be9f3ac69afdbdce6d9ac9c"
Jul 24 17:47:32.298094 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:32.298075    2581 scope.go:117] "RemoveContainer" containerID="b4e5a6a6713e170246006981da355cc38c13b9168be9f3ac69afdbdce6d9ac9c"
Jul 24 17:47:32.298379 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:47:32.298355    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4e5a6a6713e170246006981da355cc38c13b9168be9f3ac69afdbdce6d9ac9c\": container with ID starting with b4e5a6a6713e170246006981da355cc38c13b9168be9f3ac69afdbdce6d9ac9c not found: ID does not exist" containerID="b4e5a6a6713e170246006981da355cc38c13b9168be9f3ac69afdbdce6d9ac9c"
Jul 24 17:47:32.298465 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:32.298387    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4e5a6a6713e170246006981da355cc38c13b9168be9f3ac69afdbdce6d9ac9c"} err="failed to get container status \"b4e5a6a6713e170246006981da355cc38c13b9168be9f3ac69afdbdce6d9ac9c\": rpc error: code = NotFound desc = could not find container \"b4e5a6a6713e170246006981da355cc38c13b9168be9f3ac69afdbdce6d9ac9c\": container with ID starting with b4e5a6a6713e170246006981da355cc38c13b9168be9f3ac69afdbdce6d9ac9c not found: ID does not exist"
Jul 24 17:47:32.309514 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:32.309486    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-387e6e/test-model-387e6e-kserve-668ccd9fb9-x76bx"]
Jul 24 17:47:32.312933 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:32.312914    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-387e6e/test-model-387e6e-kserve-668ccd9fb9-x76bx"]
Jul 24 17:47:32.433781 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:32.433732    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-22bf5f/test-model-22bf5f-kserve-6ff4f45f5-dzd97" podUID="67daf14a-ba12-464a-acfb-9f4640f382f3" containerName="main" containerID="cri-o://85b18dc06b3ee70e44a111d324109246e01fdc867ab24447ec075651cc63fa87" gracePeriod=45
Jul 24 17:47:32.620308 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:32.620277    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:47:32.701013 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:32.700988    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-22bf5f/test-model-22bf5f-kserve-6ff4f45f5-dzd97"
Jul 24 17:47:32.798802 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:32.798772    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mdkpl\" (UniqueName: \"kubernetes.io/projected/67daf14a-ba12-464a-acfb-9f4640f382f3-kube-api-access-mdkpl\") pod \"67daf14a-ba12-464a-acfb-9f4640f382f3\" (UID: \"67daf14a-ba12-464a-acfb-9f4640f382f3\") "
Jul 24 17:47:32.798959 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:32.798850    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67daf14a-ba12-464a-acfb-9f4640f382f3-home\") pod \"67daf14a-ba12-464a-acfb-9f4640f382f3\" (UID: \"67daf14a-ba12-464a-acfb-9f4640f382f3\") "
Jul 24 17:47:32.798959 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:32.798885    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67daf14a-ba12-464a-acfb-9f4640f382f3-tls-certs\") pod \"67daf14a-ba12-464a-acfb-9f4640f382f3\" (UID: \"67daf14a-ba12-464a-acfb-9f4640f382f3\") "
Jul 24 17:47:32.798959 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:32.798908    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67daf14a-ba12-464a-acfb-9f4640f382f3-model-cache\") pod \"67daf14a-ba12-464a-acfb-9f4640f382f3\" (UID: \"67daf14a-ba12-464a-acfb-9f4640f382f3\") "
Jul 24 17:47:32.798959 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:32.798926    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67daf14a-ba12-464a-acfb-9f4640f382f3-dshm\") pod \"67daf14a-ba12-464a-acfb-9f4640f382f3\" (UID: \"67daf14a-ba12-464a-acfb-9f4640f382f3\") "
Jul 24 17:47:32.799222 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:32.798968    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67daf14a-ba12-464a-acfb-9f4640f382f3-tmp-dir\") pod \"67daf14a-ba12-464a-acfb-9f4640f382f3\" (UID: \"67daf14a-ba12-464a-acfb-9f4640f382f3\") "
Jul 24 17:47:32.799222 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:32.799189    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67daf14a-ba12-464a-acfb-9f4640f382f3-home" (OuterVolumeSpecName: "home") pod "67daf14a-ba12-464a-acfb-9f4640f382f3" (UID: "67daf14a-ba12-464a-acfb-9f4640f382f3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:47:32.799222 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:32.799204    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67daf14a-ba12-464a-acfb-9f4640f382f3-model-cache" (OuterVolumeSpecName: "model-cache") pod "67daf14a-ba12-464a-acfb-9f4640f382f3" (UID: "67daf14a-ba12-464a-acfb-9f4640f382f3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:47:32.799414 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:32.799279    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67daf14a-ba12-464a-acfb-9f4640f382f3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "67daf14a-ba12-464a-acfb-9f4640f382f3" (UID: "67daf14a-ba12-464a-acfb-9f4640f382f3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:47:32.799414 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:32.799373    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67daf14a-ba12-464a-acfb-9f4640f382f3-home\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:47:32.799414 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:32.799390    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67daf14a-ba12-464a-acfb-9f4640f382f3-model-cache\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:47:32.799414 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:32.799405    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67daf14a-ba12-464a-acfb-9f4640f382f3-tmp-dir\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:47:32.801192 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:32.801162    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67daf14a-ba12-464a-acfb-9f4640f382f3-dshm" (OuterVolumeSpecName: "dshm") pod "67daf14a-ba12-464a-acfb-9f4640f382f3" (UID: "67daf14a-ba12-464a-acfb-9f4640f382f3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:47:32.801297 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:32.801212    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67daf14a-ba12-464a-acfb-9f4640f382f3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "67daf14a-ba12-464a-acfb-9f4640f382f3" (UID: "67daf14a-ba12-464a-acfb-9f4640f382f3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:47:32.801297 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:32.801252    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67daf14a-ba12-464a-acfb-9f4640f382f3-kube-api-access-mdkpl" (OuterVolumeSpecName: "kube-api-access-mdkpl") pod "67daf14a-ba12-464a-acfb-9f4640f382f3" (UID: "67daf14a-ba12-464a-acfb-9f4640f382f3"). InnerVolumeSpecName "kube-api-access-mdkpl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:47:32.899922 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:32.899880    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67daf14a-ba12-464a-acfb-9f4640f382f3-tls-certs\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:47:32.899922 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:32.899910    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67daf14a-ba12-464a-acfb-9f4640f382f3-dshm\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:47:32.899922 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:32.899919    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdkpl\" (UniqueName: \"kubernetes.io/projected/67daf14a-ba12-464a-acfb-9f4640f382f3-kube-api-access-mdkpl\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:47:32.996411 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:32.996374    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:47:33.295367 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:33.295328    2581 generic.go:356] "Generic (PLEG): container finished" podID="67daf14a-ba12-464a-acfb-9f4640f382f3" containerID="85b18dc06b3ee70e44a111d324109246e01fdc867ab24447ec075651cc63fa87" exitCode=0
Jul 24 17:47:33.295552 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:33.295374    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-22bf5f/test-model-22bf5f-kserve-6ff4f45f5-dzd97"
Jul 24 17:47:33.295552 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:33.295405    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-22bf5f/test-model-22bf5f-kserve-6ff4f45f5-dzd97" event={"ID":"67daf14a-ba12-464a-acfb-9f4640f382f3","Type":"ContainerDied","Data":"85b18dc06b3ee70e44a111d324109246e01fdc867ab24447ec075651cc63fa87"}
Jul 24 17:47:33.295552 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:33.295454    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-22bf5f/test-model-22bf5f-kserve-6ff4f45f5-dzd97" event={"ID":"67daf14a-ba12-464a-acfb-9f4640f382f3","Type":"ContainerDied","Data":"aae72e031bb403ed11caf16f720fa9cfa2cec2077a1f721d86b9d0698e909c80"}
Jul 24 17:47:33.295552 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:33.295471    2581 scope.go:117] "RemoveContainer" containerID="85b18dc06b3ee70e44a111d324109246e01fdc867ab24447ec075651cc63fa87"
Jul 24 17:47:33.305056 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:33.305020    2581 scope.go:117] "RemoveContainer" containerID="85b18dc06b3ee70e44a111d324109246e01fdc867ab24447ec075651cc63fa87"
Jul 24 17:47:33.305312 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:47:33.305295    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85b18dc06b3ee70e44a111d324109246e01fdc867ab24447ec075651cc63fa87\": container with ID starting with 85b18dc06b3ee70e44a111d324109246e01fdc867ab24447ec075651cc63fa87 not found: ID does not exist" containerID="85b18dc06b3ee70e44a111d324109246e01fdc867ab24447ec075651cc63fa87"
Jul 24 17:47:33.305379 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:33.305321    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85b18dc06b3ee70e44a111d324109246e01fdc867ab24447ec075651cc63fa87"} err="failed to get container status \"85b18dc06b3ee70e44a111d324109246e01fdc867ab24447ec075651cc63fa87\": rpc error: code = NotFound desc = could not find container \"85b18dc06b3ee70e44a111d324109246e01fdc867ab24447ec075651cc63fa87\": container with ID starting with 85b18dc06b3ee70e44a111d324109246e01fdc867ab24447ec075651cc63fa87 not found: ID does not exist"
Jul 24 17:47:33.317074 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:33.317032    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-22bf5f/test-model-22bf5f-kserve-6ff4f45f5-dzd97"]
Jul 24 17:47:33.320756 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:33.320733    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-22bf5f/test-model-22bf5f-kserve-6ff4f45f5-dzd97"]
Jul 24 17:47:33.384366 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:33.384332    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="306628fc-85be-4c4b-98dd-25cfdc8371a7" path="/var/lib/kubelet/pods/306628fc-85be-4c4b-98dd-25cfdc8371a7/volumes"
Jul 24 17:47:33.384730 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:33.384717    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67daf14a-ba12-464a-acfb-9f4640f382f3" path="/var/lib/kubelet/pods/67daf14a-ba12-464a-acfb-9f4640f382f3/volumes"
Jul 24 17:47:36.758853 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:36.758818    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:47:39.062807 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:39.062772    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:47:39.157067 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:39.157022    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:47:39.327513 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:39.327430    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:47:39.565436 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:39.565405    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:47:39.671429 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:39.671395    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-22bf5f-dgdxb"]
Jul 24 17:47:39.671788 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:39.671776    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="306628fc-85be-4c4b-98dd-25cfdc8371a7" containerName="main"
Jul 24 17:47:39.671833 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:39.671790    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="306628fc-85be-4c4b-98dd-25cfdc8371a7" containerName="main"
Jul 24 17:47:39.671833 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:39.671802    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67daf14a-ba12-464a-acfb-9f4640f382f3" containerName="main"
Jul 24 17:47:39.671833 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:39.671807    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="67daf14a-ba12-464a-acfb-9f4640f382f3" containerName="main"
Jul 24 17:47:39.671933 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:39.671878    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="306628fc-85be-4c4b-98dd-25cfdc8371a7" containerName="main"
Jul 24 17:47:39.671933 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:39.671888    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="67daf14a-ba12-464a-acfb-9f4640f382f3" containerName="main"
Jul 24 17:47:39.674832 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:39.674816    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-22bf5f-dgdxb"
Jul 24 17:47:39.677612 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:39.677589    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 17:47:39.677771 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:39.677755    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d6flx\""
Jul 24 17:47:39.681746 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:39.681724    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-22bf5f-dgdxb"]
Jul 24 17:47:39.766118 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:39.766074    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/62d20704-8b7f-446f-9513-4e51e832b485-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-22bf5f-dgdxb\" (UID: \"62d20704-8b7f-446f-9513-4e51e832b485\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-22bf5f-dgdxb"
Jul 24 17:47:39.866824 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:39.866786    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/62d20704-8b7f-446f-9513-4e51e832b485-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-22bf5f-dgdxb\" (UID: \"62d20704-8b7f-446f-9513-4e51e832b485\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-22bf5f-dgdxb"
Jul 24 17:47:39.867453 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:39.867434    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/62d20704-8b7f-446f-9513-4e51e832b485-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-22bf5f-dgdxb\" (UID: \"62d20704-8b7f-446f-9513-4e51e832b485\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-22bf5f-dgdxb"
Jul 24 17:47:39.987024 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:39.986937    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-22bf5f-dgdxb"
Jul 24 17:47:40.143756 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:40.143450    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-22bf5f-dgdxb"]
Jul 24 17:47:40.146470 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:47:40.146441    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62d20704_8b7f_446f_9513_4e51e832b485.slice/crio-81449f3750524849055b4f4532e9e0e7d699826e177d5dfa99193871aa871ab9 WatchSource:0}: Error finding container 81449f3750524849055b4f4532e9e0e7d699826e177d5dfa99193871aa871ab9: Status 404 returned error can't find the container with id 81449f3750524849055b4f4532e9e0e7d699826e177d5dfa99193871aa871ab9
Jul 24 17:47:40.148920 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:40.148905    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 17:47:40.331741 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:40.331653    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-22bf5f-dgdxb" event={"ID":"62d20704-8b7f-446f-9513-4e51e832b485","Type":"ContainerStarted","Data":"81449f3750524849055b4f4532e9e0e7d699826e177d5dfa99193871aa871ab9"}
Jul 24 17:47:43.348786 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:43.348747    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-22bf5f-dgdxb" event={"ID":"62d20704-8b7f-446f-9513-4e51e832b485","Type":"ContainerStarted","Data":"fb91a0d780954f1abe61aceb3b82edc9c081f2437108b15d553a09747edd06e7"}
Jul 24 17:47:43.366165 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:43.366121    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-22bf5f-dgdxb" podStartSLOduration=1.6012138299999998 podStartE2EDuration="4.366105309s" totalImagesPullingTime="2.764891479s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:47:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:47:43.365762172 +0000 UTC m=+2230.567811447" watchObservedRunningTime="2026-07-24 17:47:43.366105309 +0000 UTC m=+2230.568154583"
Jul 24 17:47:44.355107 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:44.355066    2581 generic.go:356] "Generic (PLEG): container finished" podID="62d20704-8b7f-446f-9513-4e51e832b485" containerID="fb91a0d780954f1abe61aceb3b82edc9c081f2437108b15d553a09747edd06e7" exitCode=0
Jul 24 17:47:44.355517 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:44.355124    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-22bf5f-dgdxb" event={"ID":"62d20704-8b7f-446f-9513-4e51e832b485","Type":"ContainerDied","Data":"fb91a0d780954f1abe61aceb3b82edc9c081f2437108b15d553a09747edd06e7"}
Jul 24 17:47:45.492414 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:45.492390    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-22bf5f-dgdxb"
Jul 24 17:47:45.622567 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:45.622467    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/62d20704-8b7f-446f-9513-4e51e832b485-maas-api-service-ca\") pod \"62d20704-8b7f-446f-9513-4e51e832b485\" (UID: \"62d20704-8b7f-446f-9513-4e51e832b485\") "
Jul 24 17:47:45.622899 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:45.622862    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62d20704-8b7f-446f-9513-4e51e832b485-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "62d20704-8b7f-446f-9513-4e51e832b485" (UID: "62d20704-8b7f-446f-9513-4e51e832b485"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:47:45.724215 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:45.724171    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/62d20704-8b7f-446f-9513-4e51e832b485-maas-api-service-ca\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:47:46.366381 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:46.366352    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-22bf5f-dgdxb"
Jul 24 17:47:46.366524 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:46.366390    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-22bf5f-dgdxb" event={"ID":"62d20704-8b7f-446f-9513-4e51e832b485","Type":"ContainerDied","Data":"81449f3750524849055b4f4532e9e0e7d699826e177d5dfa99193871aa871ab9"}
Jul 24 17:47:46.366524 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:46.366434    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="81449f3750524849055b4f4532e9e0e7d699826e177d5dfa99193871aa871ab9"
Jul 24 17:47:50.068180 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:50.068146    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:47:50.223586 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:50.223522    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:47:50.321843 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:50.321757    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:47:52.961911 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:52.961867    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:47:54.733743 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:54.733714    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-22bf5f-dgdxb"]
Jul 24 17:47:54.735397 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:54.735371    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-22bf5f-dgdxb"]
Jul 24 17:47:54.744428 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:54.744401    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:47:55.094320 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:55.094233    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:47:55.383897 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:47:55.383866    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62d20704-8b7f-446f-9513-4e51e832b485" path="/var/lib/kubelet/pods/62d20704-8b7f-446f-9513-4e51e832b485/volumes"
Jul 24 17:48:01.121789 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:01.121756    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:48:01.222168 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:01.222140    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:48:01.381544 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:01.381453    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:48:01.608492 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:01.608463    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:48:01.948557 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:01.948521    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:48:02.266783 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:02.266705    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:48:02.476166 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:02.476124    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:48:05.004216 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:05.004176    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:48:05.098430 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:05.098382    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:48:05.191739 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:05.191706    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:48:05.297872 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:05.297789    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:48:05.695277 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:05.695244    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:48:05.801541 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:05.801507    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:48:09.505948 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:09.505910    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:48:09.963242 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:09.963214    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:48:10.308990 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:10.308904    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:48:10.885500 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:10.885462    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:48:12.164057 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:12.164013    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:48:13.100066 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:13.100004    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:48:13.187979 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:13.187943    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:48:13.259294 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:13.259260    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:48:13.516248 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:13.516214    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:48:13.853381 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:13.853278    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:48:13.943118 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:13.943079    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:48:17.537808 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:17.537774    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:48:25.533694 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:25.533662    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cf30f0-zg7ph"]
Jul 24 17:48:25.534182 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:25.534058    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62d20704-8b7f-446f-9513-4e51e832b485" containerName="revoke-keys"
Jul 24 17:48:25.534182 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:25.534069    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="62d20704-8b7f-446f-9513-4e51e832b485" containerName="revoke-keys"
Jul 24 17:48:25.534182 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:25.534138    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="62d20704-8b7f-446f-9513-4e51e832b485" containerName="revoke-keys"
Jul 24 17:48:25.540435 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:25.540416    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cf30f0-zg7ph"
Jul 24 17:48:25.543280 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:25.543254    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d6flx\""
Jul 24 17:48:25.543445 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:25.543340    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 17:48:25.544176 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:25.544149    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cf30f0-zg7ph"]
Jul 24 17:48:25.706870 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:25.706824    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38e42a5f-4803-460c-9846-7e3cf2b6917a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-cf30f0-zg7ph\" (UID: \"38e42a5f-4803-460c-9846-7e3cf2b6917a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cf30f0-zg7ph"
Jul 24 17:48:25.808212 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:25.808106    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38e42a5f-4803-460c-9846-7e3cf2b6917a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-cf30f0-zg7ph\" (UID: \"38e42a5f-4803-460c-9846-7e3cf2b6917a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cf30f0-zg7ph"
Jul 24 17:48:25.808783 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:25.808762    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38e42a5f-4803-460c-9846-7e3cf2b6917a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-cf30f0-zg7ph\" (UID: \"38e42a5f-4803-460c-9846-7e3cf2b6917a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cf30f0-zg7ph"
Jul 24 17:48:25.852930 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:25.852895    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cf30f0-zg7ph"
Jul 24 17:48:25.987018 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:25.986996    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cf30f0-zg7ph"]
Jul 24 17:48:25.989214 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:48:25.989183    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38e42a5f_4803_460c_9846_7e3cf2b6917a.slice/crio-890c103d368ca0c6ed3fcc88dedbeafdf78b7c6454f622c78338c533035f0420 WatchSource:0}: Error finding container 890c103d368ca0c6ed3fcc88dedbeafdf78b7c6454f622c78338c533035f0420: Status 404 returned error can't find the container with id 890c103d368ca0c6ed3fcc88dedbeafdf78b7c6454f622c78338c533035f0420
Jul 24 17:48:26.572496 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:26.572407    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cf30f0-zg7ph" event={"ID":"38e42a5f-4803-460c-9846-7e3cf2b6917a","Type":"ContainerStarted","Data":"a66c7ee40851a8f3dacbe5ba9d4615139d1407158cbd080867ebb319c35ba7a4"}
Jul 24 17:48:26.572496 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:26.572471    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cf30f0-zg7ph" event={"ID":"38e42a5f-4803-460c-9846-7e3cf2b6917a","Type":"ContainerStarted","Data":"890c103d368ca0c6ed3fcc88dedbeafdf78b7c6454f622c78338c533035f0420"}
Jul 24 17:48:26.589491 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:26.589439    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cf30f0-zg7ph" podStartSLOduration=1.589424228 podStartE2EDuration="1.589424228s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:48:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:48:26.58791037 +0000 UTC m=+2273.789959645" watchObservedRunningTime="2026-07-24 17:48:26.589424228 +0000 UTC m=+2273.791473495"
Jul 24 17:48:27.580946 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:27.580903    2581 generic.go:356] "Generic (PLEG): container finished" podID="38e42a5f-4803-460c-9846-7e3cf2b6917a" containerID="a66c7ee40851a8f3dacbe5ba9d4615139d1407158cbd080867ebb319c35ba7a4" exitCode=0
Jul 24 17:48:27.581346 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:27.580951    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cf30f0-zg7ph" event={"ID":"38e42a5f-4803-460c-9846-7e3cf2b6917a","Type":"ContainerDied","Data":"a66c7ee40851a8f3dacbe5ba9d4615139d1407158cbd080867ebb319c35ba7a4"}
Jul 24 17:48:28.556677 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:28.556644    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:48:28.721625 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:28.721601    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cf30f0-zg7ph"
Jul 24 17:48:28.841776 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:28.841689    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38e42a5f-4803-460c-9846-7e3cf2b6917a-maas-api-service-ca\") pod \"38e42a5f-4803-460c-9846-7e3cf2b6917a\" (UID: \"38e42a5f-4803-460c-9846-7e3cf2b6917a\") "
Jul 24 17:48:28.842114 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:28.842088    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38e42a5f-4803-460c-9846-7e3cf2b6917a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "38e42a5f-4803-460c-9846-7e3cf2b6917a" (UID: "38e42a5f-4803-460c-9846-7e3cf2b6917a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:48:28.942559 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:28.942522    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38e42a5f-4803-460c-9846-7e3cf2b6917a-maas-api-service-ca\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:48:29.591626 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:29.591592    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cf30f0-zg7ph"
Jul 24 17:48:29.591796 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:29.591624    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cf30f0-zg7ph" event={"ID":"38e42a5f-4803-460c-9846-7e3cf2b6917a","Type":"ContainerDied","Data":"890c103d368ca0c6ed3fcc88dedbeafdf78b7c6454f622c78338c533035f0420"}
Jul 24 17:48:29.591796 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:29.591666    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="890c103d368ca0c6ed3fcc88dedbeafdf78b7c6454f622c78338c533035f0420"
Jul 24 17:48:40.735471 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:40.735435    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:48:40.802877 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:40.802826    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:48:40.860628 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:40.860596    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:48:44.359011 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:44.358967    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:48:45.600207 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:45.600179    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:48:45.604979 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:45.604951    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cf30f0-zg7ph"]
Jul 24 17:48:45.607781 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:45.607759    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-cf30f0-zg7ph"]
Jul 24 17:48:45.971977 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:45.971937    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:48:47.383772 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:47.383733    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38e42a5f-4803-460c-9846-7e3cf2b6917a" path="/var/lib/kubelet/pods/38e42a5f-4803-460c-9846-7e3cf2b6917a/volumes"
Jul 24 17:48:51.956067 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:51.956006    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:48:52.056362 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:52.056328    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:48:52.217500 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:52.217408    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:48:52.438330 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:52.438282    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:48:52.549120 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:52.549029    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0e9e3f-ndmgd"]
Jul 24 17:48:52.549517 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:52.549500    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38e42a5f-4803-460c-9846-7e3cf2b6917a" containerName="revoke-keys"
Jul 24 17:48:52.549568 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:52.549519    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="38e42a5f-4803-460c-9846-7e3cf2b6917a" containerName="revoke-keys"
Jul 24 17:48:52.549603 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:52.549596    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="38e42a5f-4803-460c-9846-7e3cf2b6917a" containerName="revoke-keys"
Jul 24 17:48:52.552107 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:52.552088    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0e9e3f-ndmgd"
Jul 24 17:48:52.554740 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:52.554718    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 17:48:52.554863 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:52.554787    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d6flx\""
Jul 24 17:48:52.559775 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:52.559750    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0e9e3f-ndmgd"]
Jul 24 17:48:52.663094 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:52.663032    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70bfcc1e-889c-4b77-8805-e2262dc67814-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-0e9e3f-ndmgd\" (UID: \"70bfcc1e-889c-4b77-8805-e2262dc67814\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0e9e3f-ndmgd"
Jul 24 17:48:52.763632 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:52.763594    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70bfcc1e-889c-4b77-8805-e2262dc67814-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-0e9e3f-ndmgd\" (UID: \"70bfcc1e-889c-4b77-8805-e2262dc67814\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0e9e3f-ndmgd"
Jul 24 17:48:52.764216 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:52.764199    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70bfcc1e-889c-4b77-8805-e2262dc67814-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-0e9e3f-ndmgd\" (UID: \"70bfcc1e-889c-4b77-8805-e2262dc67814\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0e9e3f-ndmgd"
Jul 24 17:48:52.863131 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:52.863025    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0e9e3f-ndmgd"
Jul 24 17:48:52.991255 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:52.991223    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0e9e3f-ndmgd"]
Jul 24 17:48:52.994209 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:48:52.994179    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70bfcc1e_889c_4b77_8805_e2262dc67814.slice/crio-4c464d870b29d4415f6921611fd36e848ed9e0e71be01dc948f82d129087c318 WatchSource:0}: Error finding container 4c464d870b29d4415f6921611fd36e848ed9e0e71be01dc948f82d129087c318: Status 404 returned error can't find the container with id 4c464d870b29d4415f6921611fd36e848ed9e0e71be01dc948f82d129087c318
Jul 24 17:48:53.709277 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:53.709244    2581 generic.go:356] "Generic (PLEG): container finished" podID="70bfcc1e-889c-4b77-8805-e2262dc67814" containerID="4dcb275f3710c3c78945a35de93ce0f081798c847bd61b951ff5f30c4bc3a397" exitCode=0
Jul 24 17:48:53.709428 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:53.709294    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0e9e3f-ndmgd" event={"ID":"70bfcc1e-889c-4b77-8805-e2262dc67814","Type":"ContainerDied","Data":"4dcb275f3710c3c78945a35de93ce0f081798c847bd61b951ff5f30c4bc3a397"}
Jul 24 17:48:53.709428 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:53.709350    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0e9e3f-ndmgd" event={"ID":"70bfcc1e-889c-4b77-8805-e2262dc67814","Type":"ContainerStarted","Data":"4c464d870b29d4415f6921611fd36e848ed9e0e71be01dc948f82d129087c318"}
Jul 24 17:48:54.848473 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:54.848449    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0e9e3f-ndmgd"
Jul 24 17:48:54.985912 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:54.985824    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70bfcc1e-889c-4b77-8805-e2262dc67814-maas-api-service-ca\") pod \"70bfcc1e-889c-4b77-8805-e2262dc67814\" (UID: \"70bfcc1e-889c-4b77-8805-e2262dc67814\") "
Jul 24 17:48:54.986282 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:54.986260    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70bfcc1e-889c-4b77-8805-e2262dc67814-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "70bfcc1e-889c-4b77-8805-e2262dc67814" (UID: "70bfcc1e-889c-4b77-8805-e2262dc67814"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:48:55.087344 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:55.087313    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70bfcc1e-889c-4b77-8805-e2262dc67814-maas-api-service-ca\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:48:55.719992 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:55.719909    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0e9e3f-ndmgd"
Jul 24 17:48:55.720170 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:55.719913    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0e9e3f-ndmgd" event={"ID":"70bfcc1e-889c-4b77-8805-e2262dc67814","Type":"ContainerDied","Data":"4c464d870b29d4415f6921611fd36e848ed9e0e71be01dc948f82d129087c318"}
Jul 24 17:48:55.720170 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:48:55.720063    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4c464d870b29d4415f6921611fd36e848ed9e0e71be01dc948f82d129087c318"
Jul 24 17:49:08.205472 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:08.205419    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:49:08.271347 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:08.271309    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:49:08.330266 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:08.330235    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:49:12.624646 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:12.624590    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0e9e3f-ndmgd"]
Jul 24 17:49:12.626339 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:12.626305    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:49:12.626487 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:12.626462    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0e9e3f-ndmgd"]
Jul 24 17:49:12.870650 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:12.870614    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:49:13.009205 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:13.009154    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:49:13.384410 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:13.384378    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70bfcc1e-889c-4b77-8805-e2262dc67814" path="/var/lib/kubelet/pods/70bfcc1e-889c-4b77-8805-e2262dc67814/volumes"
Jul 24 17:49:18.945115 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:18.945048    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:49:19.017265 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:19.017234    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:49:19.210689 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:19.210599    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:49:19.447009 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:19.446982    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:49:19.793850 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:19.793820    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:49:20.120675 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:20.120594    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:49:20.338900 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:20.338868    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:49:22.261163 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:22.261127    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:49:22.362318 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:22.362280    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:49:23.526600 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:23.526511    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:49:23.685441 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:23.685402    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:49:23.764532 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:23.764495    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:49:23.910564 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:23.910516    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:49:24.372091 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:24.371971    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:49:24.488359 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:24.488316    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:49:27.975497 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:27.975461    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:49:28.328567 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:28.328474    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:49:28.700416 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:28.700382    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:49:29.257824 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:29.257786    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:49:31.155133 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:31.155097    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:49:31.504056 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:31.503948    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:49:31.605135 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:31.605092    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:49:31.687209 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:31.687170    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:49:31.975582 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:31.975548    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:49:32.235583 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:32.235480    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:49:32.333577 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:32.333537    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:49:35.989771 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:35.989735    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:49:42.273200 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:42.273162    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:49:51.189916 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:51.189883    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-bb94b3/auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d"]
Jul 24 17:49:51.190396 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:51.190299    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70bfcc1e-889c-4b77-8805-e2262dc67814" containerName="revoke-keys"
Jul 24 17:49:51.190396 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:51.190312    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="70bfcc1e-889c-4b77-8805-e2262dc67814" containerName="revoke-keys"
Jul 24 17:49:51.190396 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:51.190393    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="70bfcc1e-889c-4b77-8805-e2262dc67814" containerName="revoke-keys"
Jul 24 17:49:51.193251 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:51.193235    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-bb94b3/auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d"
Jul 24 17:49:51.196551 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:51.196403    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-bb94b3\"/\"openshift-service-ca.crt\""
Jul 24 17:49:51.197613 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:51.197583    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-bb94b3\"/\"default-dockercfg-kddnf\""
Jul 24 17:49:51.197727 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:51.197617    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-bb94b3\"/\"auth-test-model-bb94b3-kserve-self-signed-certs\""
Jul 24 17:49:51.197727 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:51.197625    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-bb94b3\"/\"kube-root-ca.crt\""
Jul 24 17:49:51.212621 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:51.212593    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-bb94b3/auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d"]
Jul 24 17:49:51.219098 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:51.219069    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b87c2542-3fbe-4eaa-9213-8ac3f35b5df5-home\") pod \"auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d\" (UID: \"b87c2542-3fbe-4eaa-9213-8ac3f35b5df5\") " pod="ai-tenant-e2e-shared-a-bb94b3/auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d"
Jul 24 17:49:51.219220 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:51.219166    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b87c2542-3fbe-4eaa-9213-8ac3f35b5df5-model-cache\") pod \"auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d\" (UID: \"b87c2542-3fbe-4eaa-9213-8ac3f35b5df5\") " pod="ai-tenant-e2e-shared-a-bb94b3/auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d"
Jul 24 17:49:51.219220 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:51.219190    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b87c2542-3fbe-4eaa-9213-8ac3f35b5df5-dshm\") pod \"auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d\" (UID: \"b87c2542-3fbe-4eaa-9213-8ac3f35b5df5\") " pod="ai-tenant-e2e-shared-a-bb94b3/auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d"
Jul 24 17:49:51.219299 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:51.219226    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b87c2542-3fbe-4eaa-9213-8ac3f35b5df5-tls-certs\") pod \"auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d\" (UID: \"b87c2542-3fbe-4eaa-9213-8ac3f35b5df5\") " pod="ai-tenant-e2e-shared-a-bb94b3/auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d"
Jul 24 17:49:51.219299 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:51.219250    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b87c2542-3fbe-4eaa-9213-8ac3f35b5df5-tmp-dir\") pod \"auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d\" (UID: \"b87c2542-3fbe-4eaa-9213-8ac3f35b5df5\") " pod="ai-tenant-e2e-shared-a-bb94b3/auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d"
Jul 24 17:49:51.219363 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:51.219317    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wp54p\" (UniqueName: \"kubernetes.io/projected/b87c2542-3fbe-4eaa-9213-8ac3f35b5df5-kube-api-access-wp54p\") pod \"auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d\" (UID: \"b87c2542-3fbe-4eaa-9213-8ac3f35b5df5\") " pod="ai-tenant-e2e-shared-a-bb94b3/auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d"
Jul 24 17:49:51.232392 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:51.232366    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:49:51.319801 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:51.319761    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b87c2542-3fbe-4eaa-9213-8ac3f35b5df5-tls-certs\") pod \"auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d\" (UID: \"b87c2542-3fbe-4eaa-9213-8ac3f35b5df5\") " pod="ai-tenant-e2e-shared-a-bb94b3/auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d"
Jul 24 17:49:51.319801 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:51.319805    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b87c2542-3fbe-4eaa-9213-8ac3f35b5df5-tmp-dir\") pod \"auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d\" (UID: \"b87c2542-3fbe-4eaa-9213-8ac3f35b5df5\") " pod="ai-tenant-e2e-shared-a-bb94b3/auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d"
Jul 24 17:49:51.320026 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:51.319947    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wp54p\" (UniqueName: \"kubernetes.io/projected/b87c2542-3fbe-4eaa-9213-8ac3f35b5df5-kube-api-access-wp54p\") pod \"auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d\" (UID: \"b87c2542-3fbe-4eaa-9213-8ac3f35b5df5\") " pod="ai-tenant-e2e-shared-a-bb94b3/auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d"
Jul 24 17:49:51.320122 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:51.320102    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b87c2542-3fbe-4eaa-9213-8ac3f35b5df5-home\") pod \"auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d\" (UID: \"b87c2542-3fbe-4eaa-9213-8ac3f35b5df5\") " pod="ai-tenant-e2e-shared-a-bb94b3/auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d"
Jul 24 17:49:51.320198 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:51.320181    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b87c2542-3fbe-4eaa-9213-8ac3f35b5df5-model-cache\") pod \"auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d\" (UID: \"b87c2542-3fbe-4eaa-9213-8ac3f35b5df5\") " pod="ai-tenant-e2e-shared-a-bb94b3/auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d"
Jul 24 17:49:51.320264 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:51.320214    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b87c2542-3fbe-4eaa-9213-8ac3f35b5df5-dshm\") pod \"auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d\" (UID: \"b87c2542-3fbe-4eaa-9213-8ac3f35b5df5\") " pod="ai-tenant-e2e-shared-a-bb94b3/auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d"
Jul 24 17:49:51.320264 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:51.320215    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b87c2542-3fbe-4eaa-9213-8ac3f35b5df5-tmp-dir\") pod \"auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d\" (UID: \"b87c2542-3fbe-4eaa-9213-8ac3f35b5df5\") " pod="ai-tenant-e2e-shared-a-bb94b3/auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d"
Jul 24 17:49:51.320457 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:51.320437    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b87c2542-3fbe-4eaa-9213-8ac3f35b5df5-home\") pod \"auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d\" (UID: \"b87c2542-3fbe-4eaa-9213-8ac3f35b5df5\") " pod="ai-tenant-e2e-shared-a-bb94b3/auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d"
Jul 24 17:49:51.320457 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:51.320452    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b87c2542-3fbe-4eaa-9213-8ac3f35b5df5-model-cache\") pod \"auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d\" (UID: \"b87c2542-3fbe-4eaa-9213-8ac3f35b5df5\") " pod="ai-tenant-e2e-shared-a-bb94b3/auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d"
Jul 24 17:49:51.322438 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:51.322415    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b87c2542-3fbe-4eaa-9213-8ac3f35b5df5-dshm\") pod \"auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d\" (UID: \"b87c2542-3fbe-4eaa-9213-8ac3f35b5df5\") " pod="ai-tenant-e2e-shared-a-bb94b3/auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d"
Jul 24 17:49:51.322553 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:51.322517    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b87c2542-3fbe-4eaa-9213-8ac3f35b5df5-tls-certs\") pod \"auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d\" (UID: \"b87c2542-3fbe-4eaa-9213-8ac3f35b5df5\") " pod="ai-tenant-e2e-shared-a-bb94b3/auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d"
Jul 24 17:49:51.329395 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:51.329368    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wp54p\" (UniqueName: \"kubernetes.io/projected/b87c2542-3fbe-4eaa-9213-8ac3f35b5df5-kube-api-access-wp54p\") pod \"auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d\" (UID: \"b87c2542-3fbe-4eaa-9213-8ac3f35b5df5\") " pod="ai-tenant-e2e-shared-a-bb94b3/auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d"
Jul 24 17:49:51.503814 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:51.503725    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-bb94b3/auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d"
Jul 24 17:49:51.650248 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:51.650223    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-bb94b3/auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d"]
Jul 24 17:49:51.652198 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:49:51.652167    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb87c2542_3fbe_4eaa_9213_8ac3f35b5df5.slice/crio-f8deea109a1761a81aa9dc9478772158d6b4514882775fdbeed477a55a2877c2 WatchSource:0}: Error finding container f8deea109a1761a81aa9dc9478772158d6b4514882775fdbeed477a55a2877c2: Status 404 returned error can't find the container with id f8deea109a1761a81aa9dc9478772158d6b4514882775fdbeed477a55a2877c2
Jul 24 17:49:52.005116 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:52.005075    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-bb94b3/auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d" event={"ID":"b87c2542-3fbe-4eaa-9213-8ac3f35b5df5","Type":"ContainerStarted","Data":"8c9e70e87cb46f26caf5b3ecf94f116dc4690be7b53c021b4a2536843d61f0e4"}
Jul 24 17:49:52.005314 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:52.005128    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-bb94b3/auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d" event={"ID":"b87c2542-3fbe-4eaa-9213-8ac3f35b5df5","Type":"ContainerStarted","Data":"f8deea109a1761a81aa9dc9478772158d6b4514882775fdbeed477a55a2877c2"}
Jul 24 17:49:52.026346 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:52.026290    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-bb94b3/auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d" podStartSLOduration=1.026276474 podStartE2EDuration="1.026276474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:49:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:49:52.024897841 +0000 UTC m=+2359.226947128" watchObservedRunningTime="2026-07-24 17:49:52.026276474 +0000 UTC m=+2359.228325749"
Jul 24 17:49:54.461451 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:49:54.461408    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:50:01.504895 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:01.504850    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-bb94b3/auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d"
Jul 24 17:50:01.505542 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:01.504909    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-bb94b3/auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d"
Jul 24 17:50:01.517145 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:01.517120    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-bb94b3/auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d"
Jul 24 17:50:02.072421 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:02.072392    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-bb94b3/auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d"
Jul 24 17:50:05.982514 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:05.982477    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-5c7cfc/auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m"]
Jul 24 17:50:05.986242 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:05.986221    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5c7cfc/auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m"
Jul 24 17:50:05.989417 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:05.989396    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-5c7cfc\"/\"openshift-service-ca.crt\""
Jul 24 17:50:05.989590 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:05.989572    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-5c7cfc\"/\"default-dockercfg-hkndk\""
Jul 24 17:50:05.990684 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:05.990667    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-5c7cfc\"/\"kube-root-ca.crt\""
Jul 24 17:50:05.990817 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:05.990800    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-5c7cfc\"/\"auth-test-model-5c7cfc-kserve-self-signed-certs\""
Jul 24 17:50:06.002559 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:06.002536    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-5c7cfc/auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m"]
Jul 24 17:50:06.039107 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:06.039067    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:50:06.063273 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:06.063243    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3fe6317-c46c-4e61-b0ec-5632b132c705-home\") pod \"auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m\" (UID: \"d3fe6317-c46c-4e61-b0ec-5632b132c705\") " pod="ai-tenant-e2e-shared-b-5c7cfc/auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m"
Jul 24 17:50:06.063413 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:06.063288    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3fe6317-c46c-4e61-b0ec-5632b132c705-dshm\") pod \"auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m\" (UID: \"d3fe6317-c46c-4e61-b0ec-5632b132c705\") " pod="ai-tenant-e2e-shared-b-5c7cfc/auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m"
Jul 24 17:50:06.063413 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:06.063354    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3fe6317-c46c-4e61-b0ec-5632b132c705-model-cache\") pod \"auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m\" (UID: \"d3fe6317-c46c-4e61-b0ec-5632b132c705\") " pod="ai-tenant-e2e-shared-b-5c7cfc/auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m"
Jul 24 17:50:06.063413 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:06.063372    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3fe6317-c46c-4e61-b0ec-5632b132c705-tls-certs\") pod \"auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m\" (UID: \"d3fe6317-c46c-4e61-b0ec-5632b132c705\") " pod="ai-tenant-e2e-shared-b-5c7cfc/auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m"
Jul 24 17:50:06.063581 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:06.063455    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmfj2\" (UniqueName: \"kubernetes.io/projected/d3fe6317-c46c-4e61-b0ec-5632b132c705-kube-api-access-cmfj2\") pod \"auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m\" (UID: \"d3fe6317-c46c-4e61-b0ec-5632b132c705\") " pod="ai-tenant-e2e-shared-b-5c7cfc/auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m"
Jul 24 17:50:06.063581 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:06.063508    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3fe6317-c46c-4e61-b0ec-5632b132c705-tmp-dir\") pod \"auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m\" (UID: \"d3fe6317-c46c-4e61-b0ec-5632b132c705\") " pod="ai-tenant-e2e-shared-b-5c7cfc/auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m"
Jul 24 17:50:06.164350 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:06.164309    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3fe6317-c46c-4e61-b0ec-5632b132c705-model-cache\") pod \"auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m\" (UID: \"d3fe6317-c46c-4e61-b0ec-5632b132c705\") " pod="ai-tenant-e2e-shared-b-5c7cfc/auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m"
Jul 24 17:50:06.164350 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:06.164358    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3fe6317-c46c-4e61-b0ec-5632b132c705-tls-certs\") pod \"auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m\" (UID: \"d3fe6317-c46c-4e61-b0ec-5632b132c705\") " pod="ai-tenant-e2e-shared-b-5c7cfc/auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m"
Jul 24 17:50:06.164625 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:06.164391    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmfj2\" (UniqueName: \"kubernetes.io/projected/d3fe6317-c46c-4e61-b0ec-5632b132c705-kube-api-access-cmfj2\") pod \"auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m\" (UID: \"d3fe6317-c46c-4e61-b0ec-5632b132c705\") " pod="ai-tenant-e2e-shared-b-5c7cfc/auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m"
Jul 24 17:50:06.164625 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:06.164416    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3fe6317-c46c-4e61-b0ec-5632b132c705-tmp-dir\") pod \"auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m\" (UID: \"d3fe6317-c46c-4e61-b0ec-5632b132c705\") " pod="ai-tenant-e2e-shared-b-5c7cfc/auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m"
Jul 24 17:50:06.164625 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:06.164578    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3fe6317-c46c-4e61-b0ec-5632b132c705-home\") pod \"auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m\" (UID: \"d3fe6317-c46c-4e61-b0ec-5632b132c705\") " pod="ai-tenant-e2e-shared-b-5c7cfc/auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m"
Jul 24 17:50:06.164791 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:06.164644    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3fe6317-c46c-4e61-b0ec-5632b132c705-dshm\") pod \"auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m\" (UID: \"d3fe6317-c46c-4e61-b0ec-5632b132c705\") " pod="ai-tenant-e2e-shared-b-5c7cfc/auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m"
Jul 24 17:50:06.164791 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:06.164746    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3fe6317-c46c-4e61-b0ec-5632b132c705-model-cache\") pod \"auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m\" (UID: \"d3fe6317-c46c-4e61-b0ec-5632b132c705\") " pod="ai-tenant-e2e-shared-b-5c7cfc/auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m"
Jul 24 17:50:06.164791 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:06.164775    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3fe6317-c46c-4e61-b0ec-5632b132c705-tmp-dir\") pod \"auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m\" (UID: \"d3fe6317-c46c-4e61-b0ec-5632b132c705\") " pod="ai-tenant-e2e-shared-b-5c7cfc/auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m"
Jul 24 17:50:06.164922 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:06.164892    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3fe6317-c46c-4e61-b0ec-5632b132c705-home\") pod \"auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m\" (UID: \"d3fe6317-c46c-4e61-b0ec-5632b132c705\") " pod="ai-tenant-e2e-shared-b-5c7cfc/auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m"
Jul 24 17:50:06.166808 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:06.166786    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3fe6317-c46c-4e61-b0ec-5632b132c705-dshm\") pod \"auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m\" (UID: \"d3fe6317-c46c-4e61-b0ec-5632b132c705\") " pod="ai-tenant-e2e-shared-b-5c7cfc/auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m"
Jul 24 17:50:06.167108 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:06.167089    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3fe6317-c46c-4e61-b0ec-5632b132c705-tls-certs\") pod \"auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m\" (UID: \"d3fe6317-c46c-4e61-b0ec-5632b132c705\") " pod="ai-tenant-e2e-shared-b-5c7cfc/auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m"
Jul 24 17:50:06.176328 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:06.176306    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmfj2\" (UniqueName: \"kubernetes.io/projected/d3fe6317-c46c-4e61-b0ec-5632b132c705-kube-api-access-cmfj2\") pod \"auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m\" (UID: \"d3fe6317-c46c-4e61-b0ec-5632b132c705\") " pod="ai-tenant-e2e-shared-b-5c7cfc/auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m"
Jul 24 17:50:06.295854 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:06.295750    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5c7cfc/auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m"
Jul 24 17:50:06.436070 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:06.436018    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-5c7cfc/auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m"]
Jul 24 17:50:06.437872 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:50:06.437831    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd3fe6317_c46c_4e61_b0ec_5632b132c705.slice/crio-0008a16cf01ba441eaa9fdd89383a697248187d7654e4edfd117725e850ebcb1 WatchSource:0}: Error finding container 0008a16cf01ba441eaa9fdd89383a697248187d7654e4edfd117725e850ebcb1: Status 404 returned error can't find the container with id 0008a16cf01ba441eaa9fdd89383a697248187d7654e4edfd117725e850ebcb1
Jul 24 17:50:07.086519 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:07.086485    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5c7cfc/auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m" event={"ID":"d3fe6317-c46c-4e61-b0ec-5632b132c705","Type":"ContainerStarted","Data":"fcc1369d62407dc683848184e91ab0fb700ea5ac3e1a2b19cfa3769e5b2087b7"}
Jul 24 17:50:07.086929 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:07.086530    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5c7cfc/auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m" event={"ID":"d3fe6317-c46c-4e61-b0ec-5632b132c705","Type":"ContainerStarted","Data":"0008a16cf01ba441eaa9fdd89383a697248187d7654e4edfd117725e850ebcb1"}
Jul 24 17:50:07.108182 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:07.108130    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-5c7cfc/auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m" podStartSLOduration=2.108113091 podStartE2EDuration="2.108113091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:50:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:50:07.106126772 +0000 UTC m=+2374.308176041" watchObservedRunningTime="2026-07-24 17:50:07.108113091 +0000 UTC m=+2374.310162365"
Jul 24 17:50:09.383547 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:09.383511    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:50:16.296208 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:16.296162    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-5c7cfc/auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m"
Jul 24 17:50:16.296637 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:16.296227    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-5c7cfc/auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m"
Jul 24 17:50:16.308662 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:16.308637    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-5c7cfc/auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m"
Jul 24 17:50:17.153682 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:17.153657    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-5c7cfc/auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m"
Jul 24 17:50:37.482791 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:37.482755    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:50:43.828381 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:43.828342    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:50:56.365951 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:50:56.365856    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:51:12.529921 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:51:12.529889    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:51:29.580508 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:51:29.580465    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:51:43.464914 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:51:43.464879    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:51:48.198368 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:51:48.198333    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:52:01.775769 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:01.775732    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:52:21.082108 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:21.082063    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:52:25.298668 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:25.298585    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-bb94b3/auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d"]
Jul 24 17:52:25.354602 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:25.354547    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:52:26.368511 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:26.368474    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-5c7cfc/auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m"]
Jul 24 17:52:26.397614 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:26.397577    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:52:38.470615 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:38.470580    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:52:40.318139 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:40.318095    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-bb94b3/auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d" podUID="b87c2542-3fbe-4eaa-9213-8ac3f35b5df5" containerName="main" containerID="cri-o://8c9e70e87cb46f26caf5b3ecf94f116dc4690be7b53c021b4a2536843d61f0e4" gracePeriod=45
Jul 24 17:52:40.568505 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:40.568443    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-bb94b3/auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d"
Jul 24 17:52:40.675385 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:40.675358    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b87c2542-3fbe-4eaa-9213-8ac3f35b5df5-model-cache\") pod \"b87c2542-3fbe-4eaa-9213-8ac3f35b5df5\" (UID: \"b87c2542-3fbe-4eaa-9213-8ac3f35b5df5\") "
Jul 24 17:52:40.675385 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:40.675390    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wp54p\" (UniqueName: \"kubernetes.io/projected/b87c2542-3fbe-4eaa-9213-8ac3f35b5df5-kube-api-access-wp54p\") pod \"b87c2542-3fbe-4eaa-9213-8ac3f35b5df5\" (UID: \"b87c2542-3fbe-4eaa-9213-8ac3f35b5df5\") "
Jul 24 17:52:40.675632 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:40.675413    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b87c2542-3fbe-4eaa-9213-8ac3f35b5df5-tmp-dir\") pod \"b87c2542-3fbe-4eaa-9213-8ac3f35b5df5\" (UID: \"b87c2542-3fbe-4eaa-9213-8ac3f35b5df5\") "
Jul 24 17:52:40.675632 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:40.675490    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b87c2542-3fbe-4eaa-9213-8ac3f35b5df5-tls-certs\") pod \"b87c2542-3fbe-4eaa-9213-8ac3f35b5df5\" (UID: \"b87c2542-3fbe-4eaa-9213-8ac3f35b5df5\") "
Jul 24 17:52:40.675632 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:40.675560    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b87c2542-3fbe-4eaa-9213-8ac3f35b5df5-dshm\") pod \"b87c2542-3fbe-4eaa-9213-8ac3f35b5df5\" (UID: \"b87c2542-3fbe-4eaa-9213-8ac3f35b5df5\") "
Jul 24 17:52:40.675632 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:40.675590    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b87c2542-3fbe-4eaa-9213-8ac3f35b5df5-home\") pod \"b87c2542-3fbe-4eaa-9213-8ac3f35b5df5\" (UID: \"b87c2542-3fbe-4eaa-9213-8ac3f35b5df5\") "
Jul 24 17:52:40.675852 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:40.675662    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b87c2542-3fbe-4eaa-9213-8ac3f35b5df5-model-cache" (OuterVolumeSpecName: "model-cache") pod "b87c2542-3fbe-4eaa-9213-8ac3f35b5df5" (UID: "b87c2542-3fbe-4eaa-9213-8ac3f35b5df5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:52:40.675852 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:40.675711    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b87c2542-3fbe-4eaa-9213-8ac3f35b5df5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b87c2542-3fbe-4eaa-9213-8ac3f35b5df5" (UID: "b87c2542-3fbe-4eaa-9213-8ac3f35b5df5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:52:40.675963 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:40.675870    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b87c2542-3fbe-4eaa-9213-8ac3f35b5df5-home" (OuterVolumeSpecName: "home") pod "b87c2542-3fbe-4eaa-9213-8ac3f35b5df5" (UID: "b87c2542-3fbe-4eaa-9213-8ac3f35b5df5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:52:40.675963 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:40.675944    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b87c2542-3fbe-4eaa-9213-8ac3f35b5df5-model-cache\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:52:40.676321 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:40.675967    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b87c2542-3fbe-4eaa-9213-8ac3f35b5df5-tmp-dir\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:52:40.677709 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:40.677687    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b87c2542-3fbe-4eaa-9213-8ac3f35b5df5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b87c2542-3fbe-4eaa-9213-8ac3f35b5df5" (UID: "b87c2542-3fbe-4eaa-9213-8ac3f35b5df5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:52:40.677818 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:40.677707    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b87c2542-3fbe-4eaa-9213-8ac3f35b5df5-dshm" (OuterVolumeSpecName: "dshm") pod "b87c2542-3fbe-4eaa-9213-8ac3f35b5df5" (UID: "b87c2542-3fbe-4eaa-9213-8ac3f35b5df5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:52:40.677818 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:40.677729    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b87c2542-3fbe-4eaa-9213-8ac3f35b5df5-kube-api-access-wp54p" (OuterVolumeSpecName: "kube-api-access-wp54p") pod "b87c2542-3fbe-4eaa-9213-8ac3f35b5df5" (UID: "b87c2542-3fbe-4eaa-9213-8ac3f35b5df5"). InnerVolumeSpecName "kube-api-access-wp54p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:52:40.776942 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:40.776903    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b87c2542-3fbe-4eaa-9213-8ac3f35b5df5-tls-certs\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:52:40.776942 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:40.776929    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b87c2542-3fbe-4eaa-9213-8ac3f35b5df5-dshm\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:52:40.776942 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:40.776936    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b87c2542-3fbe-4eaa-9213-8ac3f35b5df5-home\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:52:40.776942 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:40.776945    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wp54p\" (UniqueName: \"kubernetes.io/projected/b87c2542-3fbe-4eaa-9213-8ac3f35b5df5-kube-api-access-wp54p\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:52:40.875156 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:40.875003    2581 generic.go:356] "Generic (PLEG): container finished" podID="b87c2542-3fbe-4eaa-9213-8ac3f35b5df5" containerID="8c9e70e87cb46f26caf5b3ecf94f116dc4690be7b53c021b4a2536843d61f0e4" exitCode=0
Jul 24 17:52:40.875156 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:40.875093    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-bb94b3/auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d"
Jul 24 17:52:40.875156 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:40.875106    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-bb94b3/auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d" event={"ID":"b87c2542-3fbe-4eaa-9213-8ac3f35b5df5","Type":"ContainerDied","Data":"8c9e70e87cb46f26caf5b3ecf94f116dc4690be7b53c021b4a2536843d61f0e4"}
Jul 24 17:52:40.875465 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:40.875165    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-bb94b3/auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d" event={"ID":"b87c2542-3fbe-4eaa-9213-8ac3f35b5df5","Type":"ContainerDied","Data":"f8deea109a1761a81aa9dc9478772158d6b4514882775fdbeed477a55a2877c2"}
Jul 24 17:52:40.875465 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:40.875188    2581 scope.go:117] "RemoveContainer" containerID="8c9e70e87cb46f26caf5b3ecf94f116dc4690be7b53c021b4a2536843d61f0e4"
Jul 24 17:52:40.888278 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:40.888253    2581 scope.go:117] "RemoveContainer" containerID="8c9e70e87cb46f26caf5b3ecf94f116dc4690be7b53c021b4a2536843d61f0e4"
Jul 24 17:52:40.888584 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:52:40.888562    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c9e70e87cb46f26caf5b3ecf94f116dc4690be7b53c021b4a2536843d61f0e4\": container with ID starting with 8c9e70e87cb46f26caf5b3ecf94f116dc4690be7b53c021b4a2536843d61f0e4 not found: ID does not exist" containerID="8c9e70e87cb46f26caf5b3ecf94f116dc4690be7b53c021b4a2536843d61f0e4"
Jul 24 17:52:40.888658 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:40.888596    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c9e70e87cb46f26caf5b3ecf94f116dc4690be7b53c021b4a2536843d61f0e4"} err="failed to get container status \"8c9e70e87cb46f26caf5b3ecf94f116dc4690be7b53c021b4a2536843d61f0e4\": rpc error: code = NotFound desc = could not find container \"8c9e70e87cb46f26caf5b3ecf94f116dc4690be7b53c021b4a2536843d61f0e4\": container with ID starting with 8c9e70e87cb46f26caf5b3ecf94f116dc4690be7b53c021b4a2536843d61f0e4 not found: ID does not exist"
Jul 24 17:52:40.905053 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:40.905003    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-bb94b3/auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d"]
Jul 24 17:52:40.911952 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:40.911923    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-bb94b3/auth-test-model-bb94b3-kserve-65cbdd65f5-nx25d"]
Jul 24 17:52:41.383439 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:41.383408    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b87c2542-3fbe-4eaa-9213-8ac3f35b5df5" path="/var/lib/kubelet/pods/b87c2542-3fbe-4eaa-9213-8ac3f35b5df5/volumes"
Jul 24 17:52:41.387865 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:41.387832    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-5c7cfc/auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m" podUID="d3fe6317-c46c-4e61-b0ec-5632b132c705" containerName="main" containerID="cri-o://fcc1369d62407dc683848184e91ab0fb700ea5ac3e1a2b19cfa3769e5b2087b7" gracePeriod=45
Jul 24 17:52:41.640234 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:41.640174    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5c7cfc/auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m"
Jul 24 17:52:41.785913 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:41.785882    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3fe6317-c46c-4e61-b0ec-5632b132c705-model-cache\") pod \"d3fe6317-c46c-4e61-b0ec-5632b132c705\" (UID: \"d3fe6317-c46c-4e61-b0ec-5632b132c705\") "
Jul 24 17:52:41.785913 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:41.785917    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3fe6317-c46c-4e61-b0ec-5632b132c705-tls-certs\") pod \"d3fe6317-c46c-4e61-b0ec-5632b132c705\" (UID: \"d3fe6317-c46c-4e61-b0ec-5632b132c705\") "
Jul 24 17:52:41.786181 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:41.785958    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3fe6317-c46c-4e61-b0ec-5632b132c705-tmp-dir\") pod \"d3fe6317-c46c-4e61-b0ec-5632b132c705\" (UID: \"d3fe6317-c46c-4e61-b0ec-5632b132c705\") "
Jul 24 17:52:41.786181 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:41.785988    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3fe6317-c46c-4e61-b0ec-5632b132c705-dshm\") pod \"d3fe6317-c46c-4e61-b0ec-5632b132c705\" (UID: \"d3fe6317-c46c-4e61-b0ec-5632b132c705\") "
Jul 24 17:52:41.786181 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:41.786031    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cmfj2\" (UniqueName: \"kubernetes.io/projected/d3fe6317-c46c-4e61-b0ec-5632b132c705-kube-api-access-cmfj2\") pod \"d3fe6317-c46c-4e61-b0ec-5632b132c705\" (UID: \"d3fe6317-c46c-4e61-b0ec-5632b132c705\") "
Jul 24 17:52:41.786181 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:41.786100    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3fe6317-c46c-4e61-b0ec-5632b132c705-home\") pod \"d3fe6317-c46c-4e61-b0ec-5632b132c705\" (UID: \"d3fe6317-c46c-4e61-b0ec-5632b132c705\") "
Jul 24 17:52:41.786404 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:41.786162    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3fe6317-c46c-4e61-b0ec-5632b132c705-model-cache" (OuterVolumeSpecName: "model-cache") pod "d3fe6317-c46c-4e61-b0ec-5632b132c705" (UID: "d3fe6317-c46c-4e61-b0ec-5632b132c705"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:52:41.786404 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:41.786237    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3fe6317-c46c-4e61-b0ec-5632b132c705-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d3fe6317-c46c-4e61-b0ec-5632b132c705" (UID: "d3fe6317-c46c-4e61-b0ec-5632b132c705"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:52:41.786404 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:41.786388    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3fe6317-c46c-4e61-b0ec-5632b132c705-home" (OuterVolumeSpecName: "home") pod "d3fe6317-c46c-4e61-b0ec-5632b132c705" (UID: "d3fe6317-c46c-4e61-b0ec-5632b132c705"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:52:41.786509 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:41.786423    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3fe6317-c46c-4e61-b0ec-5632b132c705-model-cache\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:52:41.786509 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:41.786439    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3fe6317-c46c-4e61-b0ec-5632b132c705-tmp-dir\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:52:41.788338 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:41.788315    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3fe6317-c46c-4e61-b0ec-5632b132c705-dshm" (OuterVolumeSpecName: "dshm") pod "d3fe6317-c46c-4e61-b0ec-5632b132c705" (UID: "d3fe6317-c46c-4e61-b0ec-5632b132c705"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:52:41.788445 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:41.788337    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3fe6317-c46c-4e61-b0ec-5632b132c705-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d3fe6317-c46c-4e61-b0ec-5632b132c705" (UID: "d3fe6317-c46c-4e61-b0ec-5632b132c705"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:52:41.788445 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:41.788373    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3fe6317-c46c-4e61-b0ec-5632b132c705-kube-api-access-cmfj2" (OuterVolumeSpecName: "kube-api-access-cmfj2") pod "d3fe6317-c46c-4e61-b0ec-5632b132c705" (UID: "d3fe6317-c46c-4e61-b0ec-5632b132c705"). InnerVolumeSpecName "kube-api-access-cmfj2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:52:41.882123 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:41.882082    2581 generic.go:356] "Generic (PLEG): container finished" podID="d3fe6317-c46c-4e61-b0ec-5632b132c705" containerID="fcc1369d62407dc683848184e91ab0fb700ea5ac3e1a2b19cfa3769e5b2087b7" exitCode=0
Jul 24 17:52:41.882317 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:41.882130    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5c7cfc/auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m"
Jul 24 17:52:41.882317 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:41.882132    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5c7cfc/auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m" event={"ID":"d3fe6317-c46c-4e61-b0ec-5632b132c705","Type":"ContainerDied","Data":"fcc1369d62407dc683848184e91ab0fb700ea5ac3e1a2b19cfa3769e5b2087b7"}
Jul 24 17:52:41.882317 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:41.882265    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5c7cfc/auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m" event={"ID":"d3fe6317-c46c-4e61-b0ec-5632b132c705","Type":"ContainerDied","Data":"0008a16cf01ba441eaa9fdd89383a697248187d7654e4edfd117725e850ebcb1"}
Jul 24 17:52:41.882317 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:41.882296    2581 scope.go:117] "RemoveContainer" containerID="fcc1369d62407dc683848184e91ab0fb700ea5ac3e1a2b19cfa3769e5b2087b7"
Jul 24 17:52:41.887401 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:41.887375    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3fe6317-c46c-4e61-b0ec-5632b132c705-tls-certs\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:52:41.887401 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:41.887401    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3fe6317-c46c-4e61-b0ec-5632b132c705-dshm\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:52:41.887589 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:41.887411    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cmfj2\" (UniqueName: \"kubernetes.io/projected/d3fe6317-c46c-4e61-b0ec-5632b132c705-kube-api-access-cmfj2\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:52:41.887589 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:41.887418    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3fe6317-c46c-4e61-b0ec-5632b132c705-home\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:52:41.891609 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:41.891594    2581 scope.go:117] "RemoveContainer" containerID="fcc1369d62407dc683848184e91ab0fb700ea5ac3e1a2b19cfa3769e5b2087b7"
Jul 24 17:52:41.891836 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:52:41.891820    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcc1369d62407dc683848184e91ab0fb700ea5ac3e1a2b19cfa3769e5b2087b7\": container with ID starting with fcc1369d62407dc683848184e91ab0fb700ea5ac3e1a2b19cfa3769e5b2087b7 not found: ID does not exist" containerID="fcc1369d62407dc683848184e91ab0fb700ea5ac3e1a2b19cfa3769e5b2087b7"
Jul 24 17:52:41.891875 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:41.891845    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcc1369d62407dc683848184e91ab0fb700ea5ac3e1a2b19cfa3769e5b2087b7"} err="failed to get container status \"fcc1369d62407dc683848184e91ab0fb700ea5ac3e1a2b19cfa3769e5b2087b7\": rpc error: code = NotFound desc = could not find container \"fcc1369d62407dc683848184e91ab0fb700ea5ac3e1a2b19cfa3769e5b2087b7\": container with ID starting with fcc1369d62407dc683848184e91ab0fb700ea5ac3e1a2b19cfa3769e5b2087b7 not found: ID does not exist"
Jul 24 17:52:41.917167 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:41.917130    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-5c7cfc/auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m"]
Jul 24 17:52:41.923551 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:41.923529    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-5c7cfc/auth-test-model-5c7cfc-kserve-f65b6c899-sjf7m"]
Jul 24 17:52:43.384197 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:52:43.384166    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3fe6317-c46c-4e61-b0ec-5632b132c705" path="/var/lib/kubelet/pods/d3fe6317-c46c-4e61-b0ec-5632b132c705/volumes"
Jul 24 17:53:00.745121 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:00.745080    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-bb94b3/sub-test-model-bb94b3-kserve-8658946795-268jn"]
Jul 24 17:53:00.745619 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:00.745483    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b87c2542-3fbe-4eaa-9213-8ac3f35b5df5" containerName="main"
Jul 24 17:53:00.745619 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:00.745493    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b87c2542-3fbe-4eaa-9213-8ac3f35b5df5" containerName="main"
Jul 24 17:53:00.745619 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:00.745503    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3fe6317-c46c-4e61-b0ec-5632b132c705" containerName="main"
Jul 24 17:53:00.745619 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:00.745509    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3fe6317-c46c-4e61-b0ec-5632b132c705" containerName="main"
Jul 24 17:53:00.745619 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:00.745575    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3fe6317-c46c-4e61-b0ec-5632b132c705" containerName="main"
Jul 24 17:53:00.745619 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:00.745583    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b87c2542-3fbe-4eaa-9213-8ac3f35b5df5" containerName="main"
Jul 24 17:53:00.749771 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:00.749750    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-bb94b3/sub-test-model-bb94b3-kserve-8658946795-268jn"
Jul 24 17:53:00.753667 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:00.753642    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-bb94b3\"/\"kube-root-ca.crt\""
Jul 24 17:53:00.754637 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:00.754614    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-bb94b3\"/\"openshift-service-ca.crt\""
Jul 24 17:53:00.755014 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:00.754995    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-bb94b3\"/\"default-dockercfg-kddnf\""
Jul 24 17:53:00.755081 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:00.755005    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-bb94b3\"/\"sub-test-model-bb94b3-kserve-self-signed-certs\""
Jul 24 17:53:00.765200 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:00.765168    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sk5jx\" (UniqueName: \"kubernetes.io/projected/2b3eec05-7eee-4d52-8342-9a001f645b33-kube-api-access-sk5jx\") pod \"sub-test-model-bb94b3-kserve-8658946795-268jn\" (UID: \"2b3eec05-7eee-4d52-8342-9a001f645b33\") " pod="ai-tenant-e2e-shared-a-bb94b3/sub-test-model-bb94b3-kserve-8658946795-268jn"
Jul 24 17:53:00.765356 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:00.765209    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b3eec05-7eee-4d52-8342-9a001f645b33-home\") pod \"sub-test-model-bb94b3-kserve-8658946795-268jn\" (UID: \"2b3eec05-7eee-4d52-8342-9a001f645b33\") " pod="ai-tenant-e2e-shared-a-bb94b3/sub-test-model-bb94b3-kserve-8658946795-268jn"
Jul 24 17:53:00.765356 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:00.765234    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b3eec05-7eee-4d52-8342-9a001f645b33-dshm\") pod \"sub-test-model-bb94b3-kserve-8658946795-268jn\" (UID: \"2b3eec05-7eee-4d52-8342-9a001f645b33\") " pod="ai-tenant-e2e-shared-a-bb94b3/sub-test-model-bb94b3-kserve-8658946795-268jn"
Jul 24 17:53:00.765356 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:00.765284    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b3eec05-7eee-4d52-8342-9a001f645b33-model-cache\") pod \"sub-test-model-bb94b3-kserve-8658946795-268jn\" (UID: \"2b3eec05-7eee-4d52-8342-9a001f645b33\") " pod="ai-tenant-e2e-shared-a-bb94b3/sub-test-model-bb94b3-kserve-8658946795-268jn"
Jul 24 17:53:00.765356 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:00.765328    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b3eec05-7eee-4d52-8342-9a001f645b33-tmp-dir\") pod \"sub-test-model-bb94b3-kserve-8658946795-268jn\" (UID: \"2b3eec05-7eee-4d52-8342-9a001f645b33\") " pod="ai-tenant-e2e-shared-a-bb94b3/sub-test-model-bb94b3-kserve-8658946795-268jn"
Jul 24 17:53:00.765531 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:00.765360    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b3eec05-7eee-4d52-8342-9a001f645b33-tls-certs\") pod \"sub-test-model-bb94b3-kserve-8658946795-268jn\" (UID: \"2b3eec05-7eee-4d52-8342-9a001f645b33\") " pod="ai-tenant-e2e-shared-a-bb94b3/sub-test-model-bb94b3-kserve-8658946795-268jn"
Jul 24 17:53:00.793846 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:00.793816    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-bb94b3/sub-test-model-bb94b3-kserve-8658946795-268jn"]
Jul 24 17:53:00.809152 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:00.809124    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:53:00.866605 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:00.866559    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b3eec05-7eee-4d52-8342-9a001f645b33-tmp-dir\") pod \"sub-test-model-bb94b3-kserve-8658946795-268jn\" (UID: \"2b3eec05-7eee-4d52-8342-9a001f645b33\") " pod="ai-tenant-e2e-shared-a-bb94b3/sub-test-model-bb94b3-kserve-8658946795-268jn"
Jul 24 17:53:00.866605 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:00.866607    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b3eec05-7eee-4d52-8342-9a001f645b33-tls-certs\") pod \"sub-test-model-bb94b3-kserve-8658946795-268jn\" (UID: \"2b3eec05-7eee-4d52-8342-9a001f645b33\") " pod="ai-tenant-e2e-shared-a-bb94b3/sub-test-model-bb94b3-kserve-8658946795-268jn"
Jul 24 17:53:00.866889 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:00.866684    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk5jx\" (UniqueName: \"kubernetes.io/projected/2b3eec05-7eee-4d52-8342-9a001f645b33-kube-api-access-sk5jx\") pod \"sub-test-model-bb94b3-kserve-8658946795-268jn\" (UID: \"2b3eec05-7eee-4d52-8342-9a001f645b33\") " pod="ai-tenant-e2e-shared-a-bb94b3/sub-test-model-bb94b3-kserve-8658946795-268jn"
Jul 24 17:53:00.866889 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:00.866710    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b3eec05-7eee-4d52-8342-9a001f645b33-home\") pod \"sub-test-model-bb94b3-kserve-8658946795-268jn\" (UID: \"2b3eec05-7eee-4d52-8342-9a001f645b33\") " pod="ai-tenant-e2e-shared-a-bb94b3/sub-test-model-bb94b3-kserve-8658946795-268jn"
Jul 24 17:53:00.866889 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:00.866731    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b3eec05-7eee-4d52-8342-9a001f645b33-dshm\") pod \"sub-test-model-bb94b3-kserve-8658946795-268jn\" (UID: \"2b3eec05-7eee-4d52-8342-9a001f645b33\") " pod="ai-tenant-e2e-shared-a-bb94b3/sub-test-model-bb94b3-kserve-8658946795-268jn"
Jul 24 17:53:00.866889 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:00.866748    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b3eec05-7eee-4d52-8342-9a001f645b33-model-cache\") pod \"sub-test-model-bb94b3-kserve-8658946795-268jn\" (UID: \"2b3eec05-7eee-4d52-8342-9a001f645b33\") " pod="ai-tenant-e2e-shared-a-bb94b3/sub-test-model-bb94b3-kserve-8658946795-268jn"
Jul 24 17:53:00.867149 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:00.867030    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b3eec05-7eee-4d52-8342-9a001f645b33-tmp-dir\") pod \"sub-test-model-bb94b3-kserve-8658946795-268jn\" (UID: \"2b3eec05-7eee-4d52-8342-9a001f645b33\") " pod="ai-tenant-e2e-shared-a-bb94b3/sub-test-model-bb94b3-kserve-8658946795-268jn"
Jul 24 17:53:00.867149 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:00.867101    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b3eec05-7eee-4d52-8342-9a001f645b33-home\") pod \"sub-test-model-bb94b3-kserve-8658946795-268jn\" (UID: \"2b3eec05-7eee-4d52-8342-9a001f645b33\") " pod="ai-tenant-e2e-shared-a-bb94b3/sub-test-model-bb94b3-kserve-8658946795-268jn"
Jul 24 17:53:00.867149 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:00.867121    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b3eec05-7eee-4d52-8342-9a001f645b33-model-cache\") pod \"sub-test-model-bb94b3-kserve-8658946795-268jn\" (UID: \"2b3eec05-7eee-4d52-8342-9a001f645b33\") " pod="ai-tenant-e2e-shared-a-bb94b3/sub-test-model-bb94b3-kserve-8658946795-268jn"
Jul 24 17:53:00.869247 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:00.869221    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b3eec05-7eee-4d52-8342-9a001f645b33-dshm\") pod \"sub-test-model-bb94b3-kserve-8658946795-268jn\" (UID: \"2b3eec05-7eee-4d52-8342-9a001f645b33\") " pod="ai-tenant-e2e-shared-a-bb94b3/sub-test-model-bb94b3-kserve-8658946795-268jn"
Jul 24 17:53:00.869438 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:00.869418    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b3eec05-7eee-4d52-8342-9a001f645b33-tls-certs\") pod \"sub-test-model-bb94b3-kserve-8658946795-268jn\" (UID: \"2b3eec05-7eee-4d52-8342-9a001f645b33\") " pod="ai-tenant-e2e-shared-a-bb94b3/sub-test-model-bb94b3-kserve-8658946795-268jn"
Jul 24 17:53:00.877264 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:00.877239    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sk5jx\" (UniqueName: \"kubernetes.io/projected/2b3eec05-7eee-4d52-8342-9a001f645b33-kube-api-access-sk5jx\") pod \"sub-test-model-bb94b3-kserve-8658946795-268jn\" (UID: \"2b3eec05-7eee-4d52-8342-9a001f645b33\") " pod="ai-tenant-e2e-shared-a-bb94b3/sub-test-model-bb94b3-kserve-8658946795-268jn"
Jul 24 17:53:01.059721 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:01.059630    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-bb94b3/sub-test-model-bb94b3-kserve-8658946795-268jn"
Jul 24 17:53:01.215327 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:01.215252    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-bb94b3/sub-test-model-bb94b3-kserve-8658946795-268jn"]
Jul 24 17:53:01.218365 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:53:01.218340    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b3eec05_7eee_4d52_8342_9a001f645b33.slice/crio-74c457b6fea15bd62107c5b464c4b0340851d31bca621b061760c8a298e70429 WatchSource:0}: Error finding container 74c457b6fea15bd62107c5b464c4b0340851d31bca621b061760c8a298e70429: Status 404 returned error can't find the container with id 74c457b6fea15bd62107c5b464c4b0340851d31bca621b061760c8a298e70429
Jul 24 17:53:01.990800 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:01.990764    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-bb94b3/sub-test-model-bb94b3-kserve-8658946795-268jn" event={"ID":"2b3eec05-7eee-4d52-8342-9a001f645b33","Type":"ContainerStarted","Data":"36db89b3795382c363e4f3f277fc3e60e8c02d0957500b9a2658dc7ed02488b9"}
Jul 24 17:53:01.990800 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:01.990805    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-bb94b3/sub-test-model-bb94b3-kserve-8658946795-268jn" event={"ID":"2b3eec05-7eee-4d52-8342-9a001f645b33","Type":"ContainerStarted","Data":"74c457b6fea15bd62107c5b464c4b0340851d31bca621b061760c8a298e70429"}
Jul 24 17:53:02.025668 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:02.025609    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-bb94b3/sub-test-model-bb94b3-kserve-8658946795-268jn" podStartSLOduration=2.025591715 podStartE2EDuration="2.025591715s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:53:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:53:02.024537039 +0000 UTC m=+2549.226586314" watchObservedRunningTime="2026-07-24 17:53:02.025591715 +0000 UTC m=+2549.227640988"
Jul 24 17:53:04.389875 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:04.389821    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:53:11.060554 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:11.060523    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-bb94b3/sub-test-model-bb94b3-kserve-8658946795-268jn"
Jul 24 17:53:11.060982 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:11.060580    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-bb94b3/sub-test-model-bb94b3-kserve-8658946795-268jn"
Jul 24 17:53:11.073241 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:11.073218    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-bb94b3/sub-test-model-bb94b3-kserve-8658946795-268jn"
Jul 24 17:53:12.049782 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:12.049753    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-bb94b3/sub-test-model-bb94b3-kserve-8658946795-268jn"
Jul 24 17:53:19.655659 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:19.655630    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-5c7cfc/sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n"]
Jul 24 17:53:19.658981 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:19.658963    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5c7cfc/sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n"
Jul 24 17:53:19.663009 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:19.662982    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-5c7cfc\"/\"openshift-service-ca.crt\""
Jul 24 17:53:19.663009 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:19.663004    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-5c7cfc\"/\"kube-root-ca.crt\""
Jul 24 17:53:19.663203 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:19.662979    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-5c7cfc\"/\"sub-test-model-5c7cfc-kserve-self-signed-certs\""
Jul 24 17:53:19.663203 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:19.662981    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-5c7cfc\"/\"default-dockercfg-hkndk\""
Jul 24 17:53:19.670027 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:19.670004    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-5c7cfc/sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n"]
Jul 24 17:53:19.683365 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:19.683313    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:53:19.752576 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:19.752541    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1575d87c-d212-4866-854f-818dd6899f87-model-cache\") pod \"sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n\" (UID: \"1575d87c-d212-4866-854f-818dd6899f87\") " pod="ai-tenant-e2e-shared-b-5c7cfc/sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n"
Jul 24 17:53:19.752747 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:19.752603    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmlfq\" (UniqueName: \"kubernetes.io/projected/1575d87c-d212-4866-854f-818dd6899f87-kube-api-access-cmlfq\") pod \"sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n\" (UID: \"1575d87c-d212-4866-854f-818dd6899f87\") " pod="ai-tenant-e2e-shared-b-5c7cfc/sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n"
Jul 24 17:53:19.752747 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:19.752657    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1575d87c-d212-4866-854f-818dd6899f87-tmp-dir\") pod \"sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n\" (UID: \"1575d87c-d212-4866-854f-818dd6899f87\") " pod="ai-tenant-e2e-shared-b-5c7cfc/sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n"
Jul 24 17:53:19.752747 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:19.752701    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1575d87c-d212-4866-854f-818dd6899f87-dshm\") pod \"sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n\" (UID: \"1575d87c-d212-4866-854f-818dd6899f87\") " pod="ai-tenant-e2e-shared-b-5c7cfc/sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n"
Jul 24 17:53:19.752747 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:19.752736    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1575d87c-d212-4866-854f-818dd6899f87-home\") pod \"sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n\" (UID: \"1575d87c-d212-4866-854f-818dd6899f87\") " pod="ai-tenant-e2e-shared-b-5c7cfc/sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n"
Jul 24 17:53:19.752891 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:19.752755    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1575d87c-d212-4866-854f-818dd6899f87-tls-certs\") pod \"sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n\" (UID: \"1575d87c-d212-4866-854f-818dd6899f87\") " pod="ai-tenant-e2e-shared-b-5c7cfc/sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n"
Jul 24 17:53:19.853479 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:19.853439    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1575d87c-d212-4866-854f-818dd6899f87-home\") pod \"sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n\" (UID: \"1575d87c-d212-4866-854f-818dd6899f87\") " pod="ai-tenant-e2e-shared-b-5c7cfc/sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n"
Jul 24 17:53:19.853479 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:19.853486    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1575d87c-d212-4866-854f-818dd6899f87-tls-certs\") pod \"sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n\" (UID: \"1575d87c-d212-4866-854f-818dd6899f87\") " pod="ai-tenant-e2e-shared-b-5c7cfc/sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n"
Jul 24 17:53:19.853701 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:19.853558    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1575d87c-d212-4866-854f-818dd6899f87-model-cache\") pod \"sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n\" (UID: \"1575d87c-d212-4866-854f-818dd6899f87\") " pod="ai-tenant-e2e-shared-b-5c7cfc/sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n"
Jul 24 17:53:19.853701 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:19.853603    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmlfq\" (UniqueName: \"kubernetes.io/projected/1575d87c-d212-4866-854f-818dd6899f87-kube-api-access-cmlfq\") pod \"sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n\" (UID: \"1575d87c-d212-4866-854f-818dd6899f87\") " pod="ai-tenant-e2e-shared-b-5c7cfc/sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n"
Jul 24 17:53:19.853701 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:19.853626    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1575d87c-d212-4866-854f-818dd6899f87-tmp-dir\") pod \"sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n\" (UID: \"1575d87c-d212-4866-854f-818dd6899f87\") " pod="ai-tenant-e2e-shared-b-5c7cfc/sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n"
Jul 24 17:53:19.853701 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:19.853652    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1575d87c-d212-4866-854f-818dd6899f87-dshm\") pod \"sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n\" (UID: \"1575d87c-d212-4866-854f-818dd6899f87\") " pod="ai-tenant-e2e-shared-b-5c7cfc/sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n"
Jul 24 17:53:19.853914 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:19.853867    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1575d87c-d212-4866-854f-818dd6899f87-home\") pod \"sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n\" (UID: \"1575d87c-d212-4866-854f-818dd6899f87\") " pod="ai-tenant-e2e-shared-b-5c7cfc/sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n"
Jul 24 17:53:19.854032 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:19.854011    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1575d87c-d212-4866-854f-818dd6899f87-model-cache\") pod \"sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n\" (UID: \"1575d87c-d212-4866-854f-818dd6899f87\") " pod="ai-tenant-e2e-shared-b-5c7cfc/sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n"
Jul 24 17:53:19.854141 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:19.854090    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1575d87c-d212-4866-854f-818dd6899f87-tmp-dir\") pod \"sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n\" (UID: \"1575d87c-d212-4866-854f-818dd6899f87\") " pod="ai-tenant-e2e-shared-b-5c7cfc/sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n"
Jul 24 17:53:19.855945 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:19.855926    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1575d87c-d212-4866-854f-818dd6899f87-dshm\") pod \"sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n\" (UID: \"1575d87c-d212-4866-854f-818dd6899f87\") " pod="ai-tenant-e2e-shared-b-5c7cfc/sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n"
Jul 24 17:53:19.856229 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:19.856213    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1575d87c-d212-4866-854f-818dd6899f87-tls-certs\") pod \"sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n\" (UID: \"1575d87c-d212-4866-854f-818dd6899f87\") " pod="ai-tenant-e2e-shared-b-5c7cfc/sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n"
Jul 24 17:53:19.861975 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:19.861953    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmlfq\" (UniqueName: \"kubernetes.io/projected/1575d87c-d212-4866-854f-818dd6899f87-kube-api-access-cmlfq\") pod \"sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n\" (UID: \"1575d87c-d212-4866-854f-818dd6899f87\") " pod="ai-tenant-e2e-shared-b-5c7cfc/sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n"
Jul 24 17:53:19.969060 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:19.968951    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5c7cfc/sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n"
Jul 24 17:53:20.110926 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:20.110901    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-5c7cfc/sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n"]
Jul 24 17:53:20.112191 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:53:20.112165    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1575d87c_d212_4866_854f_818dd6899f87.slice/crio-be89c24fb9b6505fcab6e4b0fd5caa7b41e1b8d1ba55646cd975d762b55b8b2f WatchSource:0}: Error finding container be89c24fb9b6505fcab6e4b0fd5caa7b41e1b8d1ba55646cd975d762b55b8b2f: Status 404 returned error can't find the container with id be89c24fb9b6505fcab6e4b0fd5caa7b41e1b8d1ba55646cd975d762b55b8b2f
Jul 24 17:53:21.087872 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:21.087829    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5c7cfc/sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n" event={"ID":"1575d87c-d212-4866-854f-818dd6899f87","Type":"ContainerStarted","Data":"1f4fe4ad1d804850b9816d71341ecbd384af8af149b157ed711bf2ab0b3c83ff"}
Jul 24 17:53:21.087872 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:21.087876    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5c7cfc/sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n" event={"ID":"1575d87c-d212-4866-854f-818dd6899f87","Type":"ContainerStarted","Data":"be89c24fb9b6505fcab6e4b0fd5caa7b41e1b8d1ba55646cd975d762b55b8b2f"}
Jul 24 17:53:23.264311 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:23.264247    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-5c7cfc/sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n" podStartSLOduration=4.264224241 podStartE2EDuration="4.264224241s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:53:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:53:21.123120567 +0000 UTC m=+2568.325169841" watchObservedRunningTime="2026-07-24 17:53:23.264224241 +0000 UTC m=+2570.466273519"
Jul 24 17:53:23.265596 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:23.265573    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:53:29.969238 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:29.969200    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-5c7cfc/sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n"
Jul 24 17:53:29.969626 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:29.969255    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-5c7cfc/sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n"
Jul 24 17:53:29.981713 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:29.981683    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-5c7cfc/sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n"
Jul 24 17:53:30.144466 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:30.144435    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-5c7cfc/sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n"
Jul 24 17:53:58.758424 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:53:58.758346    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:54:17.959429 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:17.959392    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-bb94b3/sub-test-model-bb94b3-kserve-8658946795-268jn"]
Jul 24 17:54:17.983170 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:17.983137    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:54:18.874902 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:18.874866    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-5c7cfc/sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n"]
Jul 24 17:54:18.920471 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:18.920440    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:54:23.059963 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:23.059921    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:54:29.466962 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:29.466929    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-bb94b3/rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6"]
Jul 24 17:54:29.470316 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:29.470295    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-bb94b3/rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6"
Jul 24 17:54:29.473138 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:29.473114    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-bb94b3\"/\"rate-test-model-bb94b3-kserve-self-signed-certs\""
Jul 24 17:54:29.480469 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:29.480446    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-bb94b3/rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6"]
Jul 24 17:54:29.547827 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:29.547783    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:54:29.639598 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:29.639562    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d526063-a78d-4114-8bec-f3aeb33a5860-model-cache\") pod \"rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6\" (UID: \"4d526063-a78d-4114-8bec-f3aeb33a5860\") " pod="ai-tenant-e2e-shared-a-bb94b3/rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6"
Jul 24 17:54:29.639776 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:29.639667    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d526063-a78d-4114-8bec-f3aeb33a5860-tmp-dir\") pod \"rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6\" (UID: \"4d526063-a78d-4114-8bec-f3aeb33a5860\") " pod="ai-tenant-e2e-shared-a-bb94b3/rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6"
Jul 24 17:54:29.639776 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:29.639691    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45l2d\" (UniqueName: \"kubernetes.io/projected/4d526063-a78d-4114-8bec-f3aeb33a5860-kube-api-access-45l2d\") pod \"rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6\" (UID: \"4d526063-a78d-4114-8bec-f3aeb33a5860\") " pod="ai-tenant-e2e-shared-a-bb94b3/rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6"
Jul 24 17:54:29.639776 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:29.639720    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d526063-a78d-4114-8bec-f3aeb33a5860-dshm\") pod \"rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6\" (UID: \"4d526063-a78d-4114-8bec-f3aeb33a5860\") " pod="ai-tenant-e2e-shared-a-bb94b3/rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6"
Jul 24 17:54:29.639776 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:29.639768    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d526063-a78d-4114-8bec-f3aeb33a5860-home\") pod \"rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6\" (UID: \"4d526063-a78d-4114-8bec-f3aeb33a5860\") " pod="ai-tenant-e2e-shared-a-bb94b3/rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6"
Jul 24 17:54:29.639915 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:29.639795    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d526063-a78d-4114-8bec-f3aeb33a5860-tls-certs\") pod \"rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6\" (UID: \"4d526063-a78d-4114-8bec-f3aeb33a5860\") " pod="ai-tenant-e2e-shared-a-bb94b3/rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6"
Jul 24 17:54:29.740494 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:29.740421    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d526063-a78d-4114-8bec-f3aeb33a5860-model-cache\") pod \"rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6\" (UID: \"4d526063-a78d-4114-8bec-f3aeb33a5860\") " pod="ai-tenant-e2e-shared-a-bb94b3/rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6"
Jul 24 17:54:29.740494 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:29.740489    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d526063-a78d-4114-8bec-f3aeb33a5860-tmp-dir\") pod \"rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6\" (UID: \"4d526063-a78d-4114-8bec-f3aeb33a5860\") " pod="ai-tenant-e2e-shared-a-bb94b3/rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6"
Jul 24 17:54:29.740720 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:29.740511    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45l2d\" (UniqueName: \"kubernetes.io/projected/4d526063-a78d-4114-8bec-f3aeb33a5860-kube-api-access-45l2d\") pod \"rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6\" (UID: \"4d526063-a78d-4114-8bec-f3aeb33a5860\") " pod="ai-tenant-e2e-shared-a-bb94b3/rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6"
Jul 24 17:54:29.740720 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:29.740532    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d526063-a78d-4114-8bec-f3aeb33a5860-dshm\") pod \"rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6\" (UID: \"4d526063-a78d-4114-8bec-f3aeb33a5860\") " pod="ai-tenant-e2e-shared-a-bb94b3/rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6"
Jul 24 17:54:29.740720 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:29.740567    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d526063-a78d-4114-8bec-f3aeb33a5860-home\") pod \"rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6\" (UID: \"4d526063-a78d-4114-8bec-f3aeb33a5860\") " pod="ai-tenant-e2e-shared-a-bb94b3/rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6"
Jul 24 17:54:29.740720 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:29.740592    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d526063-a78d-4114-8bec-f3aeb33a5860-tls-certs\") pod \"rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6\" (UID: \"4d526063-a78d-4114-8bec-f3aeb33a5860\") " pod="ai-tenant-e2e-shared-a-bb94b3/rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6"
Jul 24 17:54:29.740917 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:29.740816    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d526063-a78d-4114-8bec-f3aeb33a5860-model-cache\") pod \"rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6\" (UID: \"4d526063-a78d-4114-8bec-f3aeb33a5860\") " pod="ai-tenant-e2e-shared-a-bb94b3/rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6"
Jul 24 17:54:29.740917 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:29.740886    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d526063-a78d-4114-8bec-f3aeb33a5860-tmp-dir\") pod \"rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6\" (UID: \"4d526063-a78d-4114-8bec-f3aeb33a5860\") " pod="ai-tenant-e2e-shared-a-bb94b3/rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6"
Jul 24 17:54:29.740986 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:29.740959    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d526063-a78d-4114-8bec-f3aeb33a5860-home\") pod \"rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6\" (UID: \"4d526063-a78d-4114-8bec-f3aeb33a5860\") " pod="ai-tenant-e2e-shared-a-bb94b3/rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6"
Jul 24 17:54:29.743013 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:29.742989    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d526063-a78d-4114-8bec-f3aeb33a5860-dshm\") pod \"rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6\" (UID: \"4d526063-a78d-4114-8bec-f3aeb33a5860\") " pod="ai-tenant-e2e-shared-a-bb94b3/rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6"
Jul 24 17:54:29.743258 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:29.743243    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d526063-a78d-4114-8bec-f3aeb33a5860-tls-certs\") pod \"rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6\" (UID: \"4d526063-a78d-4114-8bec-f3aeb33a5860\") " pod="ai-tenant-e2e-shared-a-bb94b3/rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6"
Jul 24 17:54:29.749048 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:29.749016    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45l2d\" (UniqueName: \"kubernetes.io/projected/4d526063-a78d-4114-8bec-f3aeb33a5860-kube-api-access-45l2d\") pod \"rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6\" (UID: \"4d526063-a78d-4114-8bec-f3aeb33a5860\") " pod="ai-tenant-e2e-shared-a-bb94b3/rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6"
Jul 24 17:54:29.780541 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:29.780520    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-bb94b3/rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6"
Jul 24 17:54:29.915850 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:29.915819    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-bb94b3/rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6"]
Jul 24 17:54:29.918985 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:54:29.918956    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4d526063_a78d_4114_8bec_f3aeb33a5860.slice/crio-369c8150aef1fd916e9da5d129549889c13bc124ead5e91e03038a942531a4bb WatchSource:0}: Error finding container 369c8150aef1fd916e9da5d129549889c13bc124ead5e91e03038a942531a4bb: Status 404 returned error can't find the container with id 369c8150aef1fd916e9da5d129549889c13bc124ead5e91e03038a942531a4bb
Jul 24 17:54:30.448204 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:30.448161    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-bb94b3/rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6" event={"ID":"4d526063-a78d-4114-8bec-f3aeb33a5860","Type":"ContainerStarted","Data":"17594eb61b34d78be759d1c1dce2e9610e210c65242d3aebbf27dc32d4b5c11a"}
Jul 24 17:54:30.448204 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:30.448208    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-bb94b3/rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6" event={"ID":"4d526063-a78d-4114-8bec-f3aeb33a5860","Type":"ContainerStarted","Data":"369c8150aef1fd916e9da5d129549889c13bc124ead5e91e03038a942531a4bb"}
Jul 24 17:54:30.470693 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:30.470638    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-bb94b3/rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6" podStartSLOduration=1.470623988 podStartE2EDuration="1.470623988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:54:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:54:30.468778479 +0000 UTC m=+2637.670827752" watchObservedRunningTime="2026-07-24 17:54:30.470623988 +0000 UTC m=+2637.672673261"
Jul 24 17:54:32.979051 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:32.978958    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-bb94b3/sub-test-model-bb94b3-kserve-8658946795-268jn" podUID="2b3eec05-7eee-4d52-8342-9a001f645b33" containerName="main" containerID="cri-o://36db89b3795382c363e4f3f277fc3e60e8c02d0957500b9a2658dc7ed02488b9" gracePeriod=45
Jul 24 17:54:33.238407 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:33.238342    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-bb94b3/sub-test-model-bb94b3-kserve-8658946795-268jn"
Jul 24 17:54:33.377443 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:33.377409    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b3eec05-7eee-4d52-8342-9a001f645b33-home\") pod \"2b3eec05-7eee-4d52-8342-9a001f645b33\" (UID: \"2b3eec05-7eee-4d52-8342-9a001f645b33\") "
Jul 24 17:54:33.377626 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:33.377517    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sk5jx\" (UniqueName: \"kubernetes.io/projected/2b3eec05-7eee-4d52-8342-9a001f645b33-kube-api-access-sk5jx\") pod \"2b3eec05-7eee-4d52-8342-9a001f645b33\" (UID: \"2b3eec05-7eee-4d52-8342-9a001f645b33\") "
Jul 24 17:54:33.377626 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:33.377554    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b3eec05-7eee-4d52-8342-9a001f645b33-model-cache\") pod \"2b3eec05-7eee-4d52-8342-9a001f645b33\" (UID: \"2b3eec05-7eee-4d52-8342-9a001f645b33\") "
Jul 24 17:54:33.377626 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:33.377580    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b3eec05-7eee-4d52-8342-9a001f645b33-dshm\") pod \"2b3eec05-7eee-4d52-8342-9a001f645b33\" (UID: \"2b3eec05-7eee-4d52-8342-9a001f645b33\") "
Jul 24 17:54:33.377810 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:33.377644    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b3eec05-7eee-4d52-8342-9a001f645b33-tmp-dir\") pod \"2b3eec05-7eee-4d52-8342-9a001f645b33\" (UID: \"2b3eec05-7eee-4d52-8342-9a001f645b33\") "
Jul 24 17:54:33.377810 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:33.377672    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b3eec05-7eee-4d52-8342-9a001f645b33-tls-certs\") pod \"2b3eec05-7eee-4d52-8342-9a001f645b33\" (UID: \"2b3eec05-7eee-4d52-8342-9a001f645b33\") "
Jul 24 17:54:33.377810 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:33.377716    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b3eec05-7eee-4d52-8342-9a001f645b33-home" (OuterVolumeSpecName: "home") pod "2b3eec05-7eee-4d52-8342-9a001f645b33" (UID: "2b3eec05-7eee-4d52-8342-9a001f645b33"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:54:33.378027 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:33.377831    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b3eec05-7eee-4d52-8342-9a001f645b33-model-cache" (OuterVolumeSpecName: "model-cache") pod "2b3eec05-7eee-4d52-8342-9a001f645b33" (UID: "2b3eec05-7eee-4d52-8342-9a001f645b33"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:54:33.378027 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:33.377949    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b3eec05-7eee-4d52-8342-9a001f645b33-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2b3eec05-7eee-4d52-8342-9a001f645b33" (UID: "2b3eec05-7eee-4d52-8342-9a001f645b33"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:54:33.378160 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:33.378119    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b3eec05-7eee-4d52-8342-9a001f645b33-model-cache\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:54:33.378160 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:33.378156    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b3eec05-7eee-4d52-8342-9a001f645b33-tmp-dir\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:54:33.378227 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:33.378170    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b3eec05-7eee-4d52-8342-9a001f645b33-home\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:54:33.380456 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:33.380428    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b3eec05-7eee-4d52-8342-9a001f645b33-dshm" (OuterVolumeSpecName: "dshm") pod "2b3eec05-7eee-4d52-8342-9a001f645b33" (UID: "2b3eec05-7eee-4d52-8342-9a001f645b33"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:54:33.380579 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:33.380479    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b3eec05-7eee-4d52-8342-9a001f645b33-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2b3eec05-7eee-4d52-8342-9a001f645b33" (UID: "2b3eec05-7eee-4d52-8342-9a001f645b33"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:54:33.380579 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:33.380552    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b3eec05-7eee-4d52-8342-9a001f645b33-kube-api-access-sk5jx" (OuterVolumeSpecName: "kube-api-access-sk5jx") pod "2b3eec05-7eee-4d52-8342-9a001f645b33" (UID: "2b3eec05-7eee-4d52-8342-9a001f645b33"). InnerVolumeSpecName "kube-api-access-sk5jx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:54:33.465464 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:33.465432    2581 generic.go:356] "Generic (PLEG): container finished" podID="2b3eec05-7eee-4d52-8342-9a001f645b33" containerID="36db89b3795382c363e4f3f277fc3e60e8c02d0957500b9a2658dc7ed02488b9" exitCode=0
Jul 24 17:54:33.465621 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:33.465471    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-bb94b3/sub-test-model-bb94b3-kserve-8658946795-268jn"
Jul 24 17:54:33.465621 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:33.465519    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-bb94b3/sub-test-model-bb94b3-kserve-8658946795-268jn" event={"ID":"2b3eec05-7eee-4d52-8342-9a001f645b33","Type":"ContainerDied","Data":"36db89b3795382c363e4f3f277fc3e60e8c02d0957500b9a2658dc7ed02488b9"}
Jul 24 17:54:33.465621 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:33.465559    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-bb94b3/sub-test-model-bb94b3-kserve-8658946795-268jn" event={"ID":"2b3eec05-7eee-4d52-8342-9a001f645b33","Type":"ContainerDied","Data":"74c457b6fea15bd62107c5b464c4b0340851d31bca621b061760c8a298e70429"}
Jul 24 17:54:33.465621 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:33.465580    2581 scope.go:117] "RemoveContainer" containerID="36db89b3795382c363e4f3f277fc3e60e8c02d0957500b9a2658dc7ed02488b9"
Jul 24 17:54:33.474301 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:33.474280    2581 scope.go:117] "RemoveContainer" containerID="36db89b3795382c363e4f3f277fc3e60e8c02d0957500b9a2658dc7ed02488b9"
Jul 24 17:54:33.474548 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:54:33.474527    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36db89b3795382c363e4f3f277fc3e60e8c02d0957500b9a2658dc7ed02488b9\": container with ID starting with 36db89b3795382c363e4f3f277fc3e60e8c02d0957500b9a2658dc7ed02488b9 not found: ID does not exist" containerID="36db89b3795382c363e4f3f277fc3e60e8c02d0957500b9a2658dc7ed02488b9"
Jul 24 17:54:33.474626 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:33.474556    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36db89b3795382c363e4f3f277fc3e60e8c02d0957500b9a2658dc7ed02488b9"} err="failed to get container status \"36db89b3795382c363e4f3f277fc3e60e8c02d0957500b9a2658dc7ed02488b9\": rpc error: code = NotFound desc = could not find container \"36db89b3795382c363e4f3f277fc3e60e8c02d0957500b9a2658dc7ed02488b9\": container with ID starting with 36db89b3795382c363e4f3f277fc3e60e8c02d0957500b9a2658dc7ed02488b9 not found: ID does not exist"
Jul 24 17:54:33.479403 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:33.479386    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sk5jx\" (UniqueName: \"kubernetes.io/projected/2b3eec05-7eee-4d52-8342-9a001f645b33-kube-api-access-sk5jx\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:54:33.479482 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:33.479405    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b3eec05-7eee-4d52-8342-9a001f645b33-dshm\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:54:33.479482 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:33.479415    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b3eec05-7eee-4d52-8342-9a001f645b33-tls-certs\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:54:33.489331 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:33.489260    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-bb94b3/sub-test-model-bb94b3-kserve-8658946795-268jn"]
Jul 24 17:54:33.491214 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:33.491193    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-bb94b3/sub-test-model-bb94b3-kserve-8658946795-268jn"]
Jul 24 17:54:33.593194 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:33.593165    2581 scope.go:117] "RemoveContainer" containerID="fb91a0d780954f1abe61aceb3b82edc9c081f2437108b15d553a09747edd06e7"
Jul 24 17:54:33.601590 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:33.601571    2581 scope.go:117] "RemoveContainer" containerID="a66c7ee40851a8f3dacbe5ba9d4615139d1407158cbd080867ebb319c35ba7a4"
Jul 24 17:54:33.893989 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:33.893954    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-5c7cfc/sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n" podUID="1575d87c-d212-4866-854f-818dd6899f87" containerName="main" containerID="cri-o://1f4fe4ad1d804850b9816d71341ecbd384af8af149b157ed711bf2ab0b3c83ff" gracePeriod=45
Jul 24 17:54:34.157943 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:34.157877    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5c7cfc/sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n"
Jul 24 17:54:34.285068 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:34.285008    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cmlfq\" (UniqueName: \"kubernetes.io/projected/1575d87c-d212-4866-854f-818dd6899f87-kube-api-access-cmlfq\") pod \"1575d87c-d212-4866-854f-818dd6899f87\" (UID: \"1575d87c-d212-4866-854f-818dd6899f87\") "
Jul 24 17:54:34.285255 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:34.285094    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1575d87c-d212-4866-854f-818dd6899f87-dshm\") pod \"1575d87c-d212-4866-854f-818dd6899f87\" (UID: \"1575d87c-d212-4866-854f-818dd6899f87\") "
Jul 24 17:54:34.285255 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:34.285161    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1575d87c-d212-4866-854f-818dd6899f87-home\") pod \"1575d87c-d212-4866-854f-818dd6899f87\" (UID: \"1575d87c-d212-4866-854f-818dd6899f87\") "
Jul 24 17:54:34.285255 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:34.285185    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1575d87c-d212-4866-854f-818dd6899f87-model-cache\") pod \"1575d87c-d212-4866-854f-818dd6899f87\" (UID: \"1575d87c-d212-4866-854f-818dd6899f87\") "
Jul 24 17:54:34.285255 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:34.285211    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1575d87c-d212-4866-854f-818dd6899f87-tls-certs\") pod \"1575d87c-d212-4866-854f-818dd6899f87\" (UID: \"1575d87c-d212-4866-854f-818dd6899f87\") "
Jul 24 17:54:34.285255 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:34.285246    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1575d87c-d212-4866-854f-818dd6899f87-tmp-dir\") pod \"1575d87c-d212-4866-854f-818dd6899f87\" (UID: \"1575d87c-d212-4866-854f-818dd6899f87\") "
Jul 24 17:54:34.285533 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:34.285434    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1575d87c-d212-4866-854f-818dd6899f87-model-cache" (OuterVolumeSpecName: "model-cache") pod "1575d87c-d212-4866-854f-818dd6899f87" (UID: "1575d87c-d212-4866-854f-818dd6899f87"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:54:34.285533 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:34.285477    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1575d87c-d212-4866-854f-818dd6899f87-home" (OuterVolumeSpecName: "home") pod "1575d87c-d212-4866-854f-818dd6899f87" (UID: "1575d87c-d212-4866-854f-818dd6899f87"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:54:34.285610 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:34.285566    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1575d87c-d212-4866-854f-818dd6899f87-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1575d87c-d212-4866-854f-818dd6899f87" (UID: "1575d87c-d212-4866-854f-818dd6899f87"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:54:34.287473 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:34.287445    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1575d87c-d212-4866-854f-818dd6899f87-dshm" (OuterVolumeSpecName: "dshm") pod "1575d87c-d212-4866-854f-818dd6899f87" (UID: "1575d87c-d212-4866-854f-818dd6899f87"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:54:34.287473 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:34.287466    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1575d87c-d212-4866-854f-818dd6899f87-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1575d87c-d212-4866-854f-818dd6899f87" (UID: "1575d87c-d212-4866-854f-818dd6899f87"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:54:34.287628 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:34.287496    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1575d87c-d212-4866-854f-818dd6899f87-kube-api-access-cmlfq" (OuterVolumeSpecName: "kube-api-access-cmlfq") pod "1575d87c-d212-4866-854f-818dd6899f87" (UID: "1575d87c-d212-4866-854f-818dd6899f87"). InnerVolumeSpecName "kube-api-access-cmlfq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:54:34.386865 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:34.386827    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cmlfq\" (UniqueName: \"kubernetes.io/projected/1575d87c-d212-4866-854f-818dd6899f87-kube-api-access-cmlfq\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:54:34.386865 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:34.386854    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1575d87c-d212-4866-854f-818dd6899f87-dshm\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:54:34.386865 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:34.386863    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1575d87c-d212-4866-854f-818dd6899f87-home\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:54:34.386865 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:34.386870    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1575d87c-d212-4866-854f-818dd6899f87-model-cache\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:54:34.387160 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:34.386880    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1575d87c-d212-4866-854f-818dd6899f87-tls-certs\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:54:34.387160 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:34.386887    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1575d87c-d212-4866-854f-818dd6899f87-tmp-dir\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:54:34.471460 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:34.471371    2581 generic.go:356] "Generic (PLEG): container finished" podID="1575d87c-d212-4866-854f-818dd6899f87" containerID="1f4fe4ad1d804850b9816d71341ecbd384af8af149b157ed711bf2ab0b3c83ff" exitCode=0
Jul 24 17:54:34.471460 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:34.471415    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5c7cfc/sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n"
Jul 24 17:54:34.471667 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:34.471459    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5c7cfc/sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n" event={"ID":"1575d87c-d212-4866-854f-818dd6899f87","Type":"ContainerDied","Data":"1f4fe4ad1d804850b9816d71341ecbd384af8af149b157ed711bf2ab0b3c83ff"}
Jul 24 17:54:34.471667 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:34.471502    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5c7cfc/sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n" event={"ID":"1575d87c-d212-4866-854f-818dd6899f87","Type":"ContainerDied","Data":"be89c24fb9b6505fcab6e4b0fd5caa7b41e1b8d1ba55646cd975d762b55b8b2f"}
Jul 24 17:54:34.471667 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:34.471527    2581 scope.go:117] "RemoveContainer" containerID="1f4fe4ad1d804850b9816d71341ecbd384af8af149b157ed711bf2ab0b3c83ff"
Jul 24 17:54:34.480862 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:34.480844    2581 scope.go:117] "RemoveContainer" containerID="1f4fe4ad1d804850b9816d71341ecbd384af8af149b157ed711bf2ab0b3c83ff"
Jul 24 17:54:34.481137 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:54:34.481119    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f4fe4ad1d804850b9816d71341ecbd384af8af149b157ed711bf2ab0b3c83ff\": container with ID starting with 1f4fe4ad1d804850b9816d71341ecbd384af8af149b157ed711bf2ab0b3c83ff not found: ID does not exist" containerID="1f4fe4ad1d804850b9816d71341ecbd384af8af149b157ed711bf2ab0b3c83ff"
Jul 24 17:54:34.481203 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:34.481145    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f4fe4ad1d804850b9816d71341ecbd384af8af149b157ed711bf2ab0b3c83ff"} err="failed to get container status \"1f4fe4ad1d804850b9816d71341ecbd384af8af149b157ed711bf2ab0b3c83ff\": rpc error: code = NotFound desc = could not find container \"1f4fe4ad1d804850b9816d71341ecbd384af8af149b157ed711bf2ab0b3c83ff\": container with ID starting with 1f4fe4ad1d804850b9816d71341ecbd384af8af149b157ed711bf2ab0b3c83ff not found: ID does not exist"
Jul 24 17:54:34.496283 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:34.496257    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-5c7cfc/sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n"]
Jul 24 17:54:34.500975 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:34.500955    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-5c7cfc/sub-test-model-5c7cfc-kserve-86f6c967d9-9l72n"]
Jul 24 17:54:35.383217 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:35.383181    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1575d87c-d212-4866-854f-818dd6899f87" path="/var/lib/kubelet/pods/1575d87c-d212-4866-854f-818dd6899f87/volumes"
Jul 24 17:54:35.383577 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:35.383563    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b3eec05-7eee-4d52-8342-9a001f645b33" path="/var/lib/kubelet/pods/2b3eec05-7eee-4d52-8342-9a001f645b33/volumes"
Jul 24 17:54:39.781296 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:39.781266    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-bb94b3/rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6"
Jul 24 17:54:39.781654 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:39.781509    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-bb94b3/rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6"
Jul 24 17:54:39.793690 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:39.793667    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-bb94b3/rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6"
Jul 24 17:54:40.512674 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:40.512644    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-bb94b3/rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6"
Jul 24 17:54:40.566781 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:54:40.566752    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:55:11.255570 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:11.255534    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:55:12.610070 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:12.610018    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:55:14.953397 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:14.953349    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:55:15.054821 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:15.054784    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:55:18.848088 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:18.848033    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-5c7cfc/rate-test-model-5c7cfc-kserve-68db896dc-dhbjv"]
Jul 24 17:55:18.848521 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:18.848448    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1575d87c-d212-4866-854f-818dd6899f87" containerName="main"
Jul 24 17:55:18.848521 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:18.848458    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1575d87c-d212-4866-854f-818dd6899f87" containerName="main"
Jul 24 17:55:18.848521 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:18.848466    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b3eec05-7eee-4d52-8342-9a001f645b33" containerName="main"
Jul 24 17:55:18.848521 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:18.848471    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b3eec05-7eee-4d52-8342-9a001f645b33" containerName="main"
Jul 24 17:55:18.848649 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:18.848529    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1575d87c-d212-4866-854f-818dd6899f87" containerName="main"
Jul 24 17:55:18.848649 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:18.848538    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b3eec05-7eee-4d52-8342-9a001f645b33" containerName="main"
Jul 24 17:55:18.851544 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:18.851528    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5c7cfc/rate-test-model-5c7cfc-kserve-68db896dc-dhbjv"
Jul 24 17:55:18.854513 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:18.854493    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-5c7cfc\"/\"kube-root-ca.crt\""
Jul 24 17:55:18.855691 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:18.855654    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-5c7cfc\"/\"default-dockercfg-hkndk\""
Jul 24 17:55:18.855869 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:18.855675    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-5c7cfc\"/\"rate-test-model-5c7cfc-kserve-self-signed-certs\""
Jul 24 17:55:18.855960 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:18.855685    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-5c7cfc\"/\"openshift-service-ca.crt\""
Jul 24 17:55:18.861910 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:18.861889    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-5c7cfc/rate-test-model-5c7cfc-kserve-68db896dc-dhbjv"]
Jul 24 17:55:18.883594 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:18.883567    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:55:19.008325 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:19.008285    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9fff7ecd-e51f-45f8-8310-c0031babb64c-tmp-dir\") pod \"rate-test-model-5c7cfc-kserve-68db896dc-dhbjv\" (UID: \"9fff7ecd-e51f-45f8-8310-c0031babb64c\") " pod="ai-tenant-e2e-shared-b-5c7cfc/rate-test-model-5c7cfc-kserve-68db896dc-dhbjv"
Jul 24 17:55:19.008525 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:19.008336    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m286s\" (UniqueName: \"kubernetes.io/projected/9fff7ecd-e51f-45f8-8310-c0031babb64c-kube-api-access-m286s\") pod \"rate-test-model-5c7cfc-kserve-68db896dc-dhbjv\" (UID: \"9fff7ecd-e51f-45f8-8310-c0031babb64c\") " pod="ai-tenant-e2e-shared-b-5c7cfc/rate-test-model-5c7cfc-kserve-68db896dc-dhbjv"
Jul 24 17:55:19.008525 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:19.008378    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9fff7ecd-e51f-45f8-8310-c0031babb64c-model-cache\") pod \"rate-test-model-5c7cfc-kserve-68db896dc-dhbjv\" (UID: \"9fff7ecd-e51f-45f8-8310-c0031babb64c\") " pod="ai-tenant-e2e-shared-b-5c7cfc/rate-test-model-5c7cfc-kserve-68db896dc-dhbjv"
Jul 24 17:55:19.008525 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:19.008445    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fff7ecd-e51f-45f8-8310-c0031babb64c-tls-certs\") pod \"rate-test-model-5c7cfc-kserve-68db896dc-dhbjv\" (UID: \"9fff7ecd-e51f-45f8-8310-c0031babb64c\") " pod="ai-tenant-e2e-shared-b-5c7cfc/rate-test-model-5c7cfc-kserve-68db896dc-dhbjv"
Jul 24 17:55:19.008525 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:19.008487    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9fff7ecd-e51f-45f8-8310-c0031babb64c-home\") pod \"rate-test-model-5c7cfc-kserve-68db896dc-dhbjv\" (UID: \"9fff7ecd-e51f-45f8-8310-c0031babb64c\") " pod="ai-tenant-e2e-shared-b-5c7cfc/rate-test-model-5c7cfc-kserve-68db896dc-dhbjv"
Jul 24 17:55:19.008525 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:19.008511    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9fff7ecd-e51f-45f8-8310-c0031babb64c-dshm\") pod \"rate-test-model-5c7cfc-kserve-68db896dc-dhbjv\" (UID: \"9fff7ecd-e51f-45f8-8310-c0031babb64c\") " pod="ai-tenant-e2e-shared-b-5c7cfc/rate-test-model-5c7cfc-kserve-68db896dc-dhbjv"
Jul 24 17:55:19.110181 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:19.110090    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9fff7ecd-e51f-45f8-8310-c0031babb64c-dshm\") pod \"rate-test-model-5c7cfc-kserve-68db896dc-dhbjv\" (UID: \"9fff7ecd-e51f-45f8-8310-c0031babb64c\") " pod="ai-tenant-e2e-shared-b-5c7cfc/rate-test-model-5c7cfc-kserve-68db896dc-dhbjv"
Jul 24 17:55:19.110322 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:19.110185    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9fff7ecd-e51f-45f8-8310-c0031babb64c-tmp-dir\") pod \"rate-test-model-5c7cfc-kserve-68db896dc-dhbjv\" (UID: \"9fff7ecd-e51f-45f8-8310-c0031babb64c\") " pod="ai-tenant-e2e-shared-b-5c7cfc/rate-test-model-5c7cfc-kserve-68db896dc-dhbjv"
Jul 24 17:55:19.110322 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:19.110225    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m286s\" (UniqueName: \"kubernetes.io/projected/9fff7ecd-e51f-45f8-8310-c0031babb64c-kube-api-access-m286s\") pod \"rate-test-model-5c7cfc-kserve-68db896dc-dhbjv\" (UID: \"9fff7ecd-e51f-45f8-8310-c0031babb64c\") " pod="ai-tenant-e2e-shared-b-5c7cfc/rate-test-model-5c7cfc-kserve-68db896dc-dhbjv"
Jul 24 17:55:19.110322 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:19.110279    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9fff7ecd-e51f-45f8-8310-c0031babb64c-model-cache\") pod \"rate-test-model-5c7cfc-kserve-68db896dc-dhbjv\" (UID: \"9fff7ecd-e51f-45f8-8310-c0031babb64c\") " pod="ai-tenant-e2e-shared-b-5c7cfc/rate-test-model-5c7cfc-kserve-68db896dc-dhbjv"
Jul 24 17:55:19.110322 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:19.110314    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fff7ecd-e51f-45f8-8310-c0031babb64c-tls-certs\") pod \"rate-test-model-5c7cfc-kserve-68db896dc-dhbjv\" (UID: \"9fff7ecd-e51f-45f8-8310-c0031babb64c\") " pod="ai-tenant-e2e-shared-b-5c7cfc/rate-test-model-5c7cfc-kserve-68db896dc-dhbjv"
Jul 24 17:55:19.110487 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:19.110340    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9fff7ecd-e51f-45f8-8310-c0031babb64c-home\") pod \"rate-test-model-5c7cfc-kserve-68db896dc-dhbjv\" (UID: \"9fff7ecd-e51f-45f8-8310-c0031babb64c\") " pod="ai-tenant-e2e-shared-b-5c7cfc/rate-test-model-5c7cfc-kserve-68db896dc-dhbjv"
Jul 24 17:55:19.110669 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:19.110638    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9fff7ecd-e51f-45f8-8310-c0031babb64c-tmp-dir\") pod \"rate-test-model-5c7cfc-kserve-68db896dc-dhbjv\" (UID: \"9fff7ecd-e51f-45f8-8310-c0031babb64c\") " pod="ai-tenant-e2e-shared-b-5c7cfc/rate-test-model-5c7cfc-kserve-68db896dc-dhbjv"
Jul 24 17:55:19.110669 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:19.110656    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9fff7ecd-e51f-45f8-8310-c0031babb64c-home\") pod \"rate-test-model-5c7cfc-kserve-68db896dc-dhbjv\" (UID: \"9fff7ecd-e51f-45f8-8310-c0031babb64c\") " pod="ai-tenant-e2e-shared-b-5c7cfc/rate-test-model-5c7cfc-kserve-68db896dc-dhbjv"
Jul 24 17:55:19.110824 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:19.110731    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9fff7ecd-e51f-45f8-8310-c0031babb64c-model-cache\") pod \"rate-test-model-5c7cfc-kserve-68db896dc-dhbjv\" (UID: \"9fff7ecd-e51f-45f8-8310-c0031babb64c\") " pod="ai-tenant-e2e-shared-b-5c7cfc/rate-test-model-5c7cfc-kserve-68db896dc-dhbjv"
Jul 24 17:55:19.112535 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:19.112508    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9fff7ecd-e51f-45f8-8310-c0031babb64c-dshm\") pod \"rate-test-model-5c7cfc-kserve-68db896dc-dhbjv\" (UID: \"9fff7ecd-e51f-45f8-8310-c0031babb64c\") " pod="ai-tenant-e2e-shared-b-5c7cfc/rate-test-model-5c7cfc-kserve-68db896dc-dhbjv"
Jul 24 17:55:19.112780 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:19.112763    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fff7ecd-e51f-45f8-8310-c0031babb64c-tls-certs\") pod \"rate-test-model-5c7cfc-kserve-68db896dc-dhbjv\" (UID: \"9fff7ecd-e51f-45f8-8310-c0031babb64c\") " pod="ai-tenant-e2e-shared-b-5c7cfc/rate-test-model-5c7cfc-kserve-68db896dc-dhbjv"
Jul 24 17:55:19.119184 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:19.119163    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m286s\" (UniqueName: \"kubernetes.io/projected/9fff7ecd-e51f-45f8-8310-c0031babb64c-kube-api-access-m286s\") pod \"rate-test-model-5c7cfc-kserve-68db896dc-dhbjv\" (UID: \"9fff7ecd-e51f-45f8-8310-c0031babb64c\") " pod="ai-tenant-e2e-shared-b-5c7cfc/rate-test-model-5c7cfc-kserve-68db896dc-dhbjv"
Jul 24 17:55:19.161756 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:19.161730    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5c7cfc/rate-test-model-5c7cfc-kserve-68db896dc-dhbjv"
Jul 24 17:55:19.299412 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:19.299387    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-5c7cfc/rate-test-model-5c7cfc-kserve-68db896dc-dhbjv"]
Jul 24 17:55:19.301408 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:55:19.301379    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9fff7ecd_e51f_45f8_8310_c0031babb64c.slice/crio-c3aa61c9879a61a632cc313223008e65871399bd232e7617e1e551a266e75aa0 WatchSource:0}: Error finding container c3aa61c9879a61a632cc313223008e65871399bd232e7617e1e551a266e75aa0: Status 404 returned error can't find the container with id c3aa61c9879a61a632cc313223008e65871399bd232e7617e1e551a266e75aa0
Jul 24 17:55:19.693660 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:19.693624    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5c7cfc/rate-test-model-5c7cfc-kserve-68db896dc-dhbjv" event={"ID":"9fff7ecd-e51f-45f8-8310-c0031babb64c","Type":"ContainerStarted","Data":"8a702523712ccc94f2f65795356777fb0b3eac356c9a04bd3e56980b09f02df0"}
Jul 24 17:55:19.693660 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:19.693663    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5c7cfc/rate-test-model-5c7cfc-kserve-68db896dc-dhbjv" event={"ID":"9fff7ecd-e51f-45f8-8310-c0031babb64c","Type":"ContainerStarted","Data":"c3aa61c9879a61a632cc313223008e65871399bd232e7617e1e551a266e75aa0"}
Jul 24 17:55:19.714335 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:19.714290    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-5c7cfc/rate-test-model-5c7cfc-kserve-68db896dc-dhbjv" podStartSLOduration=1.7142757400000002 podStartE2EDuration="1.71427574s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:55:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:55:19.71190627 +0000 UTC m=+2686.913955568" watchObservedRunningTime="2026-07-24 17:55:19.71427574 +0000 UTC m=+2686.916325032"
Jul 24 17:55:22.763551 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:22.763516    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:55:29.161993 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:29.161915    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-5c7cfc/rate-test-model-5c7cfc-kserve-68db896dc-dhbjv"
Jul 24 17:55:29.161993 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:29.161966    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-5c7cfc/rate-test-model-5c7cfc-kserve-68db896dc-dhbjv"
Jul 24 17:55:29.175205 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:29.175183    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-5c7cfc/rate-test-model-5c7cfc-kserve-68db896dc-dhbjv"
Jul 24 17:55:29.754737 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:29.754708    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-5c7cfc/rate-test-model-5c7cfc-kserve-68db896dc-dhbjv"
Jul 24 17:55:33.630149 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:33.630114    2581 scope.go:117] "RemoveContainer" containerID="4dcb275f3710c3c78945a35de93ce0f081798c847bd61b951ff5f30c4bc3a397"
Jul 24 17:55:36.653704 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:36.653664    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:55:55.459243 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:55:55.459193    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:56:02.683366 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:02.683332    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-bb94b3/rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6"]
Jul 24 17:56:02.711080 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:02.711030    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:56:03.575190 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:03.575148    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-5c7cfc/rate-test-model-5c7cfc-kserve-68db896dc-dhbjv"]
Jul 24 17:56:03.599586 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:03.599553    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:56:11.363763 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:11.363727    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:56:16.266562 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:16.266522    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:56:17.702872 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:17.702806    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-bb94b3/rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6" podUID="4d526063-a78d-4114-8bec-f3aeb33a5860" containerName="main" containerID="cri-o://17594eb61b34d78be759d1c1dce2e9610e210c65242d3aebbf27dc32d4b5c11a" gracePeriod=45
Jul 24 17:56:17.956517 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:17.956449    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-bb94b3/rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6"
Jul 24 17:56:17.985232 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:17.985197    2581 generic.go:356] "Generic (PLEG): container finished" podID="4d526063-a78d-4114-8bec-f3aeb33a5860" containerID="17594eb61b34d78be759d1c1dce2e9610e210c65242d3aebbf27dc32d4b5c11a" exitCode=0
Jul 24 17:56:17.985384 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:17.985243    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-bb94b3/rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6"
Jul 24 17:56:17.985384 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:17.985282    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-bb94b3/rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6" event={"ID":"4d526063-a78d-4114-8bec-f3aeb33a5860","Type":"ContainerDied","Data":"17594eb61b34d78be759d1c1dce2e9610e210c65242d3aebbf27dc32d4b5c11a"}
Jul 24 17:56:17.985384 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:17.985331    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-bb94b3/rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6" event={"ID":"4d526063-a78d-4114-8bec-f3aeb33a5860","Type":"ContainerDied","Data":"369c8150aef1fd916e9da5d129549889c13bc124ead5e91e03038a942531a4bb"}
Jul 24 17:56:17.985384 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:17.985354    2581 scope.go:117] "RemoveContainer" containerID="17594eb61b34d78be759d1c1dce2e9610e210c65242d3aebbf27dc32d4b5c11a"
Jul 24 17:56:17.995770 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:17.995748    2581 scope.go:117] "RemoveContainer" containerID="17594eb61b34d78be759d1c1dce2e9610e210c65242d3aebbf27dc32d4b5c11a"
Jul 24 17:56:17.996090 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:56:17.996063    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17594eb61b34d78be759d1c1dce2e9610e210c65242d3aebbf27dc32d4b5c11a\": container with ID starting with 17594eb61b34d78be759d1c1dce2e9610e210c65242d3aebbf27dc32d4b5c11a not found: ID does not exist" containerID="17594eb61b34d78be759d1c1dce2e9610e210c65242d3aebbf27dc32d4b5c11a"
Jul 24 17:56:17.996191 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:17.996092    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17594eb61b34d78be759d1c1dce2e9610e210c65242d3aebbf27dc32d4b5c11a"} err="failed to get container status \"17594eb61b34d78be759d1c1dce2e9610e210c65242d3aebbf27dc32d4b5c11a\": rpc error: code = NotFound desc = could not find container \"17594eb61b34d78be759d1c1dce2e9610e210c65242d3aebbf27dc32d4b5c11a\": container with ID starting with 17594eb61b34d78be759d1c1dce2e9610e210c65242d3aebbf27dc32d4b5c11a not found: ID does not exist"
Jul 24 17:56:18.064346 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.064311    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45l2d\" (UniqueName: \"kubernetes.io/projected/4d526063-a78d-4114-8bec-f3aeb33a5860-kube-api-access-45l2d\") pod \"4d526063-a78d-4114-8bec-f3aeb33a5860\" (UID: \"4d526063-a78d-4114-8bec-f3aeb33a5860\") "
Jul 24 17:56:18.064514 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.064358    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d526063-a78d-4114-8bec-f3aeb33a5860-model-cache\") pod \"4d526063-a78d-4114-8bec-f3aeb33a5860\" (UID: \"4d526063-a78d-4114-8bec-f3aeb33a5860\") "
Jul 24 17:56:18.064514 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.064389    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d526063-a78d-4114-8bec-f3aeb33a5860-dshm\") pod \"4d526063-a78d-4114-8bec-f3aeb33a5860\" (UID: \"4d526063-a78d-4114-8bec-f3aeb33a5860\") "
Jul 24 17:56:18.064514 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.064414    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d526063-a78d-4114-8bec-f3aeb33a5860-tmp-dir\") pod \"4d526063-a78d-4114-8bec-f3aeb33a5860\" (UID: \"4d526063-a78d-4114-8bec-f3aeb33a5860\") "
Jul 24 17:56:18.064514 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.064442    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d526063-a78d-4114-8bec-f3aeb33a5860-tls-certs\") pod \"4d526063-a78d-4114-8bec-f3aeb33a5860\" (UID: \"4d526063-a78d-4114-8bec-f3aeb33a5860\") "
Jul 24 17:56:18.064514 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.064477    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d526063-a78d-4114-8bec-f3aeb33a5860-home\") pod \"4d526063-a78d-4114-8bec-f3aeb33a5860\" (UID: \"4d526063-a78d-4114-8bec-f3aeb33a5860\") "
Jul 24 17:56:18.064811 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.064756    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d526063-a78d-4114-8bec-f3aeb33a5860-model-cache" (OuterVolumeSpecName: "model-cache") pod "4d526063-a78d-4114-8bec-f3aeb33a5860" (UID: "4d526063-a78d-4114-8bec-f3aeb33a5860"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:56:18.064811 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.064795    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d526063-a78d-4114-8bec-f3aeb33a5860-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4d526063-a78d-4114-8bec-f3aeb33a5860" (UID: "4d526063-a78d-4114-8bec-f3aeb33a5860"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:56:18.064963 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.064829    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d526063-a78d-4114-8bec-f3aeb33a5860-home" (OuterVolumeSpecName: "home") pod "4d526063-a78d-4114-8bec-f3aeb33a5860" (UID: "4d526063-a78d-4114-8bec-f3aeb33a5860"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:56:18.066774 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.066746    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d526063-a78d-4114-8bec-f3aeb33a5860-dshm" (OuterVolumeSpecName: "dshm") pod "4d526063-a78d-4114-8bec-f3aeb33a5860" (UID: "4d526063-a78d-4114-8bec-f3aeb33a5860"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:56:18.066881 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.066806    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d526063-a78d-4114-8bec-f3aeb33a5860-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4d526063-a78d-4114-8bec-f3aeb33a5860" (UID: "4d526063-a78d-4114-8bec-f3aeb33a5860"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:56:18.066927 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.066887    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d526063-a78d-4114-8bec-f3aeb33a5860-kube-api-access-45l2d" (OuterVolumeSpecName: "kube-api-access-45l2d") pod "4d526063-a78d-4114-8bec-f3aeb33a5860" (UID: "4d526063-a78d-4114-8bec-f3aeb33a5860"). InnerVolumeSpecName "kube-api-access-45l2d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:56:18.165279 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.165239    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d526063-a78d-4114-8bec-f3aeb33a5860-dshm\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:56:18.165279 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.165271    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d526063-a78d-4114-8bec-f3aeb33a5860-tmp-dir\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:56:18.165279 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.165282    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d526063-a78d-4114-8bec-f3aeb33a5860-tls-certs\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:56:18.165530 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.165293    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d526063-a78d-4114-8bec-f3aeb33a5860-home\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:56:18.165530 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.165305    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45l2d\" (UniqueName: \"kubernetes.io/projected/4d526063-a78d-4114-8bec-f3aeb33a5860-kube-api-access-45l2d\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:56:18.165530 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.165316    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d526063-a78d-4114-8bec-f3aeb33a5860-model-cache\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:56:18.308319 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.308286    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-bb94b3/rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6"]
Jul 24 17:56:18.313482 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.313456    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-bb94b3/rate-test-model-bb94b3-kserve-68db9b4b7f-9nmg6"]
Jul 24 17:56:18.596691 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.596572    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-5c7cfc/rate-test-model-5c7cfc-kserve-68db896dc-dhbjv" podUID="9fff7ecd-e51f-45f8-8310-c0031babb64c" containerName="main" containerID="cri-o://8a702523712ccc94f2f65795356777fb0b3eac356c9a04bd3e56980b09f02df0" gracePeriod=45
Jul 24 17:56:18.853796 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.853739    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5c7cfc/rate-test-model-5c7cfc-kserve-68db896dc-dhbjv"
Jul 24 17:56:18.972703 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.972664    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m286s\" (UniqueName: \"kubernetes.io/projected/9fff7ecd-e51f-45f8-8310-c0031babb64c-kube-api-access-m286s\") pod \"9fff7ecd-e51f-45f8-8310-c0031babb64c\" (UID: \"9fff7ecd-e51f-45f8-8310-c0031babb64c\") "
Jul 24 17:56:18.972909 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.972775    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fff7ecd-e51f-45f8-8310-c0031babb64c-tls-certs\") pod \"9fff7ecd-e51f-45f8-8310-c0031babb64c\" (UID: \"9fff7ecd-e51f-45f8-8310-c0031babb64c\") "
Jul 24 17:56:18.972909 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.972800    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9fff7ecd-e51f-45f8-8310-c0031babb64c-home\") pod \"9fff7ecd-e51f-45f8-8310-c0031babb64c\" (UID: \"9fff7ecd-e51f-45f8-8310-c0031babb64c\") "
Jul 24 17:56:18.972909 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.972837    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9fff7ecd-e51f-45f8-8310-c0031babb64c-dshm\") pod \"9fff7ecd-e51f-45f8-8310-c0031babb64c\" (UID: \"9fff7ecd-e51f-45f8-8310-c0031babb64c\") "
Jul 24 17:56:18.972909 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.972869    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9fff7ecd-e51f-45f8-8310-c0031babb64c-tmp-dir\") pod \"9fff7ecd-e51f-45f8-8310-c0031babb64c\" (UID: \"9fff7ecd-e51f-45f8-8310-c0031babb64c\") "
Jul 24 17:56:18.972909 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.972890    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9fff7ecd-e51f-45f8-8310-c0031babb64c-model-cache\") pod \"9fff7ecd-e51f-45f8-8310-c0031babb64c\" (UID: \"9fff7ecd-e51f-45f8-8310-c0031babb64c\") "
Jul 24 17:56:18.973193 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.973143    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fff7ecd-e51f-45f8-8310-c0031babb64c-home" (OuterVolumeSpecName: "home") pod "9fff7ecd-e51f-45f8-8310-c0031babb64c" (UID: "9fff7ecd-e51f-45f8-8310-c0031babb64c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:56:18.973253 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.973192    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fff7ecd-e51f-45f8-8310-c0031babb64c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9fff7ecd-e51f-45f8-8310-c0031babb64c" (UID: "9fff7ecd-e51f-45f8-8310-c0031babb64c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:56:18.973253 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.973232    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fff7ecd-e51f-45f8-8310-c0031babb64c-model-cache" (OuterVolumeSpecName: "model-cache") pod "9fff7ecd-e51f-45f8-8310-c0031babb64c" (UID: "9fff7ecd-e51f-45f8-8310-c0031babb64c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:56:18.973362 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.973264    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9fff7ecd-e51f-45f8-8310-c0031babb64c-home\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:56:18.974988 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.974964    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fff7ecd-e51f-45f8-8310-c0031babb64c-kube-api-access-m286s" (OuterVolumeSpecName: "kube-api-access-m286s") pod "9fff7ecd-e51f-45f8-8310-c0031babb64c" (UID: "9fff7ecd-e51f-45f8-8310-c0031babb64c"). InnerVolumeSpecName "kube-api-access-m286s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:56:18.975136 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.975008    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fff7ecd-e51f-45f8-8310-c0031babb64c-dshm" (OuterVolumeSpecName: "dshm") pod "9fff7ecd-e51f-45f8-8310-c0031babb64c" (UID: "9fff7ecd-e51f-45f8-8310-c0031babb64c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:56:18.975136 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.975102    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fff7ecd-e51f-45f8-8310-c0031babb64c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9fff7ecd-e51f-45f8-8310-c0031babb64c" (UID: "9fff7ecd-e51f-45f8-8310-c0031babb64c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:56:18.991726 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.991698    2581 generic.go:356] "Generic (PLEG): container finished" podID="9fff7ecd-e51f-45f8-8310-c0031babb64c" containerID="8a702523712ccc94f2f65795356777fb0b3eac356c9a04bd3e56980b09f02df0" exitCode=0
Jul 24 17:56:18.991854 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.991741    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5c7cfc/rate-test-model-5c7cfc-kserve-68db896dc-dhbjv"
Jul 24 17:56:18.991854 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.991779    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5c7cfc/rate-test-model-5c7cfc-kserve-68db896dc-dhbjv" event={"ID":"9fff7ecd-e51f-45f8-8310-c0031babb64c","Type":"ContainerDied","Data":"8a702523712ccc94f2f65795356777fb0b3eac356c9a04bd3e56980b09f02df0"}
Jul 24 17:56:18.991854 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.991814    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5c7cfc/rate-test-model-5c7cfc-kserve-68db896dc-dhbjv" event={"ID":"9fff7ecd-e51f-45f8-8310-c0031babb64c","Type":"ContainerDied","Data":"c3aa61c9879a61a632cc313223008e65871399bd232e7617e1e551a266e75aa0"}
Jul 24 17:56:18.991854 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:18.991831    2581 scope.go:117] "RemoveContainer" containerID="8a702523712ccc94f2f65795356777fb0b3eac356c9a04bd3e56980b09f02df0"
Jul 24 17:56:19.000821 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:19.000806    2581 scope.go:117] "RemoveContainer" containerID="8a702523712ccc94f2f65795356777fb0b3eac356c9a04bd3e56980b09f02df0"
Jul 24 17:56:19.001110 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:56:19.001088    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a702523712ccc94f2f65795356777fb0b3eac356c9a04bd3e56980b09f02df0\": container with ID starting with 8a702523712ccc94f2f65795356777fb0b3eac356c9a04bd3e56980b09f02df0 not found: ID does not exist" containerID="8a702523712ccc94f2f65795356777fb0b3eac356c9a04bd3e56980b09f02df0"
Jul 24 17:56:19.001186 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:19.001122    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a702523712ccc94f2f65795356777fb0b3eac356c9a04bd3e56980b09f02df0"} err="failed to get container status \"8a702523712ccc94f2f65795356777fb0b3eac356c9a04bd3e56980b09f02df0\": rpc error: code = NotFound desc = could not find container \"8a702523712ccc94f2f65795356777fb0b3eac356c9a04bd3e56980b09f02df0\": container with ID starting with 8a702523712ccc94f2f65795356777fb0b3eac356c9a04bd3e56980b09f02df0 not found: ID does not exist"
Jul 24 17:56:19.014473 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:19.014445    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-5c7cfc/rate-test-model-5c7cfc-kserve-68db896dc-dhbjv"]
Jul 24 17:56:19.018849 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:19.018827    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-5c7cfc/rate-test-model-5c7cfc-kserve-68db896dc-dhbjv"]
Jul 24 17:56:19.074656 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:19.074625    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9fff7ecd-e51f-45f8-8310-c0031babb64c-dshm\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:56:19.074656 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:19.074651    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9fff7ecd-e51f-45f8-8310-c0031babb64c-tmp-dir\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:56:19.074656 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:19.074660    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9fff7ecd-e51f-45f8-8310-c0031babb64c-model-cache\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:56:19.074856 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:19.074668    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m286s\" (UniqueName: \"kubernetes.io/projected/9fff7ecd-e51f-45f8-8310-c0031babb64c-kube-api-access-m286s\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:56:19.074856 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:19.074678    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fff7ecd-e51f-45f8-8310-c0031babb64c-tls-certs\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:56:19.383864 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:19.383827    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d526063-a78d-4114-8bec-f3aeb33a5860" path="/var/lib/kubelet/pods/4d526063-a78d-4114-8bec-f3aeb33a5860/volumes"
Jul 24 17:56:19.384265 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:19.384250    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fff7ecd-e51f-45f8-8310-c0031babb64c" path="/var/lib/kubelet/pods/9fff7ecd-e51f-45f8-8310-c0031babb64c/volumes"
Jul 24 17:56:23.610849 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:23.610814    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:56:23.905899 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:23.905868    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"]
Jul 24 17:56:23.906297 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:23.906283    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d526063-a78d-4114-8bec-f3aeb33a5860" containerName="main"
Jul 24 17:56:23.906297 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:23.906297    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d526063-a78d-4114-8bec-f3aeb33a5860" containerName="main"
Jul 24 17:56:23.906418 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:23.906308    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fff7ecd-e51f-45f8-8310-c0031babb64c" containerName="main"
Jul 24 17:56:23.906418 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:23.906314    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fff7ecd-e51f-45f8-8310-c0031babb64c" containerName="main"
Jul 24 17:56:23.906418 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:23.906380    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d526063-a78d-4114-8bec-f3aeb33a5860" containerName="main"
Jul 24 17:56:23.906418 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:23.906388    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fff7ecd-e51f-45f8-8310-c0031babb64c" containerName="main"
Jul 24 17:56:23.910821 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:23.910803    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 17:56:23.914708 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:23.914687    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 24 17:56:23.916324 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:23.916297    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-8a6414-openshift-default-dockercfg-twf95\""
Jul 24 17:56:23.934014 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:23.933979    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:56:23.935343 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:23.935321    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"]
Jul 24 17:56:24.019337 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:24.019297    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03bb2442-967f-465f-94bf-a209ae79842d-istiod-ca-cert\") pod \"e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr\" (UID: \"03bb2442-967f-465f-94bf-a209ae79842d\") " pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 17:56:24.019517 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:24.019364    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03bb2442-967f-465f-94bf-a209ae79842d-workload-certs\") pod \"e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr\" (UID: \"03bb2442-967f-465f-94bf-a209ae79842d\") " pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 17:56:24.019517 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:24.019393    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvcn4\" (UniqueName: \"kubernetes.io/projected/03bb2442-967f-465f-94bf-a209ae79842d-kube-api-access-vvcn4\") pod \"e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr\" (UID: \"03bb2442-967f-465f-94bf-a209ae79842d\") " pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 17:56:24.019517 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:24.019432    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03bb2442-967f-465f-94bf-a209ae79842d-workload-socket\") pod \"e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr\" (UID: \"03bb2442-967f-465f-94bf-a209ae79842d\") " pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 17:56:24.019517 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:24.019482    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03bb2442-967f-465f-94bf-a209ae79842d-credential-socket\") pod \"e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr\" (UID: \"03bb2442-967f-465f-94bf-a209ae79842d\") " pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 17:56:24.019656 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:24.019553    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03bb2442-967f-465f-94bf-a209ae79842d-istio-token\") pod \"e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr\" (UID: \"03bb2442-967f-465f-94bf-a209ae79842d\") " pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 17:56:24.019656 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:24.019574    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03bb2442-967f-465f-94bf-a209ae79842d-istio-envoy\") pod \"e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr\" (UID: \"03bb2442-967f-465f-94bf-a209ae79842d\") " pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 17:56:24.019656 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:24.019605    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03bb2442-967f-465f-94bf-a209ae79842d-istio-data\") pod \"e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr\" (UID: \"03bb2442-967f-465f-94bf-a209ae79842d\") " pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 17:56:24.019656 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:24.019634    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03bb2442-967f-465f-94bf-a209ae79842d-istio-podinfo\") pod \"e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr\" (UID: \"03bb2442-967f-465f-94bf-a209ae79842d\") " pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 17:56:24.026404 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:24.026335    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:56:24.120711 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:24.120677    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03bb2442-967f-465f-94bf-a209ae79842d-istio-token\") pod \"e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr\" (UID: \"03bb2442-967f-465f-94bf-a209ae79842d\") " pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 17:56:24.120711 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:24.120713    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03bb2442-967f-465f-94bf-a209ae79842d-istio-envoy\") pod \"e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr\" (UID: \"03bb2442-967f-465f-94bf-a209ae79842d\") " pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 17:56:24.120958 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:24.120734    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03bb2442-967f-465f-94bf-a209ae79842d-istio-data\") pod \"e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr\" (UID: \"03bb2442-967f-465f-94bf-a209ae79842d\") " pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 17:56:24.120958 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:24.120869    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03bb2442-967f-465f-94bf-a209ae79842d-istio-podinfo\") pod \"e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr\" (UID: \"03bb2442-967f-465f-94bf-a209ae79842d\") " pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 17:56:24.120958 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:24.120931    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03bb2442-967f-465f-94bf-a209ae79842d-istiod-ca-cert\") pod \"e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr\" (UID: \"03bb2442-967f-465f-94bf-a209ae79842d\") " pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 17:56:24.121164 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:24.121011    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03bb2442-967f-465f-94bf-a209ae79842d-workload-certs\") pod \"e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr\" (UID: \"03bb2442-967f-465f-94bf-a209ae79842d\") " pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 17:56:24.121164 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:24.121062    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvcn4\" (UniqueName: \"kubernetes.io/projected/03bb2442-967f-465f-94bf-a209ae79842d-kube-api-access-vvcn4\") pod \"e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr\" (UID: \"03bb2442-967f-465f-94bf-a209ae79842d\") " pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 17:56:24.121164 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:24.121129    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03bb2442-967f-465f-94bf-a209ae79842d-workload-socket\") pod \"e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr\" (UID: \"03bb2442-967f-465f-94bf-a209ae79842d\") " pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 17:56:24.121164 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:24.121155    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03bb2442-967f-465f-94bf-a209ae79842d-istio-data\") pod \"e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr\" (UID: \"03bb2442-967f-465f-94bf-a209ae79842d\") " pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 17:56:24.121367 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:24.121170    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03bb2442-967f-465f-94bf-a209ae79842d-credential-socket\") pod \"e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr\" (UID: \"03bb2442-967f-465f-94bf-a209ae79842d\") " pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 17:56:24.121426 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:24.121367    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03bb2442-967f-465f-94bf-a209ae79842d-workload-certs\") pod \"e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr\" (UID: \"03bb2442-967f-465f-94bf-a209ae79842d\") " pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 17:56:24.121487 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:24.121452    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03bb2442-967f-465f-94bf-a209ae79842d-credential-socket\") pod \"e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr\" (UID: \"03bb2442-967f-465f-94bf-a209ae79842d\") " pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 17:56:24.121532 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:24.121490    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03bb2442-967f-465f-94bf-a209ae79842d-workload-socket\") pod \"e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr\" (UID: \"03bb2442-967f-465f-94bf-a209ae79842d\") " pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 17:56:24.121934 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:24.121911    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03bb2442-967f-465f-94bf-a209ae79842d-istiod-ca-cert\") pod \"e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr\" (UID: \"03bb2442-967f-465f-94bf-a209ae79842d\") " pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 17:56:24.123366 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:24.123335    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03bb2442-967f-465f-94bf-a209ae79842d-istio-envoy\") pod \"e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr\" (UID: \"03bb2442-967f-465f-94bf-a209ae79842d\") " pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 17:56:24.123518 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:24.123504    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03bb2442-967f-465f-94bf-a209ae79842d-istio-podinfo\") pod \"e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr\" (UID: \"03bb2442-967f-465f-94bf-a209ae79842d\") " pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 17:56:24.131577 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:24.131554    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvcn4\" (UniqueName: \"kubernetes.io/projected/03bb2442-967f-465f-94bf-a209ae79842d-kube-api-access-vvcn4\") pod \"e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr\" (UID: \"03bb2442-967f-465f-94bf-a209ae79842d\") " pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 17:56:24.131804 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:24.131782    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03bb2442-967f-465f-94bf-a209ae79842d-istio-token\") pod \"e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr\" (UID: \"03bb2442-967f-465f-94bf-a209ae79842d\") " pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 17:56:24.221642 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:24.221538    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 17:56:24.357106 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:24.357075    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"]
Jul 24 17:56:24.360654 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:56:24.360620    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03bb2442_967f_465f_94bf_a209ae79842d.slice/crio-480b9ddbb6d616d111a4ad31f8e4d88ab8f9ed77c619e7ddfc1fe727e232c222 WatchSource:0}: Error finding container 480b9ddbb6d616d111a4ad31f8e4d88ab8f9ed77c619e7ddfc1fe727e232c222: Status 404 returned error can't find the container with id 480b9ddbb6d616d111a4ad31f8e4d88ab8f9ed77c619e7ddfc1fe727e232c222
Jul 24 17:56:24.362613 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:24.362587    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 17:56:25.032424 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:25.032392    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr" event={"ID":"03bb2442-967f-465f-94bf-a209ae79842d","Type":"ContainerStarted","Data":"480b9ddbb6d616d111a4ad31f8e4d88ab8f9ed77c619e7ddfc1fe727e232c222"}
Jul 24 17:56:27.225110 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:27.225055    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 17:56:27.225436 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:27.225142    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 17:56:27.225436 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:27.225174    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 17:56:28.050173 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:28.050137    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr" event={"ID":"03bb2442-967f-465f-94bf-a209ae79842d","Type":"ContainerStarted","Data":"855b9d431d980ce4bf2293189645f47feaebc290654dc379aafe77f8b01da87a"}
Jul 24 17:56:28.073119 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:28.073063    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr" podStartSLOduration=2.211076418 podStartE2EDuration="5.073027827s" totalImagesPullingTime="2.861951409s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:56:28.07133734 +0000 UTC m=+2755.273386622" watchObservedRunningTime="2026-07-24 17:56:28.073027827 +0000 UTC m=+2755.275077158"
Jul 24 17:56:28.222826 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:28.222773    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 17:56:28.227606 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:28.227582    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 17:56:29.054871 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:29.054839    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 17:56:29.055900 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:29.055879    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 17:56:30.019881 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:30.019835    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:56:30.246075 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:30.246020    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:56:30.366455 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:30.366375    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:56:30.793886 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:30.793853    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:56:30.897174 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:30.897134    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:56:34.529880 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:34.529842    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:56:36.156617 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:36.156581    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:56:44.056573 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:44.056531    2581 patch_prober.go:28] interesting pod/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 24 17:56:44.057011 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:44.056611    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr" podUID="03bb2442-967f-465f-94bf-a209ae79842d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 24 17:56:44.129017 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:44.128926    2581 generic.go:356] "Generic (PLEG): container finished" podID="03bb2442-967f-465f-94bf-a209ae79842d" containerID="855b9d431d980ce4bf2293189645f47feaebc290654dc379aafe77f8b01da87a" exitCode=0
Jul 24 17:56:44.129198 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:44.129014    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr" event={"ID":"03bb2442-967f-465f-94bf-a209ae79842d","Type":"ContainerDied","Data":"855b9d431d980ce4bf2293189645f47feaebc290654dc379aafe77f8b01da87a"}
Jul 24 17:56:44.129471 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:44.129456    2581 scope.go:117] "RemoveContainer" containerID="855b9d431d980ce4bf2293189645f47feaebc290654dc379aafe77f8b01da87a"
Jul 24 17:56:44.130504 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:44.130474    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 17:56:44.130629 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:44.130557    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 17:56:44.130629 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:44.130612    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 17:56:45.137358 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:45.137318    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr" event={"ID":"03bb2442-967f-465f-94bf-a209ae79842d","Type":"ContainerStarted","Data":"b65d96d11119fb351f53fe656d38961e9ae1daff6a24cde5c9436fd6866d63d1"}
Jul 24 17:56:45.222681 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:45.222631    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 17:56:45.222939 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:45.222912    2581 patch_prober.go:28] interesting pod/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 24 17:56:45.223030 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:45.222984    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr" podUID="03bb2442-967f-465f-94bf-a209ae79842d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 24 17:56:45.438917 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:45.438818    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:56:45.706092 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:45.705987    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"]
Jul 24 17:56:45.708680 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:45.708657    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 17:56:45.711631 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:45.711610    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-bd0742-openshift-default-dockercfg-pjh64\""
Jul 24 17:56:45.722291 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:45.722259    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"]
Jul 24 17:56:45.740105 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:45.740068    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-credential-socket\") pod \"e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b\" (UID: \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\") " pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 17:56:45.740302 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:45.740117    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-istio-podinfo\") pod \"e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b\" (UID: \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\") " pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 17:56:45.740302 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:45.740152    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhzsj\" (UniqueName: \"kubernetes.io/projected/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-kube-api-access-mhzsj\") pod \"e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b\" (UID: \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\") " pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 17:56:45.740434 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:45.740300    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-workload-certs\") pod \"e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b\" (UID: \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\") " pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 17:56:45.740434 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:45.740332    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-istio-envoy\") pod \"e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b\" (UID: \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\") " pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 17:56:45.740434 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:45.740408    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-istiod-ca-cert\") pod \"e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b\" (UID: \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\") " pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 17:56:45.740573 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:45.740448    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-istio-token\") pod \"e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b\" (UID: \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\") " pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 17:56:45.740573 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:45.740484    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-workload-socket\") pod \"e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b\" (UID: \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\") " pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 17:56:45.740573 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:45.740508    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-istio-data\") pod \"e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b\" (UID: \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\") " pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 17:56:45.740573 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:45.740533    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:56:45.841101 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:45.841029    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-workload-socket\") pod \"e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b\" (UID: \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\") " pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 17:56:45.841101 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:45.841096    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-istio-data\") pod \"e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b\" (UID: \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\") " pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 17:56:45.841364 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:45.841136    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-credential-socket\") pod \"e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b\" (UID: \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\") " pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 17:56:45.841364 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:45.841155    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-istio-podinfo\") pod \"e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b\" (UID: \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\") " pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 17:56:45.841364 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:45.841175    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhzsj\" (UniqueName: \"kubernetes.io/projected/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-kube-api-access-mhzsj\") pod \"e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b\" (UID: \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\") " pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 17:56:45.841364 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:45.841243    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-workload-certs\") pod \"e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b\" (UID: \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\") " pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 17:56:45.841364 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:45.841262    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-istio-envoy\") pod \"e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b\" (UID: \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\") " pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 17:56:45.841364 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:45.841313    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-istiod-ca-cert\") pod \"e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b\" (UID: \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\") " pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 17:56:45.841364 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:45.841339    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-istio-token\") pod \"e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b\" (UID: \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\") " pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 17:56:45.841723 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:45.841485    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-workload-socket\") pod \"e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b\" (UID: \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\") " pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 17:56:45.841723 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:45.841516    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-credential-socket\") pod \"e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b\" (UID: \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\") " pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 17:56:45.841723 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:45.841558    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-istio-data\") pod \"e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b\" (UID: \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\") " pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 17:56:45.841821 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:45.841746    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-workload-certs\") pod \"e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b\" (UID: \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\") " pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 17:56:45.842068 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:45.842021    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-istiod-ca-cert\") pod \"e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b\" (UID: \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\") " pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 17:56:45.843760 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:45.843738    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-istio-envoy\") pod \"e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b\" (UID: \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\") " pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 17:56:45.843927 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:45.843907    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-istio-podinfo\") pod \"e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b\" (UID: \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\") " pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 17:56:45.850433 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:45.850411    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhzsj\" (UniqueName: \"kubernetes.io/projected/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-kube-api-access-mhzsj\") pod \"e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b\" (UID: \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\") " pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 17:56:45.850433 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:45.850429    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-istio-token\") pod \"e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b\" (UID: \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\") " pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 17:56:45.908539 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:45.908459    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:56:46.020138 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:46.019998    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 17:56:46.193066 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:46.192784    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"]
Jul 24 17:56:46.198906 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:56:46.198869    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc43c7ad0_b0de_4d08_8ba1_c228038a04f1.slice/crio-6ffd8287f3a1c0ee462f9986eaa41e909456b09d9b59ab21dca50ff061a743d4 WatchSource:0}: Error finding container 6ffd8287f3a1c0ee462f9986eaa41e909456b09d9b59ab21dca50ff061a743d4: Status 404 returned error can't find the container with id 6ffd8287f3a1c0ee462f9986eaa41e909456b09d9b59ab21dca50ff061a743d4
Jul 24 17:56:46.201353 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:46.201318    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 17:56:46.201468 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:46.201403    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 17:56:46.201468 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:46.201444    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 17:56:46.221901 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:46.221875    2581 patch_prober.go:28] interesting pod/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 24 17:56:46.222000 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:46.221921    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr" podUID="03bb2442-967f-465f-94bf-a209ae79842d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 24 17:56:47.150733 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:47.150691    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b" event={"ID":"c43c7ad0-b0de-4d08-8ba1-c228038a04f1","Type":"ContainerStarted","Data":"d96d5ff79b57f7df9cfd8a83abc859a3033ca325455fb6c1a5e6a53054c1f769"}
Jul 24 17:56:47.150933 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:47.150756    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b" event={"ID":"c43c7ad0-b0de-4d08-8ba1-c228038a04f1","Type":"ContainerStarted","Data":"6ffd8287f3a1c0ee462f9986eaa41e909456b09d9b59ab21dca50ff061a743d4"}
Jul 24 17:56:47.188064 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:47.182783    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b" podStartSLOduration=2.18276247 podStartE2EDuration="2.18276247s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:56:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:56:47.182377153 +0000 UTC m=+2774.384426428" watchObservedRunningTime="2026-07-24 17:56:47.18276247 +0000 UTC m=+2774.384811747"
Jul 24 17:56:47.222675 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:47.222634    2581 patch_prober.go:28] interesting pod/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 24 17:56:47.223112 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:47.222692    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr" podUID="03bb2442-967f-465f-94bf-a209ae79842d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 24 17:56:48.020600 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:48.020549    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 17:56:48.025854 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:48.025830    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 17:56:48.155445 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:48.155409    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 17:56:48.156546 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:48.156527    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 17:56:48.222300 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:48.222264    2581 patch_prober.go:28] interesting pod/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 24 17:56:48.222484 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:48.222324    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr" podUID="03bb2442-967f-465f-94bf-a209ae79842d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 24 17:56:49.100420 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:49.100377    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:56:49.198095 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:49.198056    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:56:49.222446 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:49.222413    2581 patch_prober.go:28] interesting pod/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 24 17:56:49.222661 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:49.222472    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr" podUID="03bb2442-967f-465f-94bf-a209ae79842d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 24 17:56:49.297157 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:49.297121    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:56:49.472312 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:49.472280    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:56:49.842134 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:49.842048    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:56:49.945066 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:49.945008    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:56:50.226100 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:50.226070    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 17:56:50.226476 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:50.226311    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 17:56:50.227128 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:50.227108    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 17:56:53.611422 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:53.611336    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:56:58.765559 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:56:58.765519    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:57:07.247127 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:57:07.247092    2581 generic.go:356] "Generic (PLEG): container finished" podID="c43c7ad0-b0de-4d08-8ba1-c228038a04f1" containerID="d96d5ff79b57f7df9cfd8a83abc859a3033ca325455fb6c1a5e6a53054c1f769" exitCode=0
Jul 24 17:57:07.247589 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:57:07.247138    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b" event={"ID":"c43c7ad0-b0de-4d08-8ba1-c228038a04f1","Type":"ContainerDied","Data":"d96d5ff79b57f7df9cfd8a83abc859a3033ca325455fb6c1a5e6a53054c1f769"}
Jul 24 17:57:07.247642 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:57:07.247599    2581 scope.go:117] "RemoveContainer" containerID="d96d5ff79b57f7df9cfd8a83abc859a3033ca325455fb6c1a5e6a53054c1f769"
Jul 24 17:57:07.248738 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:57:07.248707    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 17:57:07.248794 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:57:07.248781    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 17:57:07.248830 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:57:07.248812    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 17:57:08.253663 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:57:08.253629    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b" event={"ID":"c43c7ad0-b0de-4d08-8ba1-c228038a04f1","Type":"ContainerStarted","Data":"a144bea9c43444b6e3a046d39a46a0a2828560c81e846940ec798b49a51f80c1"}
Jul 24 17:57:09.021010 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:57:09.020974    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 17:57:09.021215 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:57:09.021174    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 24 17:57:09.021269 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:57:09.021219    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b" podUID="c43c7ad0-b0de-4d08-8ba1-c228038a04f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 24 17:57:10.020483 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:57:10.020447    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 24 17:57:10.020893 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:57:10.020505    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b" podUID="c43c7ad0-b0de-4d08-8ba1-c228038a04f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 24 17:57:11.020712 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:57:11.020680    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 24 17:57:11.021125 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:57:11.020738    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b" podUID="c43c7ad0-b0de-4d08-8ba1-c228038a04f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 24 17:57:12.021458 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:57:12.021417    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 24 17:57:12.021881 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:57:12.021479    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b" podUID="c43c7ad0-b0de-4d08-8ba1-c228038a04f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 24 17:57:13.024704 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:57:13.024671    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 17:57:13.025136 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:57:13.025030    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 17:57:13.025879 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:57:13.025856    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 17:57:18.269773 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:57:18.269740    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:57:22.461415 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:57:22.461372    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:57:51.943570 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:57:51.943530    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:58:03.325681 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:03.325632    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:58:03.631791 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:03.631729    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"]
Jul 24 17:58:03.635323 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:03.635303    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:03.638228 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:03.638203    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-ab19c5-openshift-default-dockercfg-xwjn8\""
Jul 24 17:58:03.677583 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:03.677553    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"]
Jul 24 17:58:03.709084 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:03.709027    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55b15204-bf05-4d27-8592-c0fe180212c7-istiod-ca-cert\") pod \"e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7\" (UID: \"55b15204-bf05-4d27-8592-c0fe180212c7\") " pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:03.709273 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:03.709150    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55b15204-bf05-4d27-8592-c0fe180212c7-istio-data\") pod \"e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7\" (UID: \"55b15204-bf05-4d27-8592-c0fe180212c7\") " pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:03.709273 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:03.709202    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55b15204-bf05-4d27-8592-c0fe180212c7-istio-envoy\") pod \"e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7\" (UID: \"55b15204-bf05-4d27-8592-c0fe180212c7\") " pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:03.709273 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:03.709221    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55b15204-bf05-4d27-8592-c0fe180212c7-istio-token\") pod \"e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7\" (UID: \"55b15204-bf05-4d27-8592-c0fe180212c7\") " pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:03.709273 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:03.709269    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55b15204-bf05-4d27-8592-c0fe180212c7-workload-certs\") pod \"e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7\" (UID: \"55b15204-bf05-4d27-8592-c0fe180212c7\") " pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:03.709449 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:03.709297    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55b15204-bf05-4d27-8592-c0fe180212c7-credential-socket\") pod \"e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7\" (UID: \"55b15204-bf05-4d27-8592-c0fe180212c7\") " pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:03.709449 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:03.709328    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs52p\" (UniqueName: \"kubernetes.io/projected/55b15204-bf05-4d27-8592-c0fe180212c7-kube-api-access-zs52p\") pod \"e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7\" (UID: \"55b15204-bf05-4d27-8592-c0fe180212c7\") " pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:03.709449 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:03.709384    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55b15204-bf05-4d27-8592-c0fe180212c7-istio-podinfo\") pod \"e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7\" (UID: \"55b15204-bf05-4d27-8592-c0fe180212c7\") " pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:03.709449 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:03.709402    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55b15204-bf05-4d27-8592-c0fe180212c7-workload-socket\") pod \"e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7\" (UID: \"55b15204-bf05-4d27-8592-c0fe180212c7\") " pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:03.721902 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:03.721869    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:58:03.810224 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:03.810182    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55b15204-bf05-4d27-8592-c0fe180212c7-istiod-ca-cert\") pod \"e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7\" (UID: \"55b15204-bf05-4d27-8592-c0fe180212c7\") " pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:03.810432 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:03.810257    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55b15204-bf05-4d27-8592-c0fe180212c7-istio-data\") pod \"e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7\" (UID: \"55b15204-bf05-4d27-8592-c0fe180212c7\") " pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:03.810432 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:03.810288    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55b15204-bf05-4d27-8592-c0fe180212c7-istio-envoy\") pod \"e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7\" (UID: \"55b15204-bf05-4d27-8592-c0fe180212c7\") " pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:03.810432 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:03.810307    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55b15204-bf05-4d27-8592-c0fe180212c7-istio-token\") pod \"e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7\" (UID: \"55b15204-bf05-4d27-8592-c0fe180212c7\") " pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:03.810432 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:03.810345    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55b15204-bf05-4d27-8592-c0fe180212c7-workload-certs\") pod \"e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7\" (UID: \"55b15204-bf05-4d27-8592-c0fe180212c7\") " pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:03.810432 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:03.810381    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55b15204-bf05-4d27-8592-c0fe180212c7-credential-socket\") pod \"e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7\" (UID: \"55b15204-bf05-4d27-8592-c0fe180212c7\") " pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:03.810432 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:03.810406    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zs52p\" (UniqueName: \"kubernetes.io/projected/55b15204-bf05-4d27-8592-c0fe180212c7-kube-api-access-zs52p\") pod \"e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7\" (UID: \"55b15204-bf05-4d27-8592-c0fe180212c7\") " pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:03.810787 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:03.810440    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55b15204-bf05-4d27-8592-c0fe180212c7-istio-podinfo\") pod \"e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7\" (UID: \"55b15204-bf05-4d27-8592-c0fe180212c7\") " pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:03.810787 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:03.810469    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55b15204-bf05-4d27-8592-c0fe180212c7-workload-socket\") pod \"e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7\" (UID: \"55b15204-bf05-4d27-8592-c0fe180212c7\") " pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:03.810787 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:03.810736    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55b15204-bf05-4d27-8592-c0fe180212c7-istio-data\") pod \"e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7\" (UID: \"55b15204-bf05-4d27-8592-c0fe180212c7\") " pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:03.810912 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:03.810884    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55b15204-bf05-4d27-8592-c0fe180212c7-workload-certs\") pod \"e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7\" (UID: \"55b15204-bf05-4d27-8592-c0fe180212c7\") " pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:03.810912 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:03.810899    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55b15204-bf05-4d27-8592-c0fe180212c7-credential-socket\") pod \"e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7\" (UID: \"55b15204-bf05-4d27-8592-c0fe180212c7\") " pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:03.810989 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:03.810942    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55b15204-bf05-4d27-8592-c0fe180212c7-istiod-ca-cert\") pod \"e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7\" (UID: \"55b15204-bf05-4d27-8592-c0fe180212c7\") " pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:03.811136 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:03.811113    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55b15204-bf05-4d27-8592-c0fe180212c7-workload-socket\") pod \"e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7\" (UID: \"55b15204-bf05-4d27-8592-c0fe180212c7\") " pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:03.812917 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:03.812900    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55b15204-bf05-4d27-8592-c0fe180212c7-istio-envoy\") pod \"e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7\" (UID: \"55b15204-bf05-4d27-8592-c0fe180212c7\") " pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:03.813166 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:03.813148    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55b15204-bf05-4d27-8592-c0fe180212c7-istio-podinfo\") pod \"e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7\" (UID: \"55b15204-bf05-4d27-8592-c0fe180212c7\") " pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:03.822608 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:03.822574    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs52p\" (UniqueName: \"kubernetes.io/projected/55b15204-bf05-4d27-8592-c0fe180212c7-kube-api-access-zs52p\") pod \"e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7\" (UID: \"55b15204-bf05-4d27-8592-c0fe180212c7\") " pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:03.822727 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:03.822655    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55b15204-bf05-4d27-8592-c0fe180212c7-istio-token\") pod \"e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7\" (UID: \"55b15204-bf05-4d27-8592-c0fe180212c7\") " pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:03.932501 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:03.932410    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:58:03.945417 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:03.945395    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:04.099356 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:04.099326    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"]
Jul 24 17:58:04.101658 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:58:04.101632    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55b15204_bf05_4d27_8592_c0fe180212c7.slice/crio-3860c9d4b618bb7fdc4e3db37c8ba4df5943d17a2cbcca3fbe1895633e7c91a8 WatchSource:0}: Error finding container 3860c9d4b618bb7fdc4e3db37c8ba4df5943d17a2cbcca3fbe1895633e7c91a8: Status 404 returned error can't find the container with id 3860c9d4b618bb7fdc4e3db37c8ba4df5943d17a2cbcca3fbe1895633e7c91a8
Jul 24 17:58:04.103858 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:04.103822    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 17:58:04.103945 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:04.103904    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 17:58:04.103945 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:04.103942    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 17:58:04.533455 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:04.533423    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7" event={"ID":"55b15204-bf05-4d27-8592-c0fe180212c7","Type":"ContainerStarted","Data":"1c7adbf21c6508dba62e23e4b15f52355e5d6511008c78a11fe551cf61b3f399"}
Jul 24 17:58:04.533944 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:04.533471    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7" event={"ID":"55b15204-bf05-4d27-8592-c0fe180212c7","Type":"ContainerStarted","Data":"3860c9d4b618bb7fdc4e3db37c8ba4df5943d17a2cbcca3fbe1895633e7c91a8"}
Jul 24 17:58:04.561150 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:04.561103    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7" podStartSLOduration=1.561086653 podStartE2EDuration="1.561086653s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:58:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:58:04.559212051 +0000 UTC m=+2851.761261336" watchObservedRunningTime="2026-07-24 17:58:04.561086653 +0000 UTC m=+2851.763135917"
Jul 24 17:58:04.946175 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:04.946129    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:05.951226 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:05.951188    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:06.512083 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:06.512029    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:58:06.542798 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:06.542769    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:06.543728 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:06.543712    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:06.608931 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:06.608900    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:58:06.775650 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:06.775549    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:58:06.930429 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:06.930398    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:58:07.598593 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:07.598554    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:58:07.696925 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:07.696890    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:58:10.989406 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:10.989342    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:58:11.362427 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:11.362342    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:58:20.625707 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:20.625675    2581 generic.go:356] "Generic (PLEG): container finished" podID="55b15204-bf05-4d27-8592-c0fe180212c7" containerID="1c7adbf21c6508dba62e23e4b15f52355e5d6511008c78a11fe551cf61b3f399" exitCode=0
Jul 24 17:58:20.626181 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:20.625714    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7" event={"ID":"55b15204-bf05-4d27-8592-c0fe180212c7","Type":"ContainerDied","Data":"1c7adbf21c6508dba62e23e4b15f52355e5d6511008c78a11fe551cf61b3f399"}
Jul 24 17:58:20.626181 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:20.626135    2581 scope.go:117] "RemoveContainer" containerID="1c7adbf21c6508dba62e23e4b15f52355e5d6511008c78a11fe551cf61b3f399"
Jul 24 17:58:20.628849 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:20.628253    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 17:58:20.628849 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:20.628440    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 17:58:20.628849 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:20.628538    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 17:58:21.543520 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:21.543490    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:21.632803 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:21.632769    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7" event={"ID":"55b15204-bf05-4d27-8592-c0fe180212c7","Type":"ContainerStarted","Data":"a2534d7b5882b1e0f7a40f635463be16a27cf137d41927cea8e129c24e4fac64"}
Jul 24 17:58:21.895356 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:21.895322    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ab19c5-qhv2t"]
Jul 24 17:58:21.898238 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:21.898215    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ab19c5-qhv2t"
Jul 24 17:58:21.901730 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:21.901710    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d6flx\""
Jul 24 17:58:21.901860 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:21.901711    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 17:58:21.907991 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:21.907965    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ab19c5-qhv2t"]
Jul 24 17:58:21.945504 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:21.945478    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:21.946356 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:21.946337    2581 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 24 17:58:21.946420 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:21.946378    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7" podUID="55b15204-bf05-4d27-8592-c0fe180212c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 24 17:58:22.007072 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:22.007014    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1c44bfe-af3e-48da-8217-236ac8259a3e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-ab19c5-qhv2t\" (UID: \"c1c44bfe-af3e-48da-8217-236ac8259a3e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ab19c5-qhv2t"
Jul 24 17:58:22.108340 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:22.108303    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1c44bfe-af3e-48da-8217-236ac8259a3e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-ab19c5-qhv2t\" (UID: \"c1c44bfe-af3e-48da-8217-236ac8259a3e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ab19c5-qhv2t"
Jul 24 17:58:22.108970 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:22.108949    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1c44bfe-af3e-48da-8217-236ac8259a3e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-ab19c5-qhv2t\" (UID: \"c1c44bfe-af3e-48da-8217-236ac8259a3e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ab19c5-qhv2t"
Jul 24 17:58:22.209639 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:22.209541    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ab19c5-qhv2t"
Jul 24 17:58:22.383464 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:22.382866    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ab19c5-qhv2t"]
Jul 24 17:58:22.399116 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:58:22.399072    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1c44bfe_af3e_48da_8217_236ac8259a3e.slice/crio-2a89e1d9fc130e2e8800c2d9d2ac182fc8d4ab09cd4a21d09f4c5f0c52d4d8db WatchSource:0}: Error finding container 2a89e1d9fc130e2e8800c2d9d2ac182fc8d4ab09cd4a21d09f4c5f0c52d4d8db: Status 404 returned error can't find the container with id 2a89e1d9fc130e2e8800c2d9d2ac182fc8d4ab09cd4a21d09f4c5f0c52d4d8db
Jul 24 17:58:22.638887 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:22.638843    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ab19c5-qhv2t" event={"ID":"c1c44bfe-af3e-48da-8217-236ac8259a3e","Type":"ContainerStarted","Data":"2a89e1d9fc130e2e8800c2d9d2ac182fc8d4ab09cd4a21d09f4c5f0c52d4d8db"}
Jul 24 17:58:22.946466 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:22.946427    2581 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 24 17:58:22.946674 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:22.946481    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7" podUID="55b15204-bf05-4d27-8592-c0fe180212c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 24 17:58:23.644950 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:23.644908    2581 generic.go:356] "Generic (PLEG): container finished" podID="c1c44bfe-af3e-48da-8217-236ac8259a3e" containerID="6b03f3716f80a9fd59bd11de3f2a8858916ebdb57609409d9fadecc544d36fe1" exitCode=0
Jul 24 17:58:23.645382 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:23.645001    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ab19c5-qhv2t" event={"ID":"c1c44bfe-af3e-48da-8217-236ac8259a3e","Type":"ContainerDied","Data":"6b03f3716f80a9fd59bd11de3f2a8858916ebdb57609409d9fadecc544d36fe1"}
Jul 24 17:58:23.946254 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:23.946220    2581 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 24 17:58:23.946443 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:23.946276    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7" podUID="55b15204-bf05-4d27-8592-c0fe180212c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 24 17:58:24.820808 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:24.820783    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ab19c5-qhv2t"
Jul 24 17:58:24.937821 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:24.937728    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1c44bfe-af3e-48da-8217-236ac8259a3e-maas-api-service-ca\") pod \"c1c44bfe-af3e-48da-8217-236ac8259a3e\" (UID: \"c1c44bfe-af3e-48da-8217-236ac8259a3e\") "
Jul 24 17:58:24.938197 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:24.938171    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1c44bfe-af3e-48da-8217-236ac8259a3e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c1c44bfe-af3e-48da-8217-236ac8259a3e" (UID: "c1c44bfe-af3e-48da-8217-236ac8259a3e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:58:24.945998 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:24.945978    2581 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 24 17:58:24.946072 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:24.946026    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7" podUID="55b15204-bf05-4d27-8592-c0fe180212c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 24 17:58:25.039322 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:25.039290    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1c44bfe-af3e-48da-8217-236ac8259a3e-maas-api-service-ca\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:58:25.656435 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:25.656361    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ab19c5-qhv2t"
Jul 24 17:58:25.656435 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:25.656402    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ab19c5-qhv2t" event={"ID":"c1c44bfe-af3e-48da-8217-236ac8259a3e","Type":"ContainerDied","Data":"2a89e1d9fc130e2e8800c2d9d2ac182fc8d4ab09cd4a21d09f4c5f0c52d4d8db"}
Jul 24 17:58:25.656620 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:25.656448    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2a89e1d9fc130e2e8800c2d9d2ac182fc8d4ab09cd4a21d09f4c5f0c52d4d8db"
Jul 24 17:58:25.949905 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:25.949822    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:25.950327 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:25.950130    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:25.950765 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:25.950747    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:32.449294 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:32.449259    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:58:32.527231 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:32.527191    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:58:32.627759 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:32.627727    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:58:36.976539 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:36.976504    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:58:36.983938 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:36.983888    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:58:37.011523 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:37.011490    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ab19c5-qhv2t"]
Jul 24 17:58:37.020679 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:37.020644    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ab19c5-qhv2t"]
Jul 24 17:58:37.383951 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:37.383917    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1c44bfe-af3e-48da-8217-236ac8259a3e" path="/var/lib/kubelet/pods/c1c44bfe-af3e-48da-8217-236ac8259a3e/volumes"
Jul 24 17:58:37.403061 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:37.403018    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:58:43.489572 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:43.489522    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:58:43.565114 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:43.565080    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:58:43.732692 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:43.732652    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"]
Jul 24 17:58:43.732960 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:43.732904    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7" podUID="55b15204-bf05-4d27-8592-c0fe180212c7" containerName="istio-proxy" containerID="cri-o://a2534d7b5882b1e0f7a40f635463be16a27cf137d41927cea8e129c24e4fac64" gracePeriod=30
Jul 24 17:58:43.749082 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:43.748990    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:58:43.980734 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:43.980702    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:58:47.336380 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:47.336346    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a6414-nx8l4"]
Jul 24 17:58:47.336846 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:47.336804    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1c44bfe-af3e-48da-8217-236ac8259a3e" containerName="revoke-keys"
Jul 24 17:58:47.336846 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:47.336817    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1c44bfe-af3e-48da-8217-236ac8259a3e" containerName="revoke-keys"
Jul 24 17:58:47.336924 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:47.336887    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1c44bfe-af3e-48da-8217-236ac8259a3e" containerName="revoke-keys"
Jul 24 17:58:47.339354 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:47.339336    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a6414-nx8l4"
Jul 24 17:58:47.342085 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:47.342061    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d6flx\""
Jul 24 17:58:47.342192 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:47.342123    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 17:58:47.347087 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:47.347064    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a6414-nx8l4"]
Jul 24 17:58:47.357603 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:47.357572    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17333a24-ace8-4a6d-aa01-e6722dc22b77-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-8a6414-nx8l4\" (UID: \"17333a24-ace8-4a6d-aa01-e6722dc22b77\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a6414-nx8l4"
Jul 24 17:58:47.458350 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:47.458315    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17333a24-ace8-4a6d-aa01-e6722dc22b77-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-8a6414-nx8l4\" (UID: \"17333a24-ace8-4a6d-aa01-e6722dc22b77\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a6414-nx8l4"
Jul 24 17:58:47.458997 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:47.458971    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17333a24-ace8-4a6d-aa01-e6722dc22b77-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-8a6414-nx8l4\" (UID: \"17333a24-ace8-4a6d-aa01-e6722dc22b77\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a6414-nx8l4"
Jul 24 17:58:47.651242 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:47.651210    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a6414-nx8l4"
Jul 24 17:58:47.782944 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:47.782913    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a6414-nx8l4"]
Jul 24 17:58:47.784672 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:58:47.784647    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17333a24_ace8_4a6d_aa01_e6722dc22b77.slice/crio-717b6e879a028acaf2124f1372232c9d9da8049becc87502295bc015c090749e WatchSource:0}: Error finding container 717b6e879a028acaf2124f1372232c9d9da8049becc87502295bc015c090749e: Status 404 returned error can't find the container with id 717b6e879a028acaf2124f1372232c9d9da8049becc87502295bc015c090749e
Jul 24 17:58:48.774084 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:48.774023    2581 generic.go:356] "Generic (PLEG): container finished" podID="17333a24-ace8-4a6d-aa01-e6722dc22b77" containerID="987d213b15b1f8371ec83efce73be21c8c6bee6c4f962c4f32b79efa2ba7a747" exitCode=0
Jul 24 17:58:48.774419 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:48.774112    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a6414-nx8l4" event={"ID":"17333a24-ace8-4a6d-aa01-e6722dc22b77","Type":"ContainerDied","Data":"987d213b15b1f8371ec83efce73be21c8c6bee6c4f962c4f32b79efa2ba7a747"}
Jul 24 17:58:48.774419 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:48.774196    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a6414-nx8l4" event={"ID":"17333a24-ace8-4a6d-aa01-e6722dc22b77","Type":"ContainerStarted","Data":"717b6e879a028acaf2124f1372232c9d9da8049becc87502295bc015c090749e"}
Jul 24 17:58:48.988391 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:48.988358    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:49.073916 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.073830    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55b15204-bf05-4d27-8592-c0fe180212c7-workload-socket\") pod \"55b15204-bf05-4d27-8592-c0fe180212c7\" (UID: \"55b15204-bf05-4d27-8592-c0fe180212c7\") "
Jul 24 17:58:49.073916 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.073874    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55b15204-bf05-4d27-8592-c0fe180212c7-istio-podinfo\") pod \"55b15204-bf05-4d27-8592-c0fe180212c7\" (UID: \"55b15204-bf05-4d27-8592-c0fe180212c7\") "
Jul 24 17:58:49.073916 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.073902    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55b15204-bf05-4d27-8592-c0fe180212c7-istiod-ca-cert\") pod \"55b15204-bf05-4d27-8592-c0fe180212c7\" (UID: \"55b15204-bf05-4d27-8592-c0fe180212c7\") "
Jul 24 17:58:49.074224 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.073938    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55b15204-bf05-4d27-8592-c0fe180212c7-istio-envoy\") pod \"55b15204-bf05-4d27-8592-c0fe180212c7\" (UID: \"55b15204-bf05-4d27-8592-c0fe180212c7\") "
Jul 24 17:58:49.074224 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.073968    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55b15204-bf05-4d27-8592-c0fe180212c7-istio-data\") pod \"55b15204-bf05-4d27-8592-c0fe180212c7\" (UID: \"55b15204-bf05-4d27-8592-c0fe180212c7\") "
Jul 24 17:58:49.074224 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.074001    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55b15204-bf05-4d27-8592-c0fe180212c7-credential-socket\") pod \"55b15204-bf05-4d27-8592-c0fe180212c7\" (UID: \"55b15204-bf05-4d27-8592-c0fe180212c7\") "
Jul 24 17:58:49.074224 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.074070    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zs52p\" (UniqueName: \"kubernetes.io/projected/55b15204-bf05-4d27-8592-c0fe180212c7-kube-api-access-zs52p\") pod \"55b15204-bf05-4d27-8592-c0fe180212c7\" (UID: \"55b15204-bf05-4d27-8592-c0fe180212c7\") "
Jul 24 17:58:49.074224 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.074103    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55b15204-bf05-4d27-8592-c0fe180212c7-workload-certs\") pod \"55b15204-bf05-4d27-8592-c0fe180212c7\" (UID: \"55b15204-bf05-4d27-8592-c0fe180212c7\") "
Jul 24 17:58:49.074477 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.074295    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55b15204-bf05-4d27-8592-c0fe180212c7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "55b15204-bf05-4d27-8592-c0fe180212c7" (UID: "55b15204-bf05-4d27-8592-c0fe180212c7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:58:49.074477 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.074341    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55b15204-bf05-4d27-8592-c0fe180212c7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "55b15204-bf05-4d27-8592-c0fe180212c7" (UID: "55b15204-bf05-4d27-8592-c0fe180212c7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:58:49.074477 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.074372    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55b15204-bf05-4d27-8592-c0fe180212c7-istio-token\") pod \"55b15204-bf05-4d27-8592-c0fe180212c7\" (UID: \"55b15204-bf05-4d27-8592-c0fe180212c7\") "
Jul 24 17:58:49.074477 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.074408    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55b15204-bf05-4d27-8592-c0fe180212c7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "55b15204-bf05-4d27-8592-c0fe180212c7" (UID: "55b15204-bf05-4d27-8592-c0fe180212c7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:58:49.074477 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.074427    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55b15204-bf05-4d27-8592-c0fe180212c7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "55b15204-bf05-4d27-8592-c0fe180212c7" (UID: "55b15204-bf05-4d27-8592-c0fe180212c7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:58:49.074790 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.074768    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55b15204-bf05-4d27-8592-c0fe180212c7-istiod-ca-cert\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:58:49.074790 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.074795    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55b15204-bf05-4d27-8592-c0fe180212c7-credential-socket\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:58:49.074790 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.074807    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55b15204-bf05-4d27-8592-c0fe180212c7-workload-certs\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:58:49.074790 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.074820    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55b15204-bf05-4d27-8592-c0fe180212c7-workload-socket\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:58:49.075245 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.075119    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55b15204-bf05-4d27-8592-c0fe180212c7-istio-data" (OuterVolumeSpecName: "istio-data") pod "55b15204-bf05-4d27-8592-c0fe180212c7" (UID: "55b15204-bf05-4d27-8592-c0fe180212c7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:58:49.076616 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.076586    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55b15204-bf05-4d27-8592-c0fe180212c7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "55b15204-bf05-4d27-8592-c0fe180212c7" (UID: "55b15204-bf05-4d27-8592-c0fe180212c7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:58:49.076752 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.076634    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55b15204-bf05-4d27-8592-c0fe180212c7-istio-token" (OuterVolumeSpecName: "istio-token") pod "55b15204-bf05-4d27-8592-c0fe180212c7" (UID: "55b15204-bf05-4d27-8592-c0fe180212c7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:58:49.076752 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.076736    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/55b15204-bf05-4d27-8592-c0fe180212c7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "55b15204-bf05-4d27-8592-c0fe180212c7" (UID: "55b15204-bf05-4d27-8592-c0fe180212c7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 17:58:49.076752 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.076734    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55b15204-bf05-4d27-8592-c0fe180212c7-kube-api-access-zs52p" (OuterVolumeSpecName: "kube-api-access-zs52p") pod "55b15204-bf05-4d27-8592-c0fe180212c7" (UID: "55b15204-bf05-4d27-8592-c0fe180212c7"). InnerVolumeSpecName "kube-api-access-zs52p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:58:49.176016 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.175980    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zs52p\" (UniqueName: \"kubernetes.io/projected/55b15204-bf05-4d27-8592-c0fe180212c7-kube-api-access-zs52p\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:58:49.176016 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.176010    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55b15204-bf05-4d27-8592-c0fe180212c7-istio-token\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:58:49.176016 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.176020    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55b15204-bf05-4d27-8592-c0fe180212c7-istio-podinfo\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:58:49.176016 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.176028    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55b15204-bf05-4d27-8592-c0fe180212c7-istio-envoy\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:58:49.176305 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.176055    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55b15204-bf05-4d27-8592-c0fe180212c7-istio-data\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:58:49.782966 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.782880    2581 generic.go:356] "Generic (PLEG): container finished" podID="55b15204-bf05-4d27-8592-c0fe180212c7" containerID="a2534d7b5882b1e0f7a40f635463be16a27cf137d41927cea8e129c24e4fac64" exitCode=0
Jul 24 17:58:49.782966 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.782930    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"
Jul 24 17:58:49.783503 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.782963    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7" event={"ID":"55b15204-bf05-4d27-8592-c0fe180212c7","Type":"ContainerDied","Data":"a2534d7b5882b1e0f7a40f635463be16a27cf137d41927cea8e129c24e4fac64"}
Jul 24 17:58:49.783503 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.783018    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7" event={"ID":"55b15204-bf05-4d27-8592-c0fe180212c7","Type":"ContainerDied","Data":"3860c9d4b618bb7fdc4e3db37c8ba4df5943d17a2cbcca3fbe1895633e7c91a8"}
Jul 24 17:58:49.783503 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.783071    2581 scope.go:117] "RemoveContainer" containerID="a2534d7b5882b1e0f7a40f635463be16a27cf137d41927cea8e129c24e4fac64"
Jul 24 17:58:49.792752 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.792730    2581 scope.go:117] "RemoveContainer" containerID="1c7adbf21c6508dba62e23e4b15f52355e5d6511008c78a11fe551cf61b3f399"
Jul 24 17:58:49.802771 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.802743    2581 scope.go:117] "RemoveContainer" containerID="a2534d7b5882b1e0f7a40f635463be16a27cf137d41927cea8e129c24e4fac64"
Jul 24 17:58:49.803409 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:58:49.803383    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2534d7b5882b1e0f7a40f635463be16a27cf137d41927cea8e129c24e4fac64\": container with ID starting with a2534d7b5882b1e0f7a40f635463be16a27cf137d41927cea8e129c24e4fac64 not found: ID does not exist" containerID="a2534d7b5882b1e0f7a40f635463be16a27cf137d41927cea8e129c24e4fac64"
Jul 24 17:58:49.803507 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.803416    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2534d7b5882b1e0f7a40f635463be16a27cf137d41927cea8e129c24e4fac64"} err="failed to get container status \"a2534d7b5882b1e0f7a40f635463be16a27cf137d41927cea8e129c24e4fac64\": rpc error: code = NotFound desc = could not find container \"a2534d7b5882b1e0f7a40f635463be16a27cf137d41927cea8e129c24e4fac64\": container with ID starting with a2534d7b5882b1e0f7a40f635463be16a27cf137d41927cea8e129c24e4fac64 not found: ID does not exist"
Jul 24 17:58:49.803507 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.803435    2581 scope.go:117] "RemoveContainer" containerID="1c7adbf21c6508dba62e23e4b15f52355e5d6511008c78a11fe551cf61b3f399"
Jul 24 17:58:49.803950 ip-10-0-139-46 kubenswrapper[2581]: E0724 17:58:49.803927    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c7adbf21c6508dba62e23e4b15f52355e5d6511008c78a11fe551cf61b3f399\": container with ID starting with 1c7adbf21c6508dba62e23e4b15f52355e5d6511008c78a11fe551cf61b3f399 not found: ID does not exist" containerID="1c7adbf21c6508dba62e23e4b15f52355e5d6511008c78a11fe551cf61b3f399"
Jul 24 17:58:49.804073 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.803955    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c7adbf21c6508dba62e23e4b15f52355e5d6511008c78a11fe551cf61b3f399"} err="failed to get container status \"1c7adbf21c6508dba62e23e4b15f52355e5d6511008c78a11fe551cf61b3f399\": rpc error: code = NotFound desc = could not find container \"1c7adbf21c6508dba62e23e4b15f52355e5d6511008c78a11fe551cf61b3f399\": container with ID starting with 1c7adbf21c6508dba62e23e4b15f52355e5d6511008c78a11fe551cf61b3f399 not found: ID does not exist"
Jul 24 17:58:49.804132 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.804098    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"]
Jul 24 17:58:49.808695 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.808666    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-ab19c5-openshift-default-5877dfdb54-ltnm7"]
Jul 24 17:58:49.924413 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.924386    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a6414-nx8l4"
Jul 24 17:58:49.982303 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.982269    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17333a24-ace8-4a6d-aa01-e6722dc22b77-maas-api-service-ca\") pod \"17333a24-ace8-4a6d-aa01-e6722dc22b77\" (UID: \"17333a24-ace8-4a6d-aa01-e6722dc22b77\") "
Jul 24 17:58:49.982652 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:49.982631    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17333a24-ace8-4a6d-aa01-e6722dc22b77-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "17333a24-ace8-4a6d-aa01-e6722dc22b77" (UID: "17333a24-ace8-4a6d-aa01-e6722dc22b77"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:58:50.083368 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:50.083293    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17333a24-ace8-4a6d-aa01-e6722dc22b77-maas-api-service-ca\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 17:58:50.790366 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:50.790337    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a6414-nx8l4"
Jul 24 17:58:50.790812 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:50.790375    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a6414-nx8l4" event={"ID":"17333a24-ace8-4a6d-aa01-e6722dc22b77","Type":"ContainerDied","Data":"717b6e879a028acaf2124f1372232c9d9da8049becc87502295bc015c090749e"}
Jul 24 17:58:50.790812 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:50.790433    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="717b6e879a028acaf2124f1372232c9d9da8049becc87502295bc015c090749e"
Jul 24 17:58:51.384780 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:58:51.384734    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55b15204-bf05-4d27-8592-c0fe180212c7" path="/var/lib/kubelet/pods/55b15204-bf05-4d27-8592-c0fe180212c7/volumes"
Jul 24 17:59:01.384512 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:01.384481    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:59:02.797318 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:02.797268    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:59:02.903220 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:02.903124    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:59:02.999211 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:02.999166    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:59:07.420699 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:07.420662    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a6414-nx8l4"]
Jul 24 17:59:07.425139 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:07.425113    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:59:07.425507 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:07.425487    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a6414-nx8l4"]
Jul 24 17:59:07.816716 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:07.816601    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:59:09.383902 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:09.383855    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17333a24-ace8-4a6d-aa01-e6722dc22b77" path="/var/lib/kubelet/pods/17333a24-ace8-4a6d-aa01-e6722dc22b77/volumes"
Jul 24 17:59:12.477477 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:12.477434    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 17:59:12.894013 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:12.893952    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:59:46.986304 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:46.986260    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:59:47.090726 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:47.090691    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:59:49.988534 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:49.988491    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:59:50.758433 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:50.758396    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:59:56.290091 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:56.289989    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:59:56.357572 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:56.357536    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:59:56.553456 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:56.553367    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"]
Jul 24 17:59:56.553612 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:56.553567    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr" podUID="03bb2442-967f-465f-94bf-a209ae79842d" containerName="istio-proxy" containerID="cri-o://b65d96d11119fb351f53fe656d38961e9ae1daff6a24cde5c9436fd6866d63d1" gracePeriod=30
Jul 24 17:59:56.579180 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:56.579149    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:59:56.867302 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:56.867208    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 17:59:56.947886 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:56.947847    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bd0742-sgspp"]
Jul 24 17:59:56.948288 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:56.948275    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55b15204-bf05-4d27-8592-c0fe180212c7" containerName="istio-proxy"
Jul 24 17:59:56.948350 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:56.948291    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="55b15204-bf05-4d27-8592-c0fe180212c7" containerName="istio-proxy"
Jul 24 17:59:56.948350 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:56.948301    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55b15204-bf05-4d27-8592-c0fe180212c7" containerName="istio-proxy"
Jul 24 17:59:56.948350 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:56.948306    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="55b15204-bf05-4d27-8592-c0fe180212c7" containerName="istio-proxy"
Jul 24 17:59:56.948350 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:56.948322    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17333a24-ace8-4a6d-aa01-e6722dc22b77" containerName="revoke-keys"
Jul 24 17:59:56.948350 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:56.948328    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="17333a24-ace8-4a6d-aa01-e6722dc22b77" containerName="revoke-keys"
Jul 24 17:59:56.948528 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:56.948396    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="55b15204-bf05-4d27-8592-c0fe180212c7" containerName="istio-proxy"
Jul 24 17:59:56.948528 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:56.948403    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="55b15204-bf05-4d27-8592-c0fe180212c7" containerName="istio-proxy"
Jul 24 17:59:56.948528 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:56.948414    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="17333a24-ace8-4a6d-aa01-e6722dc22b77" containerName="revoke-keys"
Jul 24 17:59:56.951311 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:56.951295    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bd0742-sgspp"
Jul 24 17:59:56.953909 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:56.953889    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 17:59:56.953994 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:56.953927    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d6flx\""
Jul 24 17:59:56.958842 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:56.958819    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bd0742-sgspp"]
Jul 24 17:59:57.047025 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:57.046980    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cedea9ce-1da9-4643-b398-8c683518763c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-bd0742-sgspp\" (UID: \"cedea9ce-1da9-4643-b398-8c683518763c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bd0742-sgspp"
Jul 24 17:59:57.148678 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:57.148637    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cedea9ce-1da9-4643-b398-8c683518763c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-bd0742-sgspp\" (UID: \"cedea9ce-1da9-4643-b398-8c683518763c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bd0742-sgspp"
Jul 24 17:59:57.149328 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:57.149307    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cedea9ce-1da9-4643-b398-8c683518763c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-bd0742-sgspp\" (UID: \"cedea9ce-1da9-4643-b398-8c683518763c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bd0742-sgspp"
Jul 24 17:59:57.263603 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:57.263554    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bd0742-sgspp"
Jul 24 17:59:57.400079 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:57.400031    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bd0742-sgspp"]
Jul 24 17:59:57.402741 ip-10-0-139-46 kubenswrapper[2581]: W0724 17:59:57.402715    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcedea9ce_1da9_4643_b398_8c683518763c.slice/crio-fee83d0f9785aaf1b2e8a9561b8ad3d836de06431d944cdf6a1862f8cdb05668 WatchSource:0}: Error finding container fee83d0f9785aaf1b2e8a9561b8ad3d836de06431d944cdf6a1862f8cdb05668: Status 404 returned error can't find the container with id fee83d0f9785aaf1b2e8a9561b8ad3d836de06431d944cdf6a1862f8cdb05668
Jul 24 17:59:58.141878 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:58.141849    2581 generic.go:356] "Generic (PLEG): container finished" podID="cedea9ce-1da9-4643-b398-8c683518763c" containerID="a83cbe99479fe2db45c39d8f4469f95c7e1590566dabcc23a7e88b80e66ad333" exitCode=0
Jul 24 17:59:58.142071 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:58.141906    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bd0742-sgspp" event={"ID":"cedea9ce-1da9-4643-b398-8c683518763c","Type":"ContainerDied","Data":"a83cbe99479fe2db45c39d8f4469f95c7e1590566dabcc23a7e88b80e66ad333"}
Jul 24 17:59:58.142071 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:58.141943    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bd0742-sgspp" event={"ID":"cedea9ce-1da9-4643-b398-8c683518763c","Type":"ContainerStarted","Data":"fee83d0f9785aaf1b2e8a9561b8ad3d836de06431d944cdf6a1862f8cdb05668"}
Jul 24 17:59:59.279224 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:59.279171    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bd0742-sgspp"
Jul 24 17:59:59.371326 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:59.371292    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cedea9ce-1da9-4643-b398-8c683518763c-maas-api-service-ca\") pod \"cedea9ce-1da9-4643-b398-8c683518763c\" (UID: \"cedea9ce-1da9-4643-b398-8c683518763c\") "
Jul 24 17:59:59.371716 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:59.371688    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cedea9ce-1da9-4643-b398-8c683518763c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cedea9ce-1da9-4643-b398-8c683518763c" (UID: "cedea9ce-1da9-4643-b398-8c683518763c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:59:59.472825 ip-10-0-139-46 kubenswrapper[2581]: I0724 17:59:59.472731    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cedea9ce-1da9-4643-b398-8c683518763c-maas-api-service-ca\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 18:00:00.144588 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:00.144552    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748600-hg857"]
Jul 24 18:00:00.144987 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:00.144975    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cedea9ce-1da9-4643-b398-8c683518763c" containerName="revoke-keys"
Jul 24 18:00:00.145033 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:00.144989    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="cedea9ce-1da9-4643-b398-8c683518763c" containerName="revoke-keys"
Jul 24 18:00:00.145093 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:00.145083    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="cedea9ce-1da9-4643-b398-8c683518763c" containerName="revoke-keys"
Jul 24 18:00:00.148228 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:00.148201    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748600-hg857"
Jul 24 18:00:00.152598 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:00.152574    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bb94b3-29748600-r9989"]
Jul 24 18:00:00.155627 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:00.155608    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bd0742-sgspp"
Jul 24 18:00:00.155977 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:00.155955    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bd0742-sgspp" event={"ID":"cedea9ce-1da9-4643-b398-8c683518763c","Type":"ContainerDied","Data":"fee83d0f9785aaf1b2e8a9561b8ad3d836de06431d944cdf6a1862f8cdb05668"}
Jul 24 18:00:00.156109 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:00.155993    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fee83d0f9785aaf1b2e8a9561b8ad3d836de06431d944cdf6a1862f8cdb05668"
Jul 24 18:00:00.156109 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:00.156010    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-bd0742-29748600-pffsq"]
Jul 24 18:00:00.156109 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:00.156033    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bb94b3-29748600-r9989"
Jul 24 18:00:00.159534 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:00.159515    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5c7cfc-29748600-xsx8w"]
Jul 24 18:00:00.159638 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:00.159619    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-bd0742-29748600-pffsq"
Jul 24 18:00:00.162502 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:00.162359    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748600-hg857"
Jul 24 18:00:00.163767 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:00.163738    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5c7cfc-29748600-xsx8w"
Jul 24 18:00:00.163856 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:00.163807    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bb94b3-29748600-r9989"]
Jul 24 18:00:00.168266 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:00.168211    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748600-hg857"]
Jul 24 18:00:00.171521 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:00.171498    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5c7cfc-29748600-xsx8w"]
Jul 24 18:00:00.175988 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:00.175819    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bb94b3-29748600-r9989"
Jul 24 18:00:00.182874 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:00.182849    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-bd0742-29748600-pffsq"]
Jul 24 18:00:00.214537 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:00.214490    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-bd0742-29748600-pffsq"
Jul 24 18:00:00.223376 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:00.223353    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5c7cfc-29748600-xsx8w"
Jul 24 18:00:00.347287 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:00.347257    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748600-hg857"]
Jul 24 18:00:00.386172 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:00.384958    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bb94b3-29748600-r9989"]
Jul 24 18:00:00.393677 ip-10-0-139-46 kubenswrapper[2581]: W0724 18:00:00.393611    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24731052_120c_4686_8aed_ebf14acac5b3.slice/crio-451a38d4721f0e9c9de22d75ac384dd5e1822bc6b93054ff091d3c5305fb8a44 WatchSource:0}: Error finding container 451a38d4721f0e9c9de22d75ac384dd5e1822bc6b93054ff091d3c5305fb8a44: Status 404 returned error can't find the container with id 451a38d4721f0e9c9de22d75ac384dd5e1822bc6b93054ff091d3c5305fb8a44
Jul 24 18:00:00.414481 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:00.414449    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-bd0742-29748600-pffsq"]
Jul 24 18:00:00.417795 ip-10-0-139-46 kubenswrapper[2581]: W0724 18:00:00.417755    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a27dd7b_035d_4346_bc89_6792b98b64b7.slice/crio-b5fe04acc273afe2b303109746e1c6bec57ac4899a75d294be3e21fa36a4ed7d WatchSource:0}: Error finding container b5fe04acc273afe2b303109746e1c6bec57ac4899a75d294be3e21fa36a4ed7d: Status 404 returned error can't find the container with id b5fe04acc273afe2b303109746e1c6bec57ac4899a75d294be3e21fa36a4ed7d
Jul 24 18:00:00.434562 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:00.434541    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5c7cfc-29748600-xsx8w"]
Jul 24 18:00:00.442217 ip-10-0-139-46 kubenswrapper[2581]: W0724 18:00:00.442181    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65abf56e_4d80_4a4f_9cf0_a83c118cc549.slice/crio-2034dfdaa1a40c06274ca3e4b55e87971a9dbbe21fb080f95078a0a175ab943d WatchSource:0}: Error finding container 2034dfdaa1a40c06274ca3e4b55e87971a9dbbe21fb080f95078a0a175ab943d: Status 404 returned error can't find the container with id 2034dfdaa1a40c06274ca3e4b55e87971a9dbbe21fb080f95078a0a175ab943d
Jul 24 18:00:01.162309 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:01.162271    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bb94b3-29748600-r9989" event={"ID":"24731052-120c-4686-8aed-ebf14acac5b3","Type":"ContainerStarted","Data":"b40c51a20ad6d950255cab69d1d8f6ebb4073fbdd34acfa92db0ff3322e2c66f"}
Jul 24 18:00:01.162506 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:01.162365    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bb94b3-29748600-r9989" event={"ID":"24731052-120c-4686-8aed-ebf14acac5b3","Type":"ContainerStarted","Data":"451a38d4721f0e9c9de22d75ac384dd5e1822bc6b93054ff091d3c5305fb8a44"}
Jul 24 18:00:01.163921 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:01.163894    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748600-hg857" event={"ID":"455d7a4c-cc1d-4e2a-adae-01ccd780a242","Type":"ContainerStarted","Data":"dc98283de5cc74c8b4465c85980beba1d011755a309c25eb8b2053604f2fc3e0"}
Jul 24 18:00:01.164071 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:01.163943    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748600-hg857" event={"ID":"455d7a4c-cc1d-4e2a-adae-01ccd780a242","Type":"ContainerStarted","Data":"5fbeb39b326c9e8b0cd6e13fb5bd6964f95a2a49866dfd2dabce21358ef25fb7"}
Jul 24 18:00:01.165349 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:01.165324    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5c7cfc-29748600-xsx8w" event={"ID":"65abf56e-4d80-4a4f-9cf0-a83c118cc549","Type":"ContainerStarted","Data":"34abeabfe1c8400889434ca4d5fc54068ea453f9ae74659a26a62d643b19802e"}
Jul 24 18:00:01.165439 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:01.165375    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5c7cfc-29748600-xsx8w" event={"ID":"65abf56e-4d80-4a4f-9cf0-a83c118cc549","Type":"ContainerStarted","Data":"2034dfdaa1a40c06274ca3e4b55e87971a9dbbe21fb080f95078a0a175ab943d"}
Jul 24 18:00:01.166803 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:01.166779    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-bd0742-29748600-pffsq" event={"ID":"8a27dd7b-035d-4346-bc89-6792b98b64b7","Type":"ContainerStarted","Data":"626185ec7a231fc5db852917f0bf5215a772edca911c01a59b08f2794e5adeea"}
Jul 24 18:00:01.166891 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:01.166822    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-bd0742-29748600-pffsq" event={"ID":"8a27dd7b-035d-4346-bc89-6792b98b64b7","Type":"ContainerStarted","Data":"b5fe04acc273afe2b303109746e1c6bec57ac4899a75d294be3e21fa36a4ed7d"}
Jul 24 18:00:01.192245 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:01.192196    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bb94b3-29748600-r9989" podStartSLOduration=1.192182413 podStartE2EDuration="1.192182413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:00:01.191072947 +0000 UTC m=+2968.393122219" watchObservedRunningTime="2026-07-24 18:00:01.192182413 +0000 UTC m=+2968.394231686"
Jul 24 18:00:01.230227 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:01.230173    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5c7cfc-29748600-xsx8w" podStartSLOduration=1.230155913 podStartE2EDuration="1.230155913s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:00:01.228892358 +0000 UTC m=+2968.430941632" watchObservedRunningTime="2026-07-24 18:00:01.230155913 +0000 UTC m=+2968.432205232"
Jul 24 18:00:01.267745 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:01.267687    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748600-hg857" podStartSLOduration=1.267670501 podStartE2EDuration="1.267670501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:00:01.266092707 +0000 UTC m=+2968.468141982" watchObservedRunningTime="2026-07-24 18:00:01.267670501 +0000 UTC m=+2968.469719775"
Jul 24 18:00:01.290396 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:01.290334    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-bd0742-29748600-pffsq" podStartSLOduration=1.290318058 podStartE2EDuration="1.290318058s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:00:01.289351263 +0000 UTC m=+2968.491400538" watchObservedRunningTime="2026-07-24 18:00:01.290318058 +0000 UTC m=+2968.492367332"
Jul 24 18:00:01.813836 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:01.813810    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 18:00:01.903726 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:01.903692    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvcn4\" (UniqueName: \"kubernetes.io/projected/03bb2442-967f-465f-94bf-a209ae79842d-kube-api-access-vvcn4\") pod \"03bb2442-967f-465f-94bf-a209ae79842d\" (UID: \"03bb2442-967f-465f-94bf-a209ae79842d\") "
Jul 24 18:00:01.903726 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:01.903728    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03bb2442-967f-465f-94bf-a209ae79842d-istio-data\") pod \"03bb2442-967f-465f-94bf-a209ae79842d\" (UID: \"03bb2442-967f-465f-94bf-a209ae79842d\") "
Jul 24 18:00:01.903991 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:01.903751    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03bb2442-967f-465f-94bf-a209ae79842d-credential-socket\") pod \"03bb2442-967f-465f-94bf-a209ae79842d\" (UID: \"03bb2442-967f-465f-94bf-a209ae79842d\") "
Jul 24 18:00:01.903991 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:01.903771    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03bb2442-967f-465f-94bf-a209ae79842d-istio-podinfo\") pod \"03bb2442-967f-465f-94bf-a209ae79842d\" (UID: \"03bb2442-967f-465f-94bf-a209ae79842d\") "
Jul 24 18:00:01.903991 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:01.903809    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03bb2442-967f-465f-94bf-a209ae79842d-istio-token\") pod \"03bb2442-967f-465f-94bf-a209ae79842d\" (UID: \"03bb2442-967f-465f-94bf-a209ae79842d\") "
Jul 24 18:00:01.903991 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:01.903837    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03bb2442-967f-465f-94bf-a209ae79842d-workload-socket\") pod \"03bb2442-967f-465f-94bf-a209ae79842d\" (UID: \"03bb2442-967f-465f-94bf-a209ae79842d\") "
Jul 24 18:00:01.903991 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:01.903882    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03bb2442-967f-465f-94bf-a209ae79842d-workload-certs\") pod \"03bb2442-967f-465f-94bf-a209ae79842d\" (UID: \"03bb2442-967f-465f-94bf-a209ae79842d\") "
Jul 24 18:00:01.903991 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:01.903974    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03bb2442-967f-465f-94bf-a209ae79842d-istio-envoy\") pod \"03bb2442-967f-465f-94bf-a209ae79842d\" (UID: \"03bb2442-967f-465f-94bf-a209ae79842d\") "
Jul 24 18:00:01.904333 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:01.904063    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03bb2442-967f-465f-94bf-a209ae79842d-istiod-ca-cert\") pod \"03bb2442-967f-465f-94bf-a209ae79842d\" (UID: \"03bb2442-967f-465f-94bf-a209ae79842d\") "
Jul 24 18:00:01.904333 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:01.904103    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03bb2442-967f-465f-94bf-a209ae79842d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "03bb2442-967f-465f-94bf-a209ae79842d" (UID: "03bb2442-967f-465f-94bf-a209ae79842d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:00:01.904333 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:01.904208    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03bb2442-967f-465f-94bf-a209ae79842d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "03bb2442-967f-465f-94bf-a209ae79842d" (UID: "03bb2442-967f-465f-94bf-a209ae79842d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:00:01.904488 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:01.904415    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03bb2442-967f-465f-94bf-a209ae79842d-workload-certs\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 18:00:01.904488 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:01.904434    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03bb2442-967f-465f-94bf-a209ae79842d-credential-socket\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 18:00:01.904641 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:01.904519    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03bb2442-967f-465f-94bf-a209ae79842d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "03bb2442-967f-465f-94bf-a209ae79842d" (UID: "03bb2442-967f-465f-94bf-a209ae79842d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:00:01.904691 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:01.904652    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03bb2442-967f-465f-94bf-a209ae79842d-istio-data" (OuterVolumeSpecName: "istio-data") pod "03bb2442-967f-465f-94bf-a209ae79842d" (UID: "03bb2442-967f-465f-94bf-a209ae79842d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:00:01.904805 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:01.904763    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03bb2442-967f-465f-94bf-a209ae79842d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "03bb2442-967f-465f-94bf-a209ae79842d" (UID: "03bb2442-967f-465f-94bf-a209ae79842d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:00:01.906277 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:01.906252    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03bb2442-967f-465f-94bf-a209ae79842d-istio-token" (OuterVolumeSpecName: "istio-token") pod "03bb2442-967f-465f-94bf-a209ae79842d" (UID: "03bb2442-967f-465f-94bf-a209ae79842d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:00:01.906396 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:01.906253    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/03bb2442-967f-465f-94bf-a209ae79842d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "03bb2442-967f-465f-94bf-a209ae79842d" (UID: "03bb2442-967f-465f-94bf-a209ae79842d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 18:00:01.906664 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:01.906645    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03bb2442-967f-465f-94bf-a209ae79842d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "03bb2442-967f-465f-94bf-a209ae79842d" (UID: "03bb2442-967f-465f-94bf-a209ae79842d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:00:01.906664 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:01.906656    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03bb2442-967f-465f-94bf-a209ae79842d-kube-api-access-vvcn4" (OuterVolumeSpecName: "kube-api-access-vvcn4") pod "03bb2442-967f-465f-94bf-a209ae79842d" (UID: "03bb2442-967f-465f-94bf-a209ae79842d"). InnerVolumeSpecName "kube-api-access-vvcn4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:00:02.005306 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:02.005215    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03bb2442-967f-465f-94bf-a209ae79842d-istio-envoy\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 18:00:02.005306 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:02.005245    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03bb2442-967f-465f-94bf-a209ae79842d-istiod-ca-cert\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 18:00:02.005306 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:02.005255    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvcn4\" (UniqueName: \"kubernetes.io/projected/03bb2442-967f-465f-94bf-a209ae79842d-kube-api-access-vvcn4\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 18:00:02.005306 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:02.005264    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03bb2442-967f-465f-94bf-a209ae79842d-istio-data\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 18:00:02.005306 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:02.005272    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03bb2442-967f-465f-94bf-a209ae79842d-istio-podinfo\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 18:00:02.005306 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:02.005279    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03bb2442-967f-465f-94bf-a209ae79842d-istio-token\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 18:00:02.005306 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:02.005287    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03bb2442-967f-465f-94bf-a209ae79842d-workload-socket\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 18:00:02.173438 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:02.173402    2581 generic.go:356] "Generic (PLEG): container finished" podID="455d7a4c-cc1d-4e2a-adae-01ccd780a242" containerID="dc98283de5cc74c8b4465c85980beba1d011755a309c25eb8b2053604f2fc3e0" exitCode=0
Jul 24 18:00:02.173610 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:02.173498    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748600-hg857" event={"ID":"455d7a4c-cc1d-4e2a-adae-01ccd780a242","Type":"ContainerDied","Data":"dc98283de5cc74c8b4465c85980beba1d011755a309c25eb8b2053604f2fc3e0"}
Jul 24 18:00:02.174992 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:02.174970    2581 generic.go:356] "Generic (PLEG): container finished" podID="65abf56e-4d80-4a4f-9cf0-a83c118cc549" containerID="34abeabfe1c8400889434ca4d5fc54068ea453f9ae74659a26a62d643b19802e" exitCode=0
Jul 24 18:00:02.175126 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:02.175000    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5c7cfc-29748600-xsx8w" event={"ID":"65abf56e-4d80-4a4f-9cf0-a83c118cc549","Type":"ContainerDied","Data":"34abeabfe1c8400889434ca4d5fc54068ea453f9ae74659a26a62d643b19802e"}
Jul 24 18:00:02.176478 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:02.176455    2581 generic.go:356] "Generic (PLEG): container finished" podID="8a27dd7b-035d-4346-bc89-6792b98b64b7" containerID="626185ec7a231fc5db852917f0bf5215a772edca911c01a59b08f2794e5adeea" exitCode=0
Jul 24 18:00:02.176579 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:02.176532    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-bd0742-29748600-pffsq" event={"ID":"8a27dd7b-035d-4346-bc89-6792b98b64b7","Type":"ContainerDied","Data":"626185ec7a231fc5db852917f0bf5215a772edca911c01a59b08f2794e5adeea"}
Jul 24 18:00:02.178121 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:02.178102    2581 generic.go:356] "Generic (PLEG): container finished" podID="03bb2442-967f-465f-94bf-a209ae79842d" containerID="b65d96d11119fb351f53fe656d38961e9ae1daff6a24cde5c9436fd6866d63d1" exitCode=0
Jul 24 18:00:02.178205 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:02.178129    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr" event={"ID":"03bb2442-967f-465f-94bf-a209ae79842d","Type":"ContainerDied","Data":"b65d96d11119fb351f53fe656d38961e9ae1daff6a24cde5c9436fd6866d63d1"}
Jul 24 18:00:02.178205 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:02.178148    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"
Jul 24 18:00:02.178205 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:02.178172    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr" event={"ID":"03bb2442-967f-465f-94bf-a209ae79842d","Type":"ContainerDied","Data":"480b9ddbb6d616d111a4ad31f8e4d88ab8f9ed77c619e7ddfc1fe727e232c222"}
Jul 24 18:00:02.178205 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:02.178197    2581 scope.go:117] "RemoveContainer" containerID="b65d96d11119fb351f53fe656d38961e9ae1daff6a24cde5c9436fd6866d63d1"
Jul 24 18:00:02.179688 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:02.179664    2581 generic.go:356] "Generic (PLEG): container finished" podID="24731052-120c-4686-8aed-ebf14acac5b3" containerID="b40c51a20ad6d950255cab69d1d8f6ebb4073fbdd34acfa92db0ff3322e2c66f" exitCode=0
Jul 24 18:00:02.179776 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:02.179697    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bb94b3-29748600-r9989" event={"ID":"24731052-120c-4686-8aed-ebf14acac5b3","Type":"ContainerDied","Data":"b40c51a20ad6d950255cab69d1d8f6ebb4073fbdd34acfa92db0ff3322e2c66f"}
Jul 24 18:00:02.187663 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:02.187643    2581 scope.go:117] "RemoveContainer" containerID="855b9d431d980ce4bf2293189645f47feaebc290654dc379aafe77f8b01da87a"
Jul 24 18:00:02.197203 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:02.197183    2581 scope.go:117] "RemoveContainer" containerID="b65d96d11119fb351f53fe656d38961e9ae1daff6a24cde5c9436fd6866d63d1"
Jul 24 18:00:02.197496 ip-10-0-139-46 kubenswrapper[2581]: E0724 18:00:02.197474    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b65d96d11119fb351f53fe656d38961e9ae1daff6a24cde5c9436fd6866d63d1\": container with ID starting with b65d96d11119fb351f53fe656d38961e9ae1daff6a24cde5c9436fd6866d63d1 not found: ID does not exist" containerID="b65d96d11119fb351f53fe656d38961e9ae1daff6a24cde5c9436fd6866d63d1"
Jul 24 18:00:02.197540 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:02.197508    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b65d96d11119fb351f53fe656d38961e9ae1daff6a24cde5c9436fd6866d63d1"} err="failed to get container status \"b65d96d11119fb351f53fe656d38961e9ae1daff6a24cde5c9436fd6866d63d1\": rpc error: code = NotFound desc = could not find container \"b65d96d11119fb351f53fe656d38961e9ae1daff6a24cde5c9436fd6866d63d1\": container with ID starting with b65d96d11119fb351f53fe656d38961e9ae1daff6a24cde5c9436fd6866d63d1 not found: ID does not exist"
Jul 24 18:00:02.197540 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:02.197525    2581 scope.go:117] "RemoveContainer" containerID="855b9d431d980ce4bf2293189645f47feaebc290654dc379aafe77f8b01da87a"
Jul 24 18:00:02.197753 ip-10-0-139-46 kubenswrapper[2581]: E0724 18:00:02.197741    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"855b9d431d980ce4bf2293189645f47feaebc290654dc379aafe77f8b01da87a\": container with ID starting with 855b9d431d980ce4bf2293189645f47feaebc290654dc379aafe77f8b01da87a not found: ID does not exist" containerID="855b9d431d980ce4bf2293189645f47feaebc290654dc379aafe77f8b01da87a"
Jul 24 18:00:02.197790 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:02.197756    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"855b9d431d980ce4bf2293189645f47feaebc290654dc379aafe77f8b01da87a"} err="failed to get container status \"855b9d431d980ce4bf2293189645f47feaebc290654dc379aafe77f8b01da87a\": rpc error: code = NotFound desc = could not find container \"855b9d431d980ce4bf2293189645f47feaebc290654dc379aafe77f8b01da87a\": container with ID starting with 855b9d431d980ce4bf2293189645f47feaebc290654dc379aafe77f8b01da87a not found: ID does not exist"
Jul 24 18:00:02.226954 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:02.226922    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"]
Jul 24 18:00:02.232685 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:02.232657    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-8a6414-openshift-default-65d4b6c74-tvnsr"]
Jul 24 18:00:03.346324 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:03.346296    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-bd0742-29748600-pffsq"
Jul 24 18:00:03.386494 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:03.386472    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03bb2442-967f-465f-94bf-a209ae79842d" path="/var/lib/kubelet/pods/03bb2442-967f-465f-94bf-a209ae79842d/volumes"
Jul 24 18:00:03.397307 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:03.397288    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5c7cfc-29748600-xsx8w"
Jul 24 18:00:03.400609 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:03.400588    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bb94b3-29748600-r9989"
Jul 24 18:00:03.404081 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:03.404063    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748600-hg857"
Jul 24 18:00:04.195510 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:04.195483    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5c7cfc-29748600-xsx8w"
Jul 24 18:00:04.195735 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:04.195475    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5c7cfc-29748600-xsx8w" event={"ID":"65abf56e-4d80-4a4f-9cf0-a83c118cc549","Type":"ContainerDied","Data":"2034dfdaa1a40c06274ca3e4b55e87971a9dbbe21fb080f95078a0a175ab943d"}
Jul 24 18:00:04.195735 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:04.195650    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2034dfdaa1a40c06274ca3e4b55e87971a9dbbe21fb080f95078a0a175ab943d"
Jul 24 18:00:04.196781 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:04.196756    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-bd0742-29748600-pffsq"
Jul 24 18:00:04.196879 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:04.196791    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-bd0742-29748600-pffsq" event={"ID":"8a27dd7b-035d-4346-bc89-6792b98b64b7","Type":"ContainerDied","Data":"b5fe04acc273afe2b303109746e1c6bec57ac4899a75d294be3e21fa36a4ed7d"}
Jul 24 18:00:04.196879 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:04.196831    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b5fe04acc273afe2b303109746e1c6bec57ac4899a75d294be3e21fa36a4ed7d"
Jul 24 18:00:04.198237 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:04.198208    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bb94b3-29748600-r9989" event={"ID":"24731052-120c-4686-8aed-ebf14acac5b3","Type":"ContainerDied","Data":"451a38d4721f0e9c9de22d75ac384dd5e1822bc6b93054ff091d3c5305fb8a44"}
Jul 24 18:00:04.198355 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:04.198249    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="451a38d4721f0e9c9de22d75ac384dd5e1822bc6b93054ff091d3c5305fb8a44"
Jul 24 18:00:04.198355 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:04.198210    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bb94b3-29748600-r9989"
Jul 24 18:00:04.199775 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:04.199752    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748600-hg857" event={"ID":"455d7a4c-cc1d-4e2a-adae-01ccd780a242","Type":"ContainerDied","Data":"5fbeb39b326c9e8b0cd6e13fb5bd6964f95a2a49866dfd2dabce21358ef25fb7"}
Jul 24 18:00:04.199775 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:04.199767    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748600-hg857"
Jul 24 18:00:04.199905 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:04.199797    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5fbeb39b326c9e8b0cd6e13fb5bd6964f95a2a49866dfd2dabce21358ef25fb7"
Jul 24 18:00:07.429772 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:07.429734    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-bd0742-29748600-pffsq"]
Jul 24 18:00:07.431765 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:07.431744    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-bd0742-29748600-pffsq"]
Jul 24 18:00:07.800392 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:07.800302    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 18:00:07.870057 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:07.869997    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 18:00:09.382996 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:09.382961    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a27dd7b-035d-4346-bc89-6792b98b64b7" path="/var/lib/kubelet/pods/8a27dd7b-035d-4346-bc89-6792b98b64b7/volumes"
Jul 24 18:00:10.564628 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:10.564592    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 18:00:12.025335 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:12.025293    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bd0742-sgspp"]
Jul 24 18:00:12.028189 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:12.028153    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bd0742-sgspp"]
Jul 24 18:00:12.034088 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:12.034066    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 18:00:12.413966 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:12.413933    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 18:00:13.383103 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:13.383067    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cedea9ce-1da9-4643-b398-8c683518763c" path="/var/lib/kubelet/pods/cedea9ce-1da9-4643-b398-8c683518763c/volumes"
Jul 24 18:00:18.520151 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:18.520115    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 18:00:18.767892 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:18.767861    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"]
Jul 24 18:00:18.768113 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:18.768090    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b" podUID="c43c7ad0-b0de-4d08-8ba1-c228038a04f1" containerName="istio-proxy" containerID="cri-o://a144bea9c43444b6e3a046d39a46a0a2828560c81e846940ec798b49a51f80c1" gracePeriod=30
Jul 24 18:00:19.164470 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:19.164439    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bb94b3-6g6rf"]
Jul 24 18:00:19.164829 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:19.164817    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03bb2442-967f-465f-94bf-a209ae79842d" containerName="istio-proxy"
Jul 24 18:00:19.164878 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:19.164831    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="03bb2442-967f-465f-94bf-a209ae79842d" containerName="istio-proxy"
Jul 24 18:00:19.164878 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:19.164848    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a27dd7b-035d-4346-bc89-6792b98b64b7" containerName="cleanup"
Jul 24 18:00:19.164878 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:19.164853    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a27dd7b-035d-4346-bc89-6792b98b64b7" containerName="cleanup"
Jul 24 18:00:19.164878 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:19.164860    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="455d7a4c-cc1d-4e2a-adae-01ccd780a242" containerName="cleanup"
Jul 24 18:00:19.164878 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:19.164865    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="455d7a4c-cc1d-4e2a-adae-01ccd780a242" containerName="cleanup"
Jul 24 18:00:19.164878 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:19.164875    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65abf56e-4d80-4a4f-9cf0-a83c118cc549" containerName="cleanup"
Jul 24 18:00:19.165087 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:19.164882    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="65abf56e-4d80-4a4f-9cf0-a83c118cc549" containerName="cleanup"
Jul 24 18:00:19.165087 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:19.164890    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24731052-120c-4686-8aed-ebf14acac5b3" containerName="cleanup"
Jul 24 18:00:19.165087 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:19.164895    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="24731052-120c-4686-8aed-ebf14acac5b3" containerName="cleanup"
Jul 24 18:00:19.165087 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:19.164956    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="65abf56e-4d80-4a4f-9cf0-a83c118cc549" containerName="cleanup"
Jul 24 18:00:19.165087 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:19.164963    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="03bb2442-967f-465f-94bf-a209ae79842d" containerName="istio-proxy"
Jul 24 18:00:19.165087 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:19.164969    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="24731052-120c-4686-8aed-ebf14acac5b3" containerName="cleanup"
Jul 24 18:00:19.165087 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:19.164976    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a27dd7b-035d-4346-bc89-6792b98b64b7" containerName="cleanup"
Jul 24 18:00:19.165087 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:19.164982    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="455d7a4c-cc1d-4e2a-adae-01ccd780a242" containerName="cleanup"
Jul 24 18:00:19.168111 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:19.168094    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bb94b3-6g6rf"
Jul 24 18:00:19.170876 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:19.170854    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 18:00:19.171032 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:19.171013    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d6flx\""
Jul 24 18:00:19.177573 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:19.177550    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bb94b3-6g6rf"]
Jul 24 18:00:19.269340 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:19.269302    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70e37a20-67d9-4ce7-a24a-fa206076563c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-bb94b3-6g6rf\" (UID: \"70e37a20-67d9-4ce7-a24a-fa206076563c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bb94b3-6g6rf"
Jul 24 18:00:19.370521 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:19.370485    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70e37a20-67d9-4ce7-a24a-fa206076563c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-bb94b3-6g6rf\" (UID: \"70e37a20-67d9-4ce7-a24a-fa206076563c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bb94b3-6g6rf"
Jul 24 18:00:19.371135 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:19.371114    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70e37a20-67d9-4ce7-a24a-fa206076563c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-bb94b3-6g6rf\" (UID: \"70e37a20-67d9-4ce7-a24a-fa206076563c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bb94b3-6g6rf"
Jul 24 18:00:19.479020 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:19.478938    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bb94b3-6g6rf"
Jul 24 18:00:19.820573 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:19.820547    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bb94b3-6g6rf"]
Jul 24 18:00:19.822329 ip-10-0-139-46 kubenswrapper[2581]: W0724 18:00:19.822302    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70e37a20_67d9_4ce7_a24a_fa206076563c.slice/crio-b8bb1d3a4c4bf381f91bb1f04b344c9ff8c4ba6b4efa66689f33d5a88e6df7af WatchSource:0}: Error finding container b8bb1d3a4c4bf381f91bb1f04b344c9ff8c4ba6b4efa66689f33d5a88e6df7af: Status 404 returned error can't find the container with id b8bb1d3a4c4bf381f91bb1f04b344c9ff8c4ba6b4efa66689f33d5a88e6df7af
Jul 24 18:00:20.097422 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:20.097325    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 18:00:20.285659 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:20.285623    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bb94b3-6g6rf" event={"ID":"70e37a20-67d9-4ce7-a24a-fa206076563c","Type":"ContainerStarted","Data":"378a4d06bcdb258cc9937990e6ff3a08f0e16c4ef8577ed3a7ea2f79f71d8e97"}
Jul 24 18:00:20.285837 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:20.285683    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bb94b3-6g6rf" event={"ID":"70e37a20-67d9-4ce7-a24a-fa206076563c","Type":"ContainerStarted","Data":"b8bb1d3a4c4bf381f91bb1f04b344c9ff8c4ba6b4efa66689f33d5a88e6df7af"}
Jul 24 18:00:20.311288 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:20.311242    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bb94b3-6g6rf" podStartSLOduration=1.31122757 podStartE2EDuration="1.31122757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:00:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:00:20.308722495 +0000 UTC m=+2987.510771785" watchObservedRunningTime="2026-07-24 18:00:20.31122757 +0000 UTC m=+2987.513276890"
Jul 24 18:00:21.292328 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:21.292293    2581 generic.go:356] "Generic (PLEG): container finished" podID="70e37a20-67d9-4ce7-a24a-fa206076563c" containerID="378a4d06bcdb258cc9937990e6ff3a08f0e16c4ef8577ed3a7ea2f79f71d8e97" exitCode=0
Jul 24 18:00:21.292741 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:21.292375    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bb94b3-6g6rf" event={"ID":"70e37a20-67d9-4ce7-a24a-fa206076563c","Type":"ContainerDied","Data":"378a4d06bcdb258cc9937990e6ff3a08f0e16c4ef8577ed3a7ea2f79f71d8e97"}
Jul 24 18:00:22.465802 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:22.465774    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bb94b3-6g6rf"
Jul 24 18:00:22.601764 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:22.601671    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70e37a20-67d9-4ce7-a24a-fa206076563c-maas-api-service-ca\") pod \"70e37a20-67d9-4ce7-a24a-fa206076563c\" (UID: \"70e37a20-67d9-4ce7-a24a-fa206076563c\") "
Jul 24 18:00:22.602132 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:22.602102    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70e37a20-67d9-4ce7-a24a-fa206076563c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "70e37a20-67d9-4ce7-a24a-fa206076563c" (UID: "70e37a20-67d9-4ce7-a24a-fa206076563c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:00:22.702768 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:22.702735    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70e37a20-67d9-4ce7-a24a-fa206076563c-maas-api-service-ca\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 18:00:23.304524 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:23.304489    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bb94b3-6g6rf"
Jul 24 18:00:23.304704 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:23.304495    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bb94b3-6g6rf" event={"ID":"70e37a20-67d9-4ce7-a24a-fa206076563c","Type":"ContainerDied","Data":"b8bb1d3a4c4bf381f91bb1f04b344c9ff8c4ba6b4efa66689f33d5a88e6df7af"}
Jul 24 18:00:23.304704 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:23.304644    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b8bb1d3a4c4bf381f91bb1f04b344c9ff8c4ba6b4efa66689f33d5a88e6df7af"
Jul 24 18:00:24.026221 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.026198    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 18:00:24.115794 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.115712    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-istio-envoy\") pod \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\" (UID: \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\") "
Jul 24 18:00:24.115794 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.115751    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-workload-certs\") pod \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\" (UID: \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\") "
Jul 24 18:00:24.115794 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.115773    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-credential-socket\") pod \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\" (UID: \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\") "
Jul 24 18:00:24.116119 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.115801    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-istiod-ca-cert\") pod \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\" (UID: \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\") "
Jul 24 18:00:24.116119 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.115858    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-istio-token\") pod \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\" (UID: \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\") "
Jul 24 18:00:24.116119 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.115884    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-istio-podinfo\") pod \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\" (UID: \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\") "
Jul 24 18:00:24.116119 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.115951    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mhzsj\" (UniqueName: \"kubernetes.io/projected/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-kube-api-access-mhzsj\") pod \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\" (UID: \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\") "
Jul 24 18:00:24.116119 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.115977    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-istio-data\") pod \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\" (UID: \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\") "
Jul 24 18:00:24.116119 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.116009    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-workload-socket\") pod \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\" (UID: \"c43c7ad0-b0de-4d08-8ba1-c228038a04f1\") "
Jul 24 18:00:24.116402 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.116137    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c43c7ad0-b0de-4d08-8ba1-c228038a04f1" (UID: "c43c7ad0-b0de-4d08-8ba1-c228038a04f1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:00:24.116402 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.116175    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c43c7ad0-b0de-4d08-8ba1-c228038a04f1" (UID: "c43c7ad0-b0de-4d08-8ba1-c228038a04f1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:00:24.116402 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.116237    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c43c7ad0-b0de-4d08-8ba1-c228038a04f1" (UID: "c43c7ad0-b0de-4d08-8ba1-c228038a04f1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:00:24.116575 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.116437    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c43c7ad0-b0de-4d08-8ba1-c228038a04f1" (UID: "c43c7ad0-b0de-4d08-8ba1-c228038a04f1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:00:24.116575 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.116444    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-workload-certs\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 18:00:24.116575 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.116465    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-credential-socket\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 18:00:24.116575 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.116481    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-istiod-ca-cert\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 18:00:24.117454 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.117425    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-istio-data" (OuterVolumeSpecName: "istio-data") pod "c43c7ad0-b0de-4d08-8ba1-c228038a04f1" (UID: "c43c7ad0-b0de-4d08-8ba1-c228038a04f1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:00:24.118293 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.118264    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c43c7ad0-b0de-4d08-8ba1-c228038a04f1" (UID: "c43c7ad0-b0de-4d08-8ba1-c228038a04f1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:00:24.118395 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.118358    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-istio-token" (OuterVolumeSpecName: "istio-token") pod "c43c7ad0-b0de-4d08-8ba1-c228038a04f1" (UID: "c43c7ad0-b0de-4d08-8ba1-c228038a04f1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:00:24.118441 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.118402    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-kube-api-access-mhzsj" (OuterVolumeSpecName: "kube-api-access-mhzsj") pod "c43c7ad0-b0de-4d08-8ba1-c228038a04f1" (UID: "c43c7ad0-b0de-4d08-8ba1-c228038a04f1"). InnerVolumeSpecName "kube-api-access-mhzsj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:00:24.118441 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.118399    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c43c7ad0-b0de-4d08-8ba1-c228038a04f1" (UID: "c43c7ad0-b0de-4d08-8ba1-c228038a04f1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 18:00:24.218061 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.218007    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-istio-envoy\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 18:00:24.218061 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.218065    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-istio-token\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 18:00:24.218269 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.218078    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-istio-podinfo\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 18:00:24.218269 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.218090    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhzsj\" (UniqueName: \"kubernetes.io/projected/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-kube-api-access-mhzsj\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 18:00:24.218269 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.218103    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-istio-data\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 18:00:24.218269 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.218117    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c43c7ad0-b0de-4d08-8ba1-c228038a04f1-workload-socket\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 18:00:24.311153 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.311117    2581 generic.go:356] "Generic (PLEG): container finished" podID="c43c7ad0-b0de-4d08-8ba1-c228038a04f1" containerID="a144bea9c43444b6e3a046d39a46a0a2828560c81e846940ec798b49a51f80c1" exitCode=0
Jul 24 18:00:24.311308 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.311166    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b" event={"ID":"c43c7ad0-b0de-4d08-8ba1-c228038a04f1","Type":"ContainerDied","Data":"a144bea9c43444b6e3a046d39a46a0a2828560c81e846940ec798b49a51f80c1"}
Jul 24 18:00:24.311308 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.311183    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"
Jul 24 18:00:24.311308 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.311208    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b" event={"ID":"c43c7ad0-b0de-4d08-8ba1-c228038a04f1","Type":"ContainerDied","Data":"6ffd8287f3a1c0ee462f9986eaa41e909456b09d9b59ab21dca50ff061a743d4"}
Jul 24 18:00:24.311308 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.311237    2581 scope.go:117] "RemoveContainer" containerID="a144bea9c43444b6e3a046d39a46a0a2828560c81e846940ec798b49a51f80c1"
Jul 24 18:00:24.321542 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.321523    2581 scope.go:117] "RemoveContainer" containerID="d96d5ff79b57f7df9cfd8a83abc859a3033ca325455fb6c1a5e6a53054c1f769"
Jul 24 18:00:24.330386 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.330365    2581 scope.go:117] "RemoveContainer" containerID="a144bea9c43444b6e3a046d39a46a0a2828560c81e846940ec798b49a51f80c1"
Jul 24 18:00:24.330691 ip-10-0-139-46 kubenswrapper[2581]: E0724 18:00:24.330672    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a144bea9c43444b6e3a046d39a46a0a2828560c81e846940ec798b49a51f80c1\": container with ID starting with a144bea9c43444b6e3a046d39a46a0a2828560c81e846940ec798b49a51f80c1 not found: ID does not exist" containerID="a144bea9c43444b6e3a046d39a46a0a2828560c81e846940ec798b49a51f80c1"
Jul 24 18:00:24.330736 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.330702    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a144bea9c43444b6e3a046d39a46a0a2828560c81e846940ec798b49a51f80c1"} err="failed to get container status \"a144bea9c43444b6e3a046d39a46a0a2828560c81e846940ec798b49a51f80c1\": rpc error: code = NotFound desc = could not find container \"a144bea9c43444b6e3a046d39a46a0a2828560c81e846940ec798b49a51f80c1\": container with ID starting with a144bea9c43444b6e3a046d39a46a0a2828560c81e846940ec798b49a51f80c1 not found: ID does not exist"
Jul 24 18:00:24.330736 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.330719    2581 scope.go:117] "RemoveContainer" containerID="d96d5ff79b57f7df9cfd8a83abc859a3033ca325455fb6c1a5e6a53054c1f769"
Jul 24 18:00:24.330980 ip-10-0-139-46 kubenswrapper[2581]: E0724 18:00:24.330961    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d96d5ff79b57f7df9cfd8a83abc859a3033ca325455fb6c1a5e6a53054c1f769\": container with ID starting with d96d5ff79b57f7df9cfd8a83abc859a3033ca325455fb6c1a5e6a53054c1f769 not found: ID does not exist" containerID="d96d5ff79b57f7df9cfd8a83abc859a3033ca325455fb6c1a5e6a53054c1f769"
Jul 24 18:00:24.331023 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.330988    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d96d5ff79b57f7df9cfd8a83abc859a3033ca325455fb6c1a5e6a53054c1f769"} err="failed to get container status \"d96d5ff79b57f7df9cfd8a83abc859a3033ca325455fb6c1a5e6a53054c1f769\": rpc error: code = NotFound desc = could not find container \"d96d5ff79b57f7df9cfd8a83abc859a3033ca325455fb6c1a5e6a53054c1f769\": container with ID starting with d96d5ff79b57f7df9cfd8a83abc859a3033ca325455fb6c1a5e6a53054c1f769 not found: ID does not exist"
Jul 24 18:00:24.334530 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.334507    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"]
Jul 24 18:00:24.338831 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:24.338810    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-bd0742-openshift-default-6f69cc7bf8-5qv9b"]
Jul 24 18:00:25.384096 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:25.384031    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c43c7ad0-b0de-4d08-8ba1-c228038a04f1" path="/var/lib/kubelet/pods/c43c7ad0-b0de-4d08-8ba1-c228038a04f1/volumes"
Jul 24 18:00:29.763025 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:29.762978    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 18:00:29.764443 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:29.764418    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bb94b3-29748600-r9989"]
Jul 24 18:00:29.766225 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:29.766206    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bb94b3-29748600-r9989"]
Jul 24 18:00:29.861689 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:29.861657    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 18:00:29.958943 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:29.958906    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 18:00:31.383659 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:31.383624    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24731052-120c-4686-8aed-ebf14acac5b3" path="/var/lib/kubelet/pods/24731052-120c-4686-8aed-ebf14acac5b3/volumes"
Jul 24 18:00:32.754740 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:32.754706    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 18:00:34.228249 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:34.228115    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bb94b3-6g6rf"]
Jul 24 18:00:34.230131 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:34.230101    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bb94b3-6g6rf"]
Jul 24 18:00:34.230488 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:34.230466    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 18:00:34.616104 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:34.615981    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 18:00:35.383721 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:35.383690    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70e37a20-67d9-4ce7-a24a-fa206076563c" path="/var/lib/kubelet/pods/70e37a20-67d9-4ce7-a24a-fa206076563c/volumes"
Jul 24 18:00:40.948343 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:40.948303    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 18:00:41.026826 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:41.026770    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 18:00:41.221528 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:41.221440    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 18:00:41.441691 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:41.441654    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 18:00:41.578436 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:41.578356    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5c7cfc-7f7jc"]
Jul 24 18:00:41.578782 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:41.578768    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c43c7ad0-b0de-4d08-8ba1-c228038a04f1" containerName="istio-proxy"
Jul 24 18:00:41.578828 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:41.578784    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c43c7ad0-b0de-4d08-8ba1-c228038a04f1" containerName="istio-proxy"
Jul 24 18:00:41.578828 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:41.578803    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70e37a20-67d9-4ce7-a24a-fa206076563c" containerName="revoke-keys"
Jul 24 18:00:41.578828 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:41.578808    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="70e37a20-67d9-4ce7-a24a-fa206076563c" containerName="revoke-keys"
Jul 24 18:00:41.578828 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:41.578817    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03bb2442-967f-465f-94bf-a209ae79842d" containerName="istio-proxy"
Jul 24 18:00:41.578828 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:41.578822    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="03bb2442-967f-465f-94bf-a209ae79842d" containerName="istio-proxy"
Jul 24 18:00:41.578987 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:41.578887    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c43c7ad0-b0de-4d08-8ba1-c228038a04f1" containerName="istio-proxy"
Jul 24 18:00:41.578987 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:41.578895    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="03bb2442-967f-465f-94bf-a209ae79842d" containerName="istio-proxy"
Jul 24 18:00:41.578987 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:41.578901    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c43c7ad0-b0de-4d08-8ba1-c228038a04f1" containerName="istio-proxy"
Jul 24 18:00:41.578987 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:41.578906    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="70e37a20-67d9-4ce7-a24a-fa206076563c" containerName="revoke-keys"
Jul 24 18:00:41.581893 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:41.581878    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5c7cfc-7f7jc"
Jul 24 18:00:41.582478 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:41.582448    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b80edf0-4441-44c0-81ce-ef7b71f1643c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-5c7cfc-7f7jc\" (UID: \"4b80edf0-4441-44c0-81ce-ef7b71f1643c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5c7cfc-7f7jc"
Jul 24 18:00:41.586218 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:41.586199    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d6flx\""
Jul 24 18:00:41.586479 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:41.586466    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 18:00:41.591275 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:41.591253    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5c7cfc-7f7jc"]
Jul 24 18:00:41.683155 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:41.683122    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b80edf0-4441-44c0-81ce-ef7b71f1643c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-5c7cfc-7f7jc\" (UID: \"4b80edf0-4441-44c0-81ce-ef7b71f1643c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5c7cfc-7f7jc"
Jul 24 18:00:41.683760 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:41.683742    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b80edf0-4441-44c0-81ce-ef7b71f1643c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-5c7cfc-7f7jc\" (UID: \"4b80edf0-4441-44c0-81ce-ef7b71f1643c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5c7cfc-7f7jc"
Jul 24 18:00:41.893372 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:41.893348    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5c7cfc-7f7jc"
Jul 24 18:00:42.029895 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:42.029869    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5c7cfc-7f7jc"]
Jul 24 18:00:42.031751 ip-10-0-139-46 kubenswrapper[2581]: W0724 18:00:42.031724    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b80edf0_4441_44c0_81ce_ef7b71f1643c.slice/crio-d9d6545f8b7941548325bcf2ab92d9a512d0339778f1cb169cf595a278e41e7c WatchSource:0}: Error finding container d9d6545f8b7941548325bcf2ab92d9a512d0339778f1cb169cf595a278e41e7c: Status 404 returned error can't find the container with id d9d6545f8b7941548325bcf2ab92d9a512d0339778f1cb169cf595a278e41e7c
Jul 24 18:00:42.409080 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:42.409026    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5c7cfc-7f7jc" event={"ID":"4b80edf0-4441-44c0-81ce-ef7b71f1643c","Type":"ContainerStarted","Data":"423e5ef5c5dbd006abaea6afb5cba5dca243463957330fc6e5b17c50568f9675"}
Jul 24 18:00:42.409216 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:42.409106    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5c7cfc-7f7jc" event={"ID":"4b80edf0-4441-44c0-81ce-ef7b71f1643c","Type":"ContainerStarted","Data":"d9d6545f8b7941548325bcf2ab92d9a512d0339778f1cb169cf595a278e41e7c"}
Jul 24 18:00:42.428815 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:42.428764    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5c7cfc-7f7jc" podStartSLOduration=1.42874841 podStartE2EDuration="1.42874841s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:00:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:00:42.427715825 +0000 UTC m=+3009.629765101" watchObservedRunningTime="2026-07-24 18:00:42.42874841 +0000 UTC m=+3009.630797684"
Jul 24 18:00:42.812766 ip-10-0-139-46 kubenswrapper[2581]: E0724 18:00:42.812734    2581 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b80edf0_4441_44c0_81ce_ef7b71f1643c.slice/crio-conmon-423e5ef5c5dbd006abaea6afb5cba5dca243463957330fc6e5b17c50568f9675.scope\": RecentStats: unable to find data in memory cache]"
Jul 24 18:00:43.415231 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:43.415197    2581 generic.go:356] "Generic (PLEG): container finished" podID="4b80edf0-4441-44c0-81ce-ef7b71f1643c" containerID="423e5ef5c5dbd006abaea6afb5cba5dca243463957330fc6e5b17c50568f9675" exitCode=0
Jul 24 18:00:43.415637 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:43.415285    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5c7cfc-7f7jc" event={"ID":"4b80edf0-4441-44c0-81ce-ef7b71f1643c","Type":"ContainerDied","Data":"423e5ef5c5dbd006abaea6afb5cba5dca243463957330fc6e5b17c50568f9675"}
Jul 24 18:00:44.556912 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:44.556887    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5c7cfc-7f7jc"
Jul 24 18:00:44.607784 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:44.607750    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b80edf0-4441-44c0-81ce-ef7b71f1643c-maas-api-service-ca\") pod \"4b80edf0-4441-44c0-81ce-ef7b71f1643c\" (UID: \"4b80edf0-4441-44c0-81ce-ef7b71f1643c\") "
Jul 24 18:00:44.608172 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:44.608147    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b80edf0-4441-44c0-81ce-ef7b71f1643c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4b80edf0-4441-44c0-81ce-ef7b71f1643c" (UID: "4b80edf0-4441-44c0-81ce-ef7b71f1643c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:00:44.708995 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:44.708915    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b80edf0-4441-44c0-81ce-ef7b71f1643c-maas-api-service-ca\") on node \"ip-10-0-139-46.ec2.internal\" DevicePath \"\""
Jul 24 18:00:45.426572 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:45.426542    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5c7cfc-7f7jc"
Jul 24 18:00:45.426745 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:45.426585    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5c7cfc-7f7jc" event={"ID":"4b80edf0-4441-44c0-81ce-ef7b71f1643c","Type":"ContainerDied","Data":"d9d6545f8b7941548325bcf2ab92d9a512d0339778f1cb169cf595a278e41e7c"}
Jul 24 18:00:45.426745 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:45.426646    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d9d6545f8b7941548325bcf2ab92d9a512d0339778f1cb169cf595a278e41e7c"
Jul 24 18:00:52.007154 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:52.007114    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 18:00:52.007569 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:52.007192    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5c7cfc-29748600-xsx8w"]
Jul 24 18:00:52.009021 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:52.009000    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5c7cfc-29748600-xsx8w"]
Jul 24 18:00:52.082315 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:52.082287    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 18:00:52.166221 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:52.166167    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 18:00:53.384324 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:53.384294    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65abf56e-4d80-4a4f-9cf0-a83c118cc549" path="/var/lib/kubelet/pods/65abf56e-4d80-4a4f-9cf0-a83c118cc549/volumes"
Jul 24 18:00:54.960858 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:54.960824    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5dbvm"]
Jul 24 18:00:56.643060 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:56.643003    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5c7cfc-7f7jc"]
Jul 24 18:00:56.647137 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:56.647109    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5c7cfc-7f7jc"]
Jul 24 18:00:56.648566 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:56.648515    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 18:00:57.026683 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:57.026592    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 18:00:57.384192 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:00:57.384161    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b80edf0-4441-44c0-81ce-ef7b71f1643c" path="/var/lib/kubelet/pods/4b80edf0-4441-44c0-81ce-ef7b71f1643c/volumes"
Jul 24 18:01:03.305726 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:03.305693    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 18:01:03.382437 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:03.382402    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 18:01:03.592232 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:03.592146    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 18:01:03.817235 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:03.817194    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 18:01:10.341174 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:10.341146    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bcd74455c-r4pqr_ad445777-f665-482c-9f35-2eb21a836bb2/copy-manifests/0.log"
Jul 24 18:01:10.347943 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:10.347921    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bcd74455c-r4pqr_ad445777-f665-482c-9f35-2eb21a836bb2/manager/0.log"
Jul 24 18:01:11.159532 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:11.159501    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29748600-hg857_455d7a4c-cc1d-4e2a-adae-01ccd780a242/cleanup/0.log"
Jul 24 18:01:12.903791 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:12.903762    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-cswnr_68a9959d-3d81-4dbc-9bd9-b447935635bf/kuadrant-console-plugin/0.log"
Jul 24 18:01:13.260751 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:13.260666    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-5dbvm_10336757-378d-4415-ad40-56386dd911bb/limitador/0.log"
Jul 24 18:01:13.958774 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:13.958741    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-84b496c558-987vc_fe8593b3-ba31-4b15-9485-281e108a027f/kube-auth-proxy/0.log"
Jul 24 18:01:14.892379 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:14.892350    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-bcmtd_57141de1-829e-4358-9620-6427e97dcab1/main/0.log"
Jul 24 18:01:15.230227 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:15.230118    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk5kv_73d0f656-1ccb-4fd7-9639-6a515a9284e9/main/0.log"
Jul 24 18:01:15.459811 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:15.459780    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-g7bgv_21472c42-aa48-4fda-8e8d-b30e8ec5def9/main/0.log"
Jul 24 18:01:21.061654 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:21.061575    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-g7vgp/must-gather-59752"]
Jul 24 18:01:21.062101 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:21.062067    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b80edf0-4441-44c0-81ce-ef7b71f1643c" containerName="revoke-keys"
Jul 24 18:01:21.062101 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:21.062086    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b80edf0-4441-44c0-81ce-ef7b71f1643c" containerName="revoke-keys"
Jul 24 18:01:21.062326 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:21.062125    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c43c7ad0-b0de-4d08-8ba1-c228038a04f1" containerName="istio-proxy"
Jul 24 18:01:21.062326 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:21.062135    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c43c7ad0-b0de-4d08-8ba1-c228038a04f1" containerName="istio-proxy"
Jul 24 18:01:21.062326 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:21.062225    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b80edf0-4441-44c0-81ce-ef7b71f1643c" containerName="revoke-keys"
Jul 24 18:01:21.065760 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:21.065741    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g7vgp/must-gather-59752"
Jul 24 18:01:21.068704 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:21.068684    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-g7vgp\"/\"default-dockercfg-ml5bw\""
Jul 24 18:01:21.069963 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:21.069945    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-g7vgp\"/\"kube-root-ca.crt\""
Jul 24 18:01:21.070077 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:21.069953    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-g7vgp\"/\"openshift-service-ca.crt\""
Jul 24 18:01:21.077854 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:21.077829    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g7vgp/must-gather-59752"]
Jul 24 18:01:21.160494 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:21.160452    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ded439dd-0edc-488a-a657-e4a3e6dc2c02-must-gather-output\") pod \"must-gather-59752\" (UID: \"ded439dd-0edc-488a-a657-e4a3e6dc2c02\") " pod="openshift-must-gather-g7vgp/must-gather-59752"
Jul 24 18:01:21.160683 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:21.160582    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dt5h\" (UniqueName: \"kubernetes.io/projected/ded439dd-0edc-488a-a657-e4a3e6dc2c02-kube-api-access-6dt5h\") pod \"must-gather-59752\" (UID: \"ded439dd-0edc-488a-a657-e4a3e6dc2c02\") " pod="openshift-must-gather-g7vgp/must-gather-59752"
Jul 24 18:01:21.261557 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:21.261520    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dt5h\" (UniqueName: \"kubernetes.io/projected/ded439dd-0edc-488a-a657-e4a3e6dc2c02-kube-api-access-6dt5h\") pod \"must-gather-59752\" (UID: \"ded439dd-0edc-488a-a657-e4a3e6dc2c02\") " pod="openshift-must-gather-g7vgp/must-gather-59752"
Jul 24 18:01:21.261730 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:21.261594    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ded439dd-0edc-488a-a657-e4a3e6dc2c02-must-gather-output\") pod \"must-gather-59752\" (UID: \"ded439dd-0edc-488a-a657-e4a3e6dc2c02\") " pod="openshift-must-gather-g7vgp/must-gather-59752"
Jul 24 18:01:21.261905 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:21.261891    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ded439dd-0edc-488a-a657-e4a3e6dc2c02-must-gather-output\") pod \"must-gather-59752\" (UID: \"ded439dd-0edc-488a-a657-e4a3e6dc2c02\") " pod="openshift-must-gather-g7vgp/must-gather-59752"
Jul 24 18:01:21.271676 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:21.271641    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dt5h\" (UniqueName: \"kubernetes.io/projected/ded439dd-0edc-488a-a657-e4a3e6dc2c02-kube-api-access-6dt5h\") pod \"must-gather-59752\" (UID: \"ded439dd-0edc-488a-a657-e4a3e6dc2c02\") " pod="openshift-must-gather-g7vgp/must-gather-59752"
Jul 24 18:01:21.376497 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:21.376400    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g7vgp/must-gather-59752"
Jul 24 18:01:21.523073 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:21.522989    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g7vgp/must-gather-59752"]
Jul 24 18:01:21.525361 ip-10-0-139-46 kubenswrapper[2581]: W0724 18:01:21.525329    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podded439dd_0edc_488a_a657_e4a3e6dc2c02.slice/crio-46a25bffbf6a001e65e9891bd1864d58fa5d27481c9492174e94457d4a4b62a8 WatchSource:0}: Error finding container 46a25bffbf6a001e65e9891bd1864d58fa5d27481c9492174e94457d4a4b62a8: Status 404 returned error can't find the container with id 46a25bffbf6a001e65e9891bd1864d58fa5d27481c9492174e94457d4a4b62a8
Jul 24 18:01:21.618650 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:21.618594    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g7vgp/must-gather-59752" event={"ID":"ded439dd-0edc-488a-a657-e4a3e6dc2c02","Type":"ContainerStarted","Data":"46a25bffbf6a001e65e9891bd1864d58fa5d27481c9492174e94457d4a4b62a8"}
Jul 24 18:01:21.633209 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:21.633177    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 18:01:21.708867 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:21.708826    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 18:01:22.157939 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:22.157905    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 18:01:22.628282 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:22.628205    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g7vgp/must-gather-59752" event={"ID":"ded439dd-0edc-488a-a657-e4a3e6dc2c02","Type":"ContainerStarted","Data":"24736dbbb287b841c6ed9020ff5bd90500de68831037774e4bde24e9d7e434a1"}
Jul 24 18:01:23.636843 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:23.636762    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g7vgp/must-gather-59752" event={"ID":"ded439dd-0edc-488a-a657-e4a3e6dc2c02","Type":"ContainerStarted","Data":"3a941629ad732182f07dc4f3b903e4990df0739c1a770921b64889b8d99df33c"}
Jul 24 18:01:23.655381 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:23.655329    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-g7vgp/must-gather-59752" podStartSLOduration=1.753394937 podStartE2EDuration="2.655311182s" totalImagesPullingTime="901.916245ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:01:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:01:23.652144819 +0000 UTC m=+3050.854194094" watchObservedRunningTime="2026-07-24 18:01:23.655311182 +0000 UTC m=+3050.857360456"
Jul 24 18:01:24.106944 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:24.106851    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-v42pr_24d6b216-f67b-4d45-bd22-c9f159255177/global-pull-secret-syncer/0.log"
Jul 24 18:01:24.161076 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:24.161024    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log"
Jul 24 18:01:24.273481 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:24.273447    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-46.ec2.internal_39f953389fbb7b355964e9b9fe5538ef/haproxy/0.log"
Jul 24 18:01:27.412243 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:27.412140    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4c827_fe4d8ffb-2d02-410f-bd2c-7bd1eb961c70/konnectivity-agent/0.log" limit=1024
Jul 24 18:01:29.718549 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:29.718513    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-cswnr_68a9959d-3d81-4dbc-9bd9-b447935635bf/kuadrant-console-plugin/0.log"
Jul 24 18:01:30.050764 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:30.050626    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-5dbvm_10336757-378d-4415-ad40-56386dd911bb/limitador/0.log"
Jul 24 18:01:31.802096 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:31.802066    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_209f3273-8d94-4c95-8b73-dbe24cdfe4d2/alertmanager/0.log"
Jul 24 18:01:31.860321 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:31.860293    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_209f3273-8d94-4c95-8b73-dbe24cdfe4d2/config-reloader/0.log"
Jul 24 18:01:31.915374 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:31.915315    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_209f3273-8d94-4c95-8b73-dbe24cdfe4d2/kube-rbac-proxy-web/0.log"
Jul 24 18:01:31.956213 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:31.956118    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_209f3273-8d94-4c95-8b73-dbe24cdfe4d2/kube-rbac-proxy/0.log"
Jul 24 18:01:32.016098 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:32.015979    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_209f3273-8d94-4c95-8b73-dbe24cdfe4d2/kube-rbac-proxy-metric/0.log"
Jul 24 18:01:32.049828 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:32.049803    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_209f3273-8d94-4c95-8b73-dbe24cdfe4d2/prom-label-proxy/0.log"
Jul 24 18:01:32.073298 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:32.073271    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_209f3273-8d94-4c95-8b73-dbe24cdfe4d2/init-config-reloader/0.log"
Jul 24 18:01:32.172569 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:32.172532    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-f9qj2_fac1bdfa-4c1a-480d-9b1a-e3b025e473bb/kube-state-metrics/0.log"
Jul 24 18:01:32.192248 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:32.192220    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-f9qj2_fac1bdfa-4c1a-480d-9b1a-e3b025e473bb/kube-rbac-proxy-main/0.log"
Jul 24 18:01:32.227550 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:32.227523    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-f9qj2_fac1bdfa-4c1a-480d-9b1a-e3b025e473bb/kube-rbac-proxy-self/0.log"
Jul 24 18:01:32.257357 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:32.257329    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5dc5d7846d-8hxgs_e076379a-b0e0-4fa9-9cef-5180aede4d56/metrics-server/0.log"
Jul 24 18:01:32.298016 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:32.297990    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-m72fg_1aaa688e-4aad-4120-b7f0-64f7574e4aad/monitoring-plugin/0.log"
Jul 24 18:01:32.395788 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:32.395708    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dvz8j_d8e047ab-f0c3-48cc-84c2-19559698ab11/node-exporter/0.log"
Jul 24 18:01:32.416706 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:32.416670    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dvz8j_d8e047ab-f0c3-48cc-84c2-19559698ab11/kube-rbac-proxy/0.log"
Jul 24 18:01:32.437618 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:32.437590    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dvz8j_d8e047ab-f0c3-48cc-84c2-19559698ab11/init-textfile/0.log"
Jul 24 18:01:32.527657 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:32.527623    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-8pb55_a2596065-10a4-4cca-bdca-04d5aa1208a2/kube-rbac-proxy-main/0.log"
Jul 24 18:01:32.555317 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:32.555193    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-8pb55_a2596065-10a4-4cca-bdca-04d5aa1208a2/kube-rbac-proxy-self/0.log"
Jul 24 18:01:32.576664 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:32.576625    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-8pb55_a2596065-10a4-4cca-bdca-04d5aa1208a2/openshift-state-metrics/0.log"
Jul 24 18:01:32.611462 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:32.611430    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f5ce6020-a3c4-45cb-ae9b-086df02dd950/prometheus/0.log"
Jul 24 18:01:32.630817 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:32.630778    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f5ce6020-a3c4-45cb-ae9b-086df02dd950/config-reloader/0.log"
Jul 24 18:01:32.648390 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:32.648360    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f5ce6020-a3c4-45cb-ae9b-086df02dd950/thanos-sidecar/0.log"
Jul 24 18:01:32.668319 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:32.668289    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f5ce6020-a3c4-45cb-ae9b-086df02dd950/kube-rbac-proxy-web/0.log"
Jul 24 18:01:32.689887 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:32.689827    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f5ce6020-a3c4-45cb-ae9b-086df02dd950/kube-rbac-proxy/0.log"
Jul 24 18:01:32.712074 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:32.712014    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f5ce6020-a3c4-45cb-ae9b-086df02dd950/kube-rbac-proxy-thanos/0.log"
Jul 24 18:01:32.733994 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:32.733962    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f5ce6020-a3c4-45cb-ae9b-086df02dd950/init-config-reloader/0.log"
Jul 24 18:01:32.761231 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:32.761197    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-h88gg_546de716-d864-4169-ac8a-c9bae8467697/prometheus-operator/0.log"
Jul 24 18:01:32.782781 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:32.782748    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-h88gg_546de716-d864-4169-ac8a-c9bae8467697/kube-rbac-proxy/0.log"
Jul 24 18:01:32.806194 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:32.806119    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-6p8zw_7eba8703-9720-4dcf-9883-e47e9a46ce76/prometheus-operator-admission-webhook/0.log"
Jul 24 18:01:32.841374 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:32.841316    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7569b5b876-z2trz_31e30a4b-cb9c-42ea-a5c0-484a75fe6a12/telemeter-client/0.log"
Jul 24 18:01:32.862403 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:32.862374    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7569b5b876-z2trz_31e30a4b-cb9c-42ea-a5c0-484a75fe6a12/reload/0.log"
Jul 24 18:01:32.882247 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:32.882211    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7569b5b876-z2trz_31e30a4b-cb9c-42ea-a5c0-484a75fe6a12/kube-rbac-proxy/0.log"
Jul 24 18:01:32.925388 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:32.925361    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-dd94759f-6lss7_cce52072-dc7f-43b7-ac2d-318e50963272/thanos-query/0.log"
Jul 24 18:01:32.945421 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:32.945370    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-dd94759f-6lss7_cce52072-dc7f-43b7-ac2d-318e50963272/kube-rbac-proxy-web/0.log"
Jul 24 18:01:32.965059 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:32.965018    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-dd94759f-6lss7_cce52072-dc7f-43b7-ac2d-318e50963272/kube-rbac-proxy/0.log"
Jul 24 18:01:32.988765 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:32.988737    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-dd94759f-6lss7_cce52072-dc7f-43b7-ac2d-318e50963272/prom-label-proxy/0.log"
Jul 24 18:01:33.013935 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:33.013906    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-dd94759f-6lss7_cce52072-dc7f-43b7-ac2d-318e50963272/kube-rbac-proxy-rules/0.log"
Jul 24 18:01:33.040881 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:33.040852    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-dd94759f-6lss7_cce52072-dc7f-43b7-ac2d-318e50963272/kube-rbac-proxy-metrics/0.log"
Jul 24 18:01:34.040684 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:34.040653    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-g7vgp/perf-node-gather-daemonset-cpslp"]
Jul 24 18:01:34.045723 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:34.045696    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-cpslp"
Jul 24 18:01:34.055612 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:34.055581    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g7vgp/perf-node-gather-daemonset-cpslp"]
Jul 24 18:01:34.108152 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:34.108110    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/364a94b5-2637-4ba2-9d48-fe94de898931-podres\") pod \"perf-node-gather-daemonset-cpslp\" (UID: \"364a94b5-2637-4ba2-9d48-fe94de898931\") " pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-cpslp"
Jul 24 18:01:34.108361 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:34.108242    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/364a94b5-2637-4ba2-9d48-fe94de898931-sys\") pod \"perf-node-gather-daemonset-cpslp\" (UID: \"364a94b5-2637-4ba2-9d48-fe94de898931\") " pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-cpslp"
Jul 24 18:01:34.108361 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:34.108276    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/364a94b5-2637-4ba2-9d48-fe94de898931-lib-modules\") pod \"perf-node-gather-daemonset-cpslp\" (UID: \"364a94b5-2637-4ba2-9d48-fe94de898931\") " pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-cpslp"
Jul 24 18:01:34.108361 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:34.108315    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc5pj\" (UniqueName: \"kubernetes.io/projected/364a94b5-2637-4ba2-9d48-fe94de898931-kube-api-access-wc5pj\") pod \"perf-node-gather-daemonset-cpslp\" (UID: \"364a94b5-2637-4ba2-9d48-fe94de898931\") " pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-cpslp"
Jul 24 18:01:34.108723 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:34.108372    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/364a94b5-2637-4ba2-9d48-fe94de898931-proc\") pod \"perf-node-gather-daemonset-cpslp\" (UID: \"364a94b5-2637-4ba2-9d48-fe94de898931\") " pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-cpslp"
Jul 24 18:01:34.208899 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:34.208862    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/364a94b5-2637-4ba2-9d48-fe94de898931-sys\") pod \"perf-node-gather-daemonset-cpslp\" (UID: \"364a94b5-2637-4ba2-9d48-fe94de898931\") " pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-cpslp"
Jul 24 18:01:34.208899 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:34.208903    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/364a94b5-2637-4ba2-9d48-fe94de898931-lib-modules\") pod \"perf-node-gather-daemonset-cpslp\" (UID: \"364a94b5-2637-4ba2-9d48-fe94de898931\") " pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-cpslp"
Jul 24 18:01:34.209204 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:34.208922    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc5pj\" (UniqueName: \"kubernetes.io/projected/364a94b5-2637-4ba2-9d48-fe94de898931-kube-api-access-wc5pj\") pod \"perf-node-gather-daemonset-cpslp\" (UID: \"364a94b5-2637-4ba2-9d48-fe94de898931\") " pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-cpslp"
Jul 24 18:01:34.209204 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:34.208952    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/364a94b5-2637-4ba2-9d48-fe94de898931-proc\") pod \"perf-node-gather-daemonset-cpslp\" (UID: \"364a94b5-2637-4ba2-9d48-fe94de898931\") " pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-cpslp"
Jul 24 18:01:34.209204 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:34.208993    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/364a94b5-2637-4ba2-9d48-fe94de898931-podres\") pod \"perf-node-gather-daemonset-cpslp\" (UID: \"364a94b5-2637-4ba2-9d48-fe94de898931\") " pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-cpslp"
Jul 24 18:01:34.209204 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:34.209176    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/364a94b5-2637-4ba2-9d48-fe94de898931-sys\") pod \"perf-node-gather-daemonset-cpslp\" (UID: \"364a94b5-2637-4ba2-9d48-fe94de898931\") " pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-cpslp"
Jul 24 18:01:34.209204 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:34.209192    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/364a94b5-2637-4ba2-9d48-fe94de898931-podres\") pod \"perf-node-gather-daemonset-cpslp\" (UID: \"364a94b5-2637-4ba2-9d48-fe94de898931\") " pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-cpslp"
Jul 24 18:01:34.209492 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:34.209191    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/364a94b5-2637-4ba2-9d48-fe94de898931-lib-modules\") pod \"perf-node-gather-daemonset-cpslp\" (UID: \"364a94b5-2637-4ba2-9d48-fe94de898931\") " pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-cpslp"
Jul 24 18:01:34.209492 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:34.209232    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/364a94b5-2637-4ba2-9d48-fe94de898931-proc\") pod \"perf-node-gather-daemonset-cpslp\" (UID: \"364a94b5-2637-4ba2-9d48-fe94de898931\") " pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-cpslp"
Jul 24 18:01:34.218128 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:34.218088    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc5pj\" (UniqueName: \"kubernetes.io/projected/364a94b5-2637-4ba2-9d48-fe94de898931-kube-api-access-wc5pj\") pod \"perf-node-gather-daemonset-cpslp\" (UID: \"364a94b5-2637-4ba2-9d48-fe94de898931\") " pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-cpslp"
Jul 24 18:01:34.359108 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:34.358996    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-cpslp"
Jul 24 18:01:34.549332 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:34.549298    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g7vgp/perf-node-gather-daemonset-cpslp"]
Jul 24 18:01:34.736638 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:34.736302    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-cpslp" event={"ID":"364a94b5-2637-4ba2-9d48-fe94de898931","Type":"ContainerStarted","Data":"dddfb675e77c3809f38dcc4c14d602060a92d8e947dea19b6ca27449ecd3ae2d"}
Jul 24 18:01:34.736638 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:34.736374    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-cpslp" event={"ID":"364a94b5-2637-4ba2-9d48-fe94de898931","Type":"ContainerStarted","Data":"fce8daad5721192f19736cb35c0de086cc51df3e6bde11b45e21b47147fdaf2f"}
Jul 24 18:01:34.736897 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:34.736677    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-cpslp"
Jul 24 18:01:34.755643 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:34.755572    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-cpslp" podStartSLOduration=0.75555027 podStartE2EDuration="755.55027ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 18:01:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:01:34.754151819 +0000 UTC m=+3061.956201095" watchObservedRunningTime="2026-07-24 18:01:34.75555027 +0000 UTC m=+3061.957599545"
Jul 24 18:01:35.397486 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:35.397458    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-wf48c_d625ce61-acef-427e-9d65-75720151fcd7/download-server/0.log"
Jul 24 18:01:37.151054 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:37.151008    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tt7pb_c483050c-0fa4-4739-865c-d4aa9fda7cf1/dns/0.log"
Jul 24 18:01:37.172584 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:37.172548    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tt7pb_c483050c-0fa4-4739-865c-d4aa9fda7cf1/kube-rbac-proxy/0.log"
Jul 24 18:01:37.265680 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:37.265646    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hpxzm_ccfd714f-9f15-40f6-a06c-6054adfb9992/dns-node-resolver/0.log"
Jul 24 18:01:37.928476 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:37.928449    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hv6sp_17ed6b09-72e1-4161-9a92-cb481241bfe1/node-ca/0.log"
Jul 24 18:01:39.035807 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:39.035773    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-84b496c558-987vc_fe8593b3-ba31-4b15-9485-281e108a027f/kube-auth-proxy/0.log"
Jul 24 18:01:40.096532 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:40.096505    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sg66r_339766e7-0509-456d-8cc7-790156dfb1be/serve-healthcheck-canary/0.log"
Jul 24 18:01:40.635262 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:40.635237    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8x79m_44c6ba96-c4bc-42c6-b993-43a864544dff/kube-rbac-proxy/0.log"
Jul 24 18:01:40.653110 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:40.653074    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8x79m_44c6ba96-c4bc-42c6-b993-43a864544dff/exporter/0.log"
Jul 24 18:01:40.671396 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:40.671369    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8x79m_44c6ba96-c4bc-42c6-b993-43a864544dff/extractor/0.log"
Jul 24 18:01:40.758196 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:40.758168    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-cpslp"
Jul 24 18:01:43.960635 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:43.960598    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bcd74455c-r4pqr_ad445777-f665-482c-9f35-2eb21a836bb2/manager/0.log"
Jul 24 18:01:43.987774 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:43.987745    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bcd74455c-r4pqr_ad445777-f665-482c-9f35-2eb21a836bb2/copy-manifests/0.log"
Jul 24 18:01:45.093650 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:45.093569    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-84b496c558-987vc_fe8593b3-ba31-4b15-9485-281e108a027f/kube-auth-proxy/0.log"
Jul 24 18:01:45.259969 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:45.259942    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-57c7c45669-pf7jm_b295a22b-ef6d-4c71-9925-ca8919ab0be6/manager/0.log"
Jul 24 18:01:46.676285 ip-10-0-139-46 kubenswrapper[2581]: I0724 18:01:46.676251    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-84b496c558-987vc_fe8593b3-ba31-4b15-9485-281e108a027f/kube-auth-proxy/0.log"