Jul 30 03:34:54.258233 ip-10-0-143-117 systemd[1]: Starting Kubernetes Kubelet...
Jul 30 03:34:54.716010 ip-10-0-143-117 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 03:34:54.716010 ip-10-0-143-117 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 30 03:34:54.716010 ip-10-0-143-117 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 03:34:54.716010 ip-10-0-143-117 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 30 03:34:54.716010 ip-10-0-143-117 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 03:34:54.719059 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.718951    2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 30 03:34:54.726110 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726082    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 03:34:54.726110 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726109    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 03:34:54.726198 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726114    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 03:34:54.726198 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726118    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 03:34:54.726198 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726122    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 03:34:54.726198 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726125    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 03:34:54.726198 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726128    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 03:34:54.726198 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726131    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 03:34:54.726198 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726134    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 03:34:54.726198 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726137    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 03:34:54.726198 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726140    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 03:34:54.726198 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726143    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 03:34:54.726198 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726145    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 03:34:54.726198 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726148    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 03:34:54.726198 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726151    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 03:34:54.726198 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726154    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 03:34:54.726198 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726157    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 03:34:54.726198 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726160    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 03:34:54.726198 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726163    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 03:34:54.726198 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726165    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 03:34:54.726198 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726168    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 03:34:54.726700 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726171    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 03:34:54.726700 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726176    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 03:34:54.726700 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726179    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 03:34:54.726700 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726182    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 03:34:54.726700 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726186    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 03:34:54.726700 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726192    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 03:34:54.726700 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726195    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 03:34:54.726700 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726199    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 03:34:54.726700 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726202    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 03:34:54.726700 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726207    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 03:34:54.726700 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726211    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 03:34:54.726700 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726222    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 03:34:54.726700 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726225    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 03:34:54.726700 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726228    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 03:34:54.726700 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726231    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 03:34:54.726700 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726234    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 03:34:54.726700 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726237    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 03:34:54.726700 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726239    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 03:34:54.726700 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726242    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 03:34:54.727218 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726245    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 03:34:54.727218 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726249    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 03:34:54.727218 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726252    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 03:34:54.727218 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726255    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 03:34:54.727218 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726257    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 03:34:54.727218 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726260    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 03:34:54.727218 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726263    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 03:34:54.727218 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726266    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 03:34:54.727218 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726270    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 03:34:54.727218 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726272    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 03:34:54.727218 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726275    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 03:34:54.727218 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726278    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 03:34:54.727218 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726280    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 03:34:54.727218 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726286    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 03:34:54.727218 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726289    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 03:34:54.727218 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726292    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 03:34:54.727218 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726295    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 03:34:54.727218 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726298    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 03:34:54.727218 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726300    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 03:34:54.727218 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726303    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 03:34:54.727790 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726306    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 03:34:54.727790 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726309    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 03:34:54.727790 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726312    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 03:34:54.727790 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726316    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 03:34:54.727790 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726318    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 03:34:54.727790 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726321    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 03:34:54.727790 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726324    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 03:34:54.727790 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726327    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 03:34:54.727790 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726330    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 03:34:54.727790 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726333    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 03:34:54.727790 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726336    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 03:34:54.727790 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726338    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 03:34:54.727790 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726341    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 03:34:54.727790 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726344    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 03:34:54.727790 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726346    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 03:34:54.727790 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726349    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 03:34:54.727790 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726353    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 03:34:54.727790 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726357    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 03:34:54.727790 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726360    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 03:34:54.727790 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726363    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 03:34:54.728345 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726365    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 03:34:54.728345 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726369    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 03:34:54.728345 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726372    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 03:34:54.728345 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726375    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 03:34:54.728345 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726378    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 03:34:54.728345 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726381    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 03:34:54.728345 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726383    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 03:34:54.728345 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726386    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 03:34:54.728345 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726389    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 03:34:54.728345 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726392    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 03:34:54.728345 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726394    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 03:34:54.728345 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726397    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 03:34:54.728345 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726400    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 03:34:54.728345 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726402    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 03:34:54.728345 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726405    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 03:34:54.728345 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726408    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 03:34:54.728345 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726411    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 03:34:54.728345 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726413    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 03:34:54.728345 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.726418    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 03:34:54.728345 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727259    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 03:34:54.728873 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727269    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 03:34:54.728873 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727273    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 03:34:54.728873 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727277    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 03:34:54.728873 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727281    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 03:34:54.728873 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727284    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 03:34:54.728873 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727288    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 03:34:54.728873 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727291    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 03:34:54.728873 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727294    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 03:34:54.728873 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727298    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 03:34:54.728873 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727301    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 03:34:54.728873 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727304    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 03:34:54.728873 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727307    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 03:34:54.728873 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727310    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 03:34:54.728873 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727313    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 03:34:54.728873 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727317    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 03:34:54.728873 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727320    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 03:34:54.728873 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727324    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 03:34:54.728873 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727326    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 03:34:54.728873 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727330    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 03:34:54.728873 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727332    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 03:34:54.729431 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727335    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 03:34:54.729431 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727338    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 03:34:54.729431 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727341    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 03:34:54.729431 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727344    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 03:34:54.729431 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727346    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 03:34:54.729431 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727349    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 03:34:54.729431 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727352    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 03:34:54.729431 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727355    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 03:34:54.729431 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727357    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 03:34:54.729431 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727360    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 03:34:54.729431 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727362    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 03:34:54.729431 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727365    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 03:34:54.729431 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727368    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 03:34:54.729431 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727371    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 03:34:54.729431 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727374    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 03:34:54.729431 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727377    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 03:34:54.729431 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727379    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 03:34:54.729431 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727383    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 03:34:54.729431 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727385    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 03:34:54.730015 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727388    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 03:34:54.730015 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727391    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 03:34:54.730015 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727394    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 03:34:54.730015 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727398    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 03:34:54.730015 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727401    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 03:34:54.730015 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727404    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 03:34:54.730015 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727407    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 03:34:54.730015 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727410    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 03:34:54.730015 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727413    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 03:34:54.730015 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727416    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 03:34:54.730015 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727419    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 03:34:54.730015 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727422    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 03:34:54.730015 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727425    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 03:34:54.730015 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727428    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 03:34:54.730015 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727430    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 03:34:54.730015 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727433    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 03:34:54.730015 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727435    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 03:34:54.730015 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727439    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 03:34:54.730015 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727444    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 03:34:54.730525 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727448    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 03:34:54.730525 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727451    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 03:34:54.730525 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727454    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 03:34:54.730525 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727457    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 03:34:54.730525 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727460    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 03:34:54.730525 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727464    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 03:34:54.730525 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727466    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 03:34:54.730525 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727469    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 03:34:54.730525 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727472    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 03:34:54.730525 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727475    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 03:34:54.730525 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727477    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 03:34:54.730525 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727480    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 03:34:54.730525 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727483    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 03:34:54.730525 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727486    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 03:34:54.730525 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727491    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 03:34:54.730525 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727494    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 03:34:54.730525 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727497    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 03:34:54.730525 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727500    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 03:34:54.730525 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727503    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 03:34:54.730525 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727506    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 03:34:54.731072 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727509    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 03:34:54.731072 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727512    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 03:34:54.731072 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727520    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 03:34:54.731072 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727523    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 03:34:54.731072 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727526    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 03:34:54.731072 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727529    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 03:34:54.731072 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727532    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 03:34:54.731072 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727535    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 03:34:54.731072 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727538    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 03:34:54.731072 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727540    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 03:34:54.731072 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727543    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 03:34:54.731072 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727546    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 03:34:54.731072 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727549    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 03:34:54.731072 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727552    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 03:34:54.731072 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727554    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 03:34:54.731072 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727557    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 03:34:54.731072 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727560    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 03:34:54.731072 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727563    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 03:34:54.731072 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727565    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 03:34:54.731072 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.727568    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 03:34:54.731601 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728023    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 03:34:54.731601 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728029    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 03:34:54.731601 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728032    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 03:34:54.731601 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728036    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 03:34:54.731601 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728039    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 03:34:54.731601 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728042    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 03:34:54.731601 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728045    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 03:34:54.731601 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728048    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 03:34:54.731601 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728051    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 03:34:54.731601 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728054    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 03:34:54.731601 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728057    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 03:34:54.731601 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728060    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 03:34:54.731601 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728063    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 03:34:54.731601 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728066    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 03:34:54.731601 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728069    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 03:34:54.731601 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728072    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 03:34:54.731601 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728075    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 03:34:54.731601 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728077    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 03:34:54.731601 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728080    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 03:34:54.731601 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728083    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 03:34:54.732150 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728086    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 03:34:54.732150 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728089    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 03:34:54.732150 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728091    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 03:34:54.732150 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728094    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 03:34:54.732150 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728097    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 03:34:54.732150 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728099    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 03:34:54.732150 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728102    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 03:34:54.732150 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728105    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 03:34:54.732150 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728108    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 03:34:54.732150 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728110    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 03:34:54.732150 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728113    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 03:34:54.732150 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728116    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 03:34:54.732150 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728120    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 03:34:54.732150 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728124    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 03:34:54.732150 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728127    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 03:34:54.732150 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728130    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 03:34:54.732150 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728133    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 03:34:54.732150 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728135    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 03:34:54.732150 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728138    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 03:34:54.732150 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728141    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 03:34:54.732731 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728144    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 03:34:54.732731 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728151    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 03:34:54.732731 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728154    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 03:34:54.732731 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728157    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 03:34:54.732731 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728160    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 03:34:54.732731 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728163    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 03:34:54.732731 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728165    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 03:34:54.732731 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728169    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 03:34:54.732731 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728172    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 03:34:54.732731 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728175    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 03:34:54.732731 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728178    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 03:34:54.732731 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728181    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 03:34:54.732731 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728184    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 03:34:54.732731 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728211    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 03:34:54.732731 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728214    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 03:34:54.732731 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728217    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 03:34:54.732731 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728221    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 03:34:54.732731 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728224    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 03:34:54.732731 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728227    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 03:34:54.733265 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728230    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 03:34:54.733265 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728233    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 03:34:54.733265 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728235    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 03:34:54.733265 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728238    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 03:34:54.733265 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728241    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 03:34:54.733265 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728245    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 03:34:54.733265 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728247    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 03:34:54.733265 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728250    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 03:34:54.733265 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728255    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 03:34:54.733265 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728258    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 03:34:54.733265 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728261    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 03:34:54.733265 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728263    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 03:34:54.733265 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728266    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 03:34:54.733265 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728269    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 03:34:54.733265 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728273    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 03:34:54.733265 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728276    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 03:34:54.733265 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728279    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 03:34:54.733265 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728282    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 03:34:54.733265 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728286    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 03:34:54.733265 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728289    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 03:34:54.733811 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728292    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 03:34:54.733811 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728295    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 03:34:54.733811 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728298    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 03:34:54.733811 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728301    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 03:34:54.733811 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728304    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 03:34:54.733811 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728308    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 03:34:54.733811 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728310    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 03:34:54.733811 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728313    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 03:34:54.733811 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728318    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 03:34:54.733811 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728322    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 03:34:54.733811 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728325    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 03:34:54.733811 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728328    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 03:34:54.733811 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728331    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 03:34:54.733811 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728334    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 03:34:54.733811 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728337    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 03:34:54.733811 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728340    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 03:34:54.733811 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728344    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 03:34:54.733811 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728347    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 03:34:54.733811 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728350    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 03:34:54.734322 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.728353    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 03:34:54.734322 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728446    2576 flags.go:64] FLAG: --address="0.0.0.0"
Jul 30 03:34:54.734322 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728458    2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 30 03:34:54.734322 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728468    2576 flags.go:64] FLAG: --anonymous-auth="true"
Jul 30 03:34:54.734322 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728475    2576 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 30 03:34:54.734322 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728480    2576 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 30 03:34:54.734322 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728483    2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 30 03:34:54.734322 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728489    2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 30 03:34:54.734322 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728494    2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 30 03:34:54.734322 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728497    2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 30 03:34:54.734322 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728501    2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 30 03:34:54.734322 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728504    2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 30 03:34:54.734322 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728508    2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 30 03:34:54.734322 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728512    2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 30 03:34:54.734322 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728515    2576 flags.go:64] FLAG: --cgroup-root=""
Jul 30 03:34:54.734322 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728519    2576 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 30 03:34:54.734322 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728522    2576 flags.go:64] FLAG: --client-ca-file=""
Jul 30 03:34:54.734322 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728526    2576 flags.go:64] FLAG: --cloud-provider="external"
Jul 30 03:34:54.734322 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728529    2576 flags.go:64] FLAG: --cluster-dns="[]"
Jul 30 03:34:54.734322 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728538    2576 flags.go:64] FLAG: --cluster-domain=""
Jul 30 03:34:54.734322 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728541    2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 30 03:34:54.734322 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728545    2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 30 03:34:54.734322 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728548    2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 30 03:34:54.734322 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728552    2576 flags.go:64] FLAG: --container-log-max-files="5"
Jul 30 03:34:54.734937 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728556    2576 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 30 03:34:54.734937 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728559    2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 30 03:34:54.734937 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728563    2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 30 03:34:54.734937 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728566    2576 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 30 03:34:54.734937 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728570    2576 flags.go:64] FLAG: --contention-profiling="false"
Jul 30 03:34:54.734937 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728573    2576 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 30 03:34:54.734937 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728577    2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 30 03:34:54.734937 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728581    2576 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 30 03:34:54.734937 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728584    2576 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 30 03:34:54.734937 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728591    2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 30 03:34:54.734937 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728594    2576 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 30 03:34:54.734937 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728598    2576 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 30 03:34:54.734937 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728601    2576 flags.go:64] FLAG: --enable-load-reader="false"
Jul 30 03:34:54.734937 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728604    2576 flags.go:64] FLAG: --enable-server="true"
Jul 30 03:34:54.734937 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728608    2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 30 03:34:54.734937 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728613    2576 flags.go:64] FLAG: --event-burst="100"
Jul 30 03:34:54.734937 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728617    2576 flags.go:64] FLAG: --event-qps="50"
Jul 30 03:34:54.734937 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728620    2576 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 30 03:34:54.734937 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728624    2576 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 30 03:34:54.734937 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728627    2576 flags.go:64] FLAG: --eviction-hard=""
Jul 30 03:34:54.734937 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728632    2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 30 03:34:54.734937 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728635    2576 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 30 03:34:54.734937 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728638    2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 30 03:34:54.734937 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728642    2576 flags.go:64] FLAG: --eviction-soft=""
Jul 30 03:34:54.734937 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728646    2576 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 30 03:34:54.735618 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728649    2576 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 30 03:34:54.735618 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728653    2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 30 03:34:54.735618 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728656    2576 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 30 03:34:54.735618 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728659    2576 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 30 03:34:54.735618 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728662    2576 flags.go:64] FLAG: --fail-swap-on="true"
Jul 30 03:34:54.735618 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728665    2576 flags.go:64] FLAG: --feature-gates=""
Jul 30 03:34:54.735618 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728669    2576 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 30 03:34:54.735618 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728673    2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 30 03:34:54.735618 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728676    2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 30 03:34:54.735618 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728680    2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 30 03:34:54.735618 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728683    2576 flags.go:64] FLAG: --healthz-port="10248"
Jul 30 03:34:54.735618 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728687    2576 flags.go:64] FLAG: --help="false"
Jul 30 03:34:54.735618 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728690    2576 flags.go:64] FLAG: --hostname-override="ip-10-0-143-117.ec2.internal"
Jul 30 03:34:54.735618 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728693    2576 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 30 03:34:54.735618 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728696    2576 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 30 03:34:54.735618 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728699    2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 30 03:34:54.735618 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728705    2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 30 03:34:54.735618 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728708    2576 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 30 03:34:54.735618 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728711    2576 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 30 03:34:54.735618 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728714    2576 flags.go:64] FLAG: --image-service-endpoint=""
Jul 30 03:34:54.735618 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728718    2576 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 30 03:34:54.735618 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728721    2576 flags.go:64] FLAG: --kube-api-burst="100"
Jul 30 03:34:54.735618 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728724    2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 30 03:34:54.736311 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728728    2576 flags.go:64] FLAG: --kube-api-qps="50"
Jul 30 03:34:54.736311 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728731    2576 flags.go:64] FLAG: --kube-reserved=""
Jul 30 03:34:54.736311 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728735    2576 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 30 03:34:54.736311 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728738    2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 30 03:34:54.736311 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728741    2576 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 30 03:34:54.736311 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728744    2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 30 03:34:54.736311 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728747    2576 flags.go:64] FLAG: --lock-file=""
Jul 30 03:34:54.736311 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728751    2576 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 30 03:34:54.736311 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728754    2576 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 30 03:34:54.736311 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728758    2576 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 30 03:34:54.736311 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728764    2576 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 30 03:34:54.736311 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728767    2576 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 30 03:34:54.736311 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728770    2576 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 30 03:34:54.736311 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728773    2576 flags.go:64] FLAG: --logging-format="text"
Jul 30 03:34:54.736311 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728777    2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 30 03:34:54.736311 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728780    2576 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 30 03:34:54.736311 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728783    2576 flags.go:64] FLAG: --manifest-url=""
Jul 30 03:34:54.736311 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728787    2576 flags.go:64] FLAG: --manifest-url-header=""
Jul 30 03:34:54.736311 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728792    2576 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 30 03:34:54.736311 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728795    2576 flags.go:64] FLAG: --max-open-files="1000000"
Jul 30 03:34:54.736311 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728800    2576 flags.go:64] FLAG: --max-pods="110"
Jul 30 03:34:54.736311 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728803    2576 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 30 03:34:54.736311 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728806    2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 30 03:34:54.736311 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728809    2576 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 30 03:34:54.736311 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728813    2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 30 03:34:54.737077 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728817    2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 30 03:34:54.737077 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728821    2576 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 30 03:34:54.737077 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728825    2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 30 03:34:54.737077 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728835    2576 flags.go:64] FLAG: --node-status-max-images="50"
Jul 30 03:34:54.737077 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728838    2576 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 30 03:34:54.737077 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728842    2576 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 30 03:34:54.737077 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728846    2576 flags.go:64] FLAG: --pod-cidr=""
Jul 30 03:34:54.737077 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728849    2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 30 03:34:54.737077 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728855    2576 flags.go:64] FLAG: --pod-manifest-path=""
Jul 30 03:34:54.737077 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728858    2576 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 30 03:34:54.737077 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728861    2576 flags.go:64] FLAG: --pods-per-core="0"
Jul 30 03:34:54.737077 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728864    2576 flags.go:64] FLAG: --port="10250"
Jul 30 03:34:54.737077 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728868    2576 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 30 03:34:54.737077 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728871    2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03a7313d7bfa3a4da"
Jul 30 03:34:54.737077 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728875    2576 flags.go:64] FLAG: --qos-reserved=""
Jul 30 03:34:54.737077 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728878    2576 flags.go:64] FLAG: --read-only-port="10255"
Jul 30 03:34:54.737077 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728881    2576 flags.go:64] FLAG: --register-node="true"
Jul 30 03:34:54.737077 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728885    2576 flags.go:64] FLAG: --register-with-taints=""
Jul 30 03:34:54.737077 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728890    2576 flags.go:64] FLAG: --registry-burst="10"
Jul 30 03:34:54.737077 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728893    2576 flags.go:64] FLAG: --registry-qps="5"
Jul 30 03:34:54.737077 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728896    2576 flags.go:64] FLAG: --reserved-cpus=""
Jul 30 03:34:54.737077 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728899    2576 flags.go:64] FLAG: --reserved-memory=""
Jul 30 03:34:54.737077 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728903    2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 30 03:34:54.737077 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728906    2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 30 03:34:54.737077 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728910    2576 flags.go:64] FLAG: --rotate-certificates="false"
Jul 30 03:34:54.737753 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728914    2576 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 30 03:34:54.737753 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728917    2576 flags.go:64] FLAG: --runonce="false"
Jul 30 03:34:54.737753 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728920    2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 30 03:34:54.737753 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728923    2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 30 03:34:54.737753 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728927    2576 flags.go:64] FLAG: --seccomp-default="false"
Jul 30 03:34:54.737753 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728930    2576 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 30 03:34:54.737753 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728933    2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 30 03:34:54.737753 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728936    2576 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 30 03:34:54.737753 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728941    2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 30 03:34:54.737753 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728945    2576 flags.go:64] FLAG: --storage-driver-password="root"
Jul 30 03:34:54.737753 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728948    2576 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 30 03:34:54.737753 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728951    2576 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 30 03:34:54.737753 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728957    2576 flags.go:64] FLAG: --storage-driver-user="root"
Jul 30 03:34:54.737753 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728960    2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 30 03:34:54.737753 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728963    2576 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 30 03:34:54.737753 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728966    2576 flags.go:64] FLAG: --system-cgroups=""
Jul 30 03:34:54.737753 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728970    2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 30 03:34:54.737753 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728976    2576 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 30 03:34:54.737753 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728979    2576 flags.go:64] FLAG: --tls-cert-file=""
Jul 30 03:34:54.737753 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728982    2576 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 30 03:34:54.737753 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728987    2576 flags.go:64] FLAG: --tls-min-version=""
Jul 30 03:34:54.737753 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728990    2576 flags.go:64] FLAG: --tls-private-key-file=""
Jul 30 03:34:54.737753 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.728993    2576 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 30 03:34:54.737753 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.729009    2576 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 30 03:34:54.737753 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.729012    2576 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 30 03:34:54.738446 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.729016    2576 flags.go:64] FLAG: --v="2"
Jul 30 03:34:54.738446 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.729021    2576 flags.go:64] FLAG: --version="false"
Jul 30 03:34:54.738446 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.729029    2576 flags.go:64] FLAG: --vmodule=""
Jul 30 03:34:54.738446 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.729033    2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 30 03:34:54.738446 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.729037    2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 30 03:34:54.738446 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729166    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 03:34:54.738446 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729171    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 03:34:54.738446 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729174    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 03:34:54.738446 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729177    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 03:34:54.738446 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729180    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 03:34:54.738446 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729183    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 03:34:54.738446 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729186    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 03:34:54.738446 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729189    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 03:34:54.738446 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729192    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 03:34:54.738446 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729195    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 03:34:54.738446 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729199    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 03:34:54.738446 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729202    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 03:34:54.738446 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729205    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 03:34:54.738446 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729208    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 03:34:54.738446 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729212    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 03:34:54.738446 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729216    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 03:34:54.739044 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729218    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 03:34:54.739044 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729221    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 03:34:54.739044 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729224    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 03:34:54.739044 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729229    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 03:34:54.739044 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729232    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 03:34:54.739044 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729235    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 03:34:54.739044 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729239    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 03:34:54.739044 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729242    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 03:34:54.739044 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729245    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 03:34:54.739044 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729247    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 03:34:54.739044 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729250    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 03:34:54.739044 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729253    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 03:34:54.739044 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729256    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 03:34:54.739044 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729259    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 03:34:54.739044 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729262    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 03:34:54.739044 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729265    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 03:34:54.739044 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729268    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 03:34:54.739044 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729272    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 03:34:54.739044 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729275    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 03:34:54.739561 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729278    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 03:34:54.739561 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729281    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 03:34:54.739561 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729284    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 03:34:54.739561 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729288    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 03:34:54.739561 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729290    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 03:34:54.739561 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729293    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 03:34:54.739561 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729296    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 03:34:54.739561 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729301    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 03:34:54.739561 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729304    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 03:34:54.739561 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729308    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 03:34:54.739561 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729312    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 03:34:54.739561 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729317    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 03:34:54.739561 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729320    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 03:34:54.739561 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729323    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 03:34:54.739561 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729326    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 03:34:54.739561 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729328    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 03:34:54.739561 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729331    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 03:34:54.739561 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729334    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 03:34:54.739561 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729337    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 03:34:54.739561 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729340    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 03:34:54.740241 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729342    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 03:34:54.740241 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729345    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 03:34:54.740241 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729348    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 03:34:54.740241 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729351    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 03:34:54.740241 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729354    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 03:34:54.740241 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729357    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 03:34:54.740241 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729360    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 03:34:54.740241 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729363    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 03:34:54.740241 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729366    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 03:34:54.740241 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729369    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 03:34:54.740241 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729371    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 03:34:54.740241 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729374    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 03:34:54.740241 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729377    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 03:34:54.740241 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729380    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 03:34:54.740241 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729382    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 03:34:54.740241 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729385    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 03:34:54.740241 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729388    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 03:34:54.740241 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729390    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 03:34:54.740241 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729393    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 03:34:54.740241 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729398    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 03:34:54.740776 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729400    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 03:34:54.740776 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729403    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 03:34:54.740776 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729406    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 03:34:54.740776 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729411    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 03:34:54.740776 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729414    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 03:34:54.740776 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729417    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 03:34:54.740776 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729420    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 03:34:54.740776 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729423    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 03:34:54.740776 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729425    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 03:34:54.740776 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729428    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 03:34:54.740776 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729431    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 03:34:54.740776 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729434    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 03:34:54.740776 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729437    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 03:34:54.740776 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729440    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 03:34:54.740776 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729443    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 03:34:54.740776 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729446    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 03:34:54.740776 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729449    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 03:34:54.740776 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729452    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 03:34:54.740776 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729454    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 03:34:54.740776 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729458    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 03:34:54.741333 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729461    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 03:34:54.741333 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729463    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 03:34:54.741333 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729466    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 03:34:54.741333 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.729469    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 03:34:54.741333 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.730183    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 03:34:54.741333 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.738572    2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 30 03:34:54.741333 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.738592    2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 30 03:34:54.741333 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.738616    2576 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 30 03:34:54.741333 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.738623    2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 30 03:34:54.741333 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738698    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 03:34:54.741333 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738703    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 03:34:54.741333 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738707    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 03:34:54.741333 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738710    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 03:34:54.741333 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738714    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 03:34:54.741333 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738717    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 03:34:54.741785 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738721    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 03:34:54.741785 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738724    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 03:34:54.741785 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738727    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 03:34:54.741785 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738731    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 03:34:54.741785 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738733    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 03:34:54.741785 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738736    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 03:34:54.741785 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738740    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 03:34:54.741785 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738743    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 03:34:54.741785 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738747    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 03:34:54.741785 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738750    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 03:34:54.741785 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738753    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 03:34:54.741785 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738757    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 03:34:54.741785 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738759    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 03:34:54.741785 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738769    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 03:34:54.741785 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738772    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 03:34:54.741785 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738775    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 03:34:54.741785 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738778    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 03:34:54.741785 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738781    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 03:34:54.741785 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738784    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 03:34:54.741785 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738786    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 03:34:54.742393 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738789    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 03:34:54.742393 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738792    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 03:34:54.742393 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738795    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 03:34:54.742393 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738799    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 03:34:54.742393 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738802    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 03:34:54.742393 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738804    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 03:34:54.742393 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738808    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 03:34:54.742393 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738810    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 03:34:54.742393 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738813    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 03:34:54.742393 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738816    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 03:34:54.742393 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738819    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 03:34:54.742393 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738823    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 03:34:54.742393 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738826    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 03:34:54.742393 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738828    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 03:34:54.742393 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738831    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 03:34:54.742393 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738834    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 03:34:54.742393 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738837    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 03:34:54.742393 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738840    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 03:34:54.742393 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738842    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 03:34:54.742393 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738845    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 03:34:54.742992 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738848    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 03:34:54.742992 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738850    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 03:34:54.742992 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738853    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 03:34:54.742992 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738856    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 03:34:54.742992 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738859    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 03:34:54.742992 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738862    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 03:34:54.742992 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738870    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 03:34:54.742992 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738873    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 03:34:54.742992 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738876    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 03:34:54.742992 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738878    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 03:34:54.742992 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738881    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 03:34:54.742992 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738884    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 03:34:54.742992 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738889    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 03:34:54.742992 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738894    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 03:34:54.742992 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738897    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 03:34:54.742992 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738900    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 03:34:54.742992 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738904    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 03:34:54.742992 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738907    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 03:34:54.742992 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738909    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 03:34:54.743510 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738912    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 03:34:54.743510 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738915    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 03:34:54.743510 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738918    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 03:34:54.743510 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738920    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 03:34:54.743510 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738924    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 03:34:54.743510 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738926    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 03:34:54.743510 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738929    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 03:34:54.743510 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738932    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 03:34:54.743510 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738935    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 03:34:54.743510 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738938    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 03:34:54.743510 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738941    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 03:34:54.743510 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738943    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 03:34:54.743510 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738946    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 03:34:54.743510 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738949    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 03:34:54.743510 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738952    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 03:34:54.743510 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738955    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 03:34:54.743510 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738959    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 03:34:54.743510 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738964    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 03:34:54.743510 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738967    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 03:34:54.744197 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738971    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 03:34:54.744197 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738980    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 03:34:54.744197 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738984    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 03:34:54.744197 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738987    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 03:34:54.744197 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738990    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 03:34:54.744197 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.738994    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 03:34:54.744197 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739011    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 03:34:54.744197 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739015    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 03:34:54.744197 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739018    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 03:34:54.744197 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739022    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 03:34:54.744197 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739025    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 03:34:54.744197 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739028    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 03:34:54.744197 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739031    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 03:34:54.744197 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739034    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 03:34:54.744197 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739037    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 03:34:54.744630 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.739043    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 03:34:54.744630 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739269    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 03:34:54.744630 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739275    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 03:34:54.744630 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739279    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 03:34:54.744630 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739282    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 03:34:54.744630 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739285    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 03:34:54.744630 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739288    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 03:34:54.744630 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739291    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 03:34:54.744630 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739294    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 03:34:54.744630 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739299    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 03:34:54.744630 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739303    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 03:34:54.744630 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739307    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 03:34:54.744630 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739310    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 03:34:54.744630 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739313    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 03:34:54.744630 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739316    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 03:34:54.745128 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739318    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 03:34:54.745128 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739321    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 03:34:54.745128 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739324    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 03:34:54.745128 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739327    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 03:34:54.745128 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739339    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 03:34:54.745128 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739342    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 03:34:54.745128 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739345    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 03:34:54.745128 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739348    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 03:34:54.745128 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739350    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 03:34:54.745128 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739353    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 03:34:54.745128 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739356    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 03:34:54.745128 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739359    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 03:34:54.745128 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739361    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 03:34:54.745128 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739364    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 03:34:54.745128 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739367    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 03:34:54.745128 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739369    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 03:34:54.745128 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739372    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 03:34:54.745128 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739375    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 03:34:54.745128 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739378    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 03:34:54.745128 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739380    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 03:34:54.745666 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739383    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 03:34:54.745666 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739386    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 03:34:54.745666 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739388    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 03:34:54.745666 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739391    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 03:34:54.745666 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739394    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 03:34:54.745666 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739396    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 03:34:54.745666 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739399    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 03:34:54.745666 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739401    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 03:34:54.745666 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739404    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 03:34:54.745666 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739407    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 03:34:54.745666 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739410    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 03:34:54.745666 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739412    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 03:34:54.745666 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739415    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 03:34:54.745666 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739418    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 03:34:54.745666 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739420    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 03:34:54.745666 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739424    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 03:34:54.745666 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739427    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 03:34:54.745666 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739436    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 03:34:54.745666 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739439    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 03:34:54.745666 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739442    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 03:34:54.746344 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739445    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 03:34:54.746344 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739448    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 03:34:54.746344 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739451    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 03:34:54.746344 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739453    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 03:34:54.746344 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739456    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 03:34:54.746344 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739459    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 03:34:54.746344 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739463    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 03:34:54.746344 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739466    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 03:34:54.746344 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739469    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 03:34:54.746344 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739474    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 03:34:54.746344 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739477    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 03:34:54.746344 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739480    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 03:34:54.746344 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739483    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 03:34:54.746344 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739486    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 03:34:54.746344 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739489    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 03:34:54.746344 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739492    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 03:34:54.746344 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739494    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 03:34:54.746344 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739497    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 03:34:54.746344 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739499    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 03:34:54.747125 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739502    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 03:34:54.747125 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739505    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 03:34:54.747125 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739508    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 03:34:54.747125 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739510    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 03:34:54.747125 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739513    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 03:34:54.747125 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739516    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 03:34:54.747125 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739518    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 03:34:54.747125 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739521    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 03:34:54.747125 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739525    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 03:34:54.747125 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739528    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 03:34:54.747125 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739530    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 03:34:54.747125 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739541    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 03:34:54.747125 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739544    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 03:34:54.747125 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739547    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 03:34:54.747125 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739550    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 03:34:54.747125 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739552    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 03:34:54.747125 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739555    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 03:34:54.747125 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739557    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 03:34:54.747125 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739560    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 03:34:54.747125 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739563    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 03:34:54.748102 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739566    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 03:34:54.748102 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739569    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 03:34:54.748102 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739572    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 03:34:54.748102 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739575    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 03:34:54.748102 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739578    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 03:34:54.748102 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:54.739581    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 03:34:54.748102 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.739586    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 03:34:54.748102 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.739753    2576 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 30 03:34:54.748102 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.742127    2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 30 03:34:54.748102 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.743493    2576 server.go:1019] "Starting client certificate rotation"
Jul 30 03:34:54.748102 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.743632    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 30 03:34:54.748102 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.744622    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 30 03:34:54.748102 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.747253    2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 30 03:34:54.754841 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.754811    2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 30 03:34:54.765881 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.765849    2576 log.go:25] "Validated CRI v1 runtime API"
Jul 30 03:34:54.769435 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.769413    2576 log.go:25] "Validated CRI v1 image API"
Jul 30 03:34:54.771310 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.771280    2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 30 03:34:54.776092 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.776065    2576 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 c1ff7edb-ca81-4978-9798-28734e3dd1d6:/dev/nvme0n1p3 c980ea6b-d28f-49b0-bc94-394ecf5c5b44:/dev/nvme0n1p4]
Jul 30 03:34:54.776208 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.776089    2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 30 03:34:54.778428 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.778404    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 30 03:34:54.781873 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.781740    2576 manager.go:216] Machine: {Timestamp:2026-07-30 03:34:54.781201824 +0000 UTC m=+0.399434059 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3093194 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20304aea702af856fa796fcb87bd83 SystemUUID:ec20304a-ea70-2af8-56fa-796fcb87bd83 BootID:ca05be84-2e85-4df8-ae39-d5f55d35f800 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:03:3e:4a:40:89 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:03:3e:4a:40:89 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:62:bf:dc:3d:34:0b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 30 03:34:54.781873 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.781856    2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 30 03:34:54.782089 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.782071    2576 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 30 03:34:54.783426 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.783372    2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 30 03:34:54.783593 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.783430    2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-117.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 30 03:34:54.784326 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.784314    2576 topology_manager.go:138] "Creating topology manager with none policy"
Jul 30 03:34:54.784362 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.784329    2576 container_manager_linux.go:309] "Creating device plugin manager"
Jul 30 03:34:54.784362 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.784340    2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 30 03:34:54.786182 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.786167    2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 30 03:34:54.787505 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.787491    2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 30 03:34:54.790793 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.790774    2576 state_mem.go:36] "Initialized new in-memory state store"
Jul 30 03:34:54.790944 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.790932    2576 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 30 03:34:54.791013 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.790973    2576 kubelet.go:486] "Attempting to sync node with API server"
Jul 30 03:34:54.791013 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.790986    2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 30 03:34:54.791013 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.791011    2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 30 03:34:54.791147 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.791021    2576 kubelet.go:398] "Adding apiserver pod source"
Jul 30 03:34:54.791147 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.791032    2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 30 03:34:54.791452 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.791438    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 30 03:34:54.791493 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.791466    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 30 03:34:54.794045 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.794025    2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 30 03:34:54.795635 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.795617    2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 30 03:34:54.795686 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.795654    2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 30 03:34:54.796863 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.796850    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 30 03:34:54.796906 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.796869    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 30 03:34:54.796906 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.796876    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 30 03:34:54.796906 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.796882    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 30 03:34:54.796906 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.796891    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 30 03:34:54.796906 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.796897    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 30 03:34:54.797157 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.796909    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 30 03:34:54.797157 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.796916    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 30 03:34:54.797157 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.796926    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 30 03:34:54.797157 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.796935    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 30 03:34:54.797157 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.796949    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 30 03:34:54.797157 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.796955    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 30 03:34:54.797157 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.796983    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 30 03:34:54.797157 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.796989    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 30 03:34:54.801027 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.800991    2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 30 03:34:54.801027 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.801021    2576 server.go:1268] "Started kubelet"
Jul 30 03:34:54.801210 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:54.801074    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 30 03:34:54.801210 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:54.801099    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 30 03:34:54.801210 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.801135    2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 30 03:34:54.801376 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.801146    2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 30 03:34:54.801376 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.801336    2576 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 30 03:34:54.803949 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.803924    2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 30 03:34:54.804223 ip-10-0-143-117 systemd[1]: Started Kubernetes Kubelet.
Jul 30 03:34:54.807130 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.806408    2576 server.go:310] "Adding debug handlers to kubelet server"
Jul 30 03:34:54.809190 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:54.809168    2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 30 03:34:54.812106 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:54.810628    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-117.ec2.internal.18c6f3eb6ed00f6d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-117.ec2.internal,UID:ip-10-0-143-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-117.ec2.internal,},FirstTimestamp:2026-07-30 03:34:54.800973677 +0000 UTC m=+0.419205911,LastTimestamp:2026-07-30 03:34:54.800973677 +0000 UTC m=+0.419205911,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-117.ec2.internal,}"
Jul 30 03:34:54.813772 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.813749    2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 30 03:34:54.813772 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.813758    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 30 03:34:54.813926 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:54.813825    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-117.ec2.internal\" not found"
Jul 30 03:34:54.814830 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.814466    2576 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 30 03:34:54.814830 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.814488    2576 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 30 03:34:54.814830 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.814511    2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 30 03:34:54.814830 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.814647    2576 reconstruct.go:97] "Volume reconstruction finished"
Jul 30 03:34:54.814830 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.814659    2576 reconciler.go:29] "Reconciler: start to sync state"
Jul 30 03:34:54.814830 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.814753    2576 factory.go:55] Registering systemd factory
Jul 30 03:34:54.814830 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.814771    2576 factory.go:223] Registration of the systemd container factory successfully
Jul 30 03:34:54.815430 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.815295    2576 factory.go:153] Registering CRI-O factory
Jul 30 03:34:54.815430 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.815312    2576 factory.go:223] Registration of the crio container factory successfully
Jul 30 03:34:54.815430 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.815376    2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 30 03:34:54.815430 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.815406    2576 factory.go:103] Registering Raw factory
Jul 30 03:34:54.815430 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.815421    2576 manager.go:1196] Started watching for new ooms in manager
Jul 30 03:34:54.815962 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.815929    2576 manager.go:349] Starting recovery of all containers
Jul 30 03:34:54.823311 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:54.823270    2576 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 30 03:34:54.823476 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:54.823423    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 30 03:34:54.827290 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.827268    2576 manager.go:354] Recovery completed
Jul 30 03:34:54.832185 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.832170    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 03:34:54.834354 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.834331    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 03:34:54.834463 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.834376    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 03:34:54.834463 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.834394    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasSufficientPID"
Jul 30 03:34:54.835039 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.835020    2576 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 30 03:34:54.835039 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.835037    2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 30 03:34:54.835145 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.835054    2576 state_mem.go:36] "Initialized new in-memory state store"
Jul 30 03:34:54.835145 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.835080    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 03:34:54.835834 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.835819    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 03:34:54.835898 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.835848    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 03:34:54.835898 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.835859    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasSufficientPID"
Jul 30 03:34:54.836283 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.836268    2576 policy_none.go:49] "None policy: Start"
Jul 30 03:34:54.836319 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.836288    2576 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 30 03:34:54.836319 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.836300    2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 30 03:34:54.837321 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:54.837222    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-117.ec2.internal.18c6f3eb70cd6e60  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-117.ec2.internal,UID:ip-10-0-143-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-117.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-117.ec2.internal,},FirstTimestamp:2026-07-30 03:34:54.834355808 +0000 UTC m=+0.452588051,LastTimestamp:2026-07-30 03:34:54.834355808 +0000 UTC m=+0.452588051,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-117.ec2.internal,}"
Jul 30 03:34:54.837426 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.837323    2576 policy_none.go:47] "Start"
Jul 30 03:34:54.847096 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:54.846968    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-117.ec2.internal.18c6f3eb70cddaae  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-117.ec2.internal,UID:ip-10-0-143-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-117.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-117.ec2.internal,},FirstTimestamp:2026-07-30 03:34:54.834383534 +0000 UTC m=+0.452615770,LastTimestamp:2026-07-30 03:34:54.834383534 +0000 UTC m=+0.452615770,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-117.ec2.internal,}"
Jul 30 03:34:54.849392 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:54.849299    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-117.ec2.internal.18c6f3eb70ce1a6b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-117.ec2.internal,UID:ip-10-0-143-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-143-117.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-143-117.ec2.internal,},FirstTimestamp:2026-07-30 03:34:54.834399851 +0000 UTC m=+0.452632089,LastTimestamp:2026-07-30 03:34:54.834399851 +0000 UTC m=+0.452632089,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-117.ec2.internal,}"
Jul 30 03:34:54.859567 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:54.859478    2576 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-117.ec2.internal.18c6f3eb70cd6e60\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-117.ec2.internal.18c6f3eb70cd6e60  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-117.ec2.internal,UID:ip-10-0-143-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-117.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-117.ec2.internal,},FirstTimestamp:2026-07-30 03:34:54.834355808 +0000 UTC m=+0.452588051,LastTimestamp:2026-07-30 03:34:54.8358345 +0000 UTC m=+0.454066734,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-117.ec2.internal,}"
Jul 30 03:34:54.866075 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:54.865978    2576 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-117.ec2.internal.18c6f3eb70cddaae\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-117.ec2.internal.18c6f3eb70cddaae  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-117.ec2.internal,UID:ip-10-0-143-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-117.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-117.ec2.internal,},FirstTimestamp:2026-07-30 03:34:54.834383534 +0000 UTC m=+0.452615770,LastTimestamp:2026-07-30 03:34:54.835853627 +0000 UTC m=+0.454085861,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-117.ec2.internal,}"
Jul 30 03:34:54.876043 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.876020    2576 manager.go:337] "Starting Device Plugin manager"
Jul 30 03:34:54.876152 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:54.875983    2576 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-117.ec2.internal.18c6f3eb70ce1a6b\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-117.ec2.internal.18c6f3eb70ce1a6b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-117.ec2.internal,UID:ip-10-0-143-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-143-117.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-143-117.ec2.internal,},FirstTimestamp:2026-07-30 03:34:54.834399851 +0000 UTC m=+0.452632089,LastTimestamp:2026-07-30 03:34:54.835863185 +0000 UTC m=+0.454095420,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-117.ec2.internal,}"
Jul 30 03:34:54.876152 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:54.876069    2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 30 03:34:54.876152 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.876084    2576 server.go:87] "Starting device plugin registration server"
Jul 30 03:34:54.876416 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.876402    2576 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 30 03:34:54.876476 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.876419    2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 30 03:34:54.876523 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.876496    2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 30 03:34:54.876597 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.876581    2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 30 03:34:54.876597 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.876596    2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 30 03:34:54.877240 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:54.877212    2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 30 03:34:54.877354 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:54.877263    2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-117.ec2.internal\" not found"
Jul 30 03:34:54.898689 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:54.898570    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-117.ec2.internal.18c6f3eb73fa3d4c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-117.ec2.internal,UID:ip-10-0-143-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-143-117.ec2.internal,},FirstTimestamp:2026-07-30 03:34:54.887624012 +0000 UTC m=+0.505856236,LastTimestamp:2026-07-30 03:34:54.887624012 +0000 UTC m=+0.505856236,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-117.ec2.internal,}"
Jul 30 03:34:54.910264 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.910235    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bck56"
Jul 30 03:34:54.918388 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.918363    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bck56"
Jul 30 03:34:54.978902 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.978863    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 03:34:54.980339 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.979904    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 03:34:54.980339 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.979961    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 03:34:54.980339 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.979979    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasSufficientPID"
Jul 30 03:34:54.980339 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.980013    2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-117.ec2.internal"
Jul 30 03:34:54.996197 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:54.996110    2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-117.ec2.internal"
Jul 30 03:34:54.996197 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:54.996147    2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-117.ec2.internal\": node \"ip-10-0-143-117.ec2.internal\" not found"
Jul 30 03:34:55.038054 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:55.038012    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-117.ec2.internal\" not found"
Jul 30 03:34:55.043381 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.043345    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 30 03:34:55.044687 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.044668    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 30 03:34:55.044756 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.044711    2576 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 30 03:34:55.044756 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.044734    2576 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 30 03:34:55.044856 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:55.044771    2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 30 03:34:55.046576 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.046553    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 03:34:55.138912 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:55.138836    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-117.ec2.internal\" not found"
Jul 30 03:34:55.145176 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.145139    2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-117.ec2.internal"]
Jul 30 03:34:55.145295 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.145244    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 03:34:55.147041 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.147021    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 03:34:55.147149 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.147054    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 03:34:55.147149 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.147065    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasSufficientPID"
Jul 30 03:34:55.148661 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.148647    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 03:34:55.148771 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.148756    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal"
Jul 30 03:34:55.148827 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.148790    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 03:34:55.149478 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.149460    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 03:34:55.149596 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.149494    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 03:34:55.149596 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.149510    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasSufficientPID"
Jul 30 03:34:55.149596 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.149460    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 03:34:55.149596 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.149573    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 03:34:55.149596 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.149585    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasSufficientPID"
Jul 30 03:34:55.150905 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.150887    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-117.ec2.internal"
Jul 30 03:34:55.151010 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.150913    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 03:34:55.151650 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.151634    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 03:34:55.151741 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.151666    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 03:34:55.151741 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.151682    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasSufficientPID"
Jul 30 03:34:55.180231 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:55.180200    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-117.ec2.internal\" not found" node="ip-10-0-143-117.ec2.internal"
Jul 30 03:34:55.185356 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:55.185332    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-117.ec2.internal\" not found" node="ip-10-0-143-117.ec2.internal"
Jul 30 03:34:55.217947 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.217896    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/050b224397a0390fc072543beebab8c7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal\" (UID: \"050b224397a0390fc072543beebab8c7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal"
Jul 30 03:34:55.217947 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.217951    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/78f929edfbd95057c37ce421449a9742-config\") pod \"kube-apiserver-proxy-ip-10-0-143-117.ec2.internal\" (UID: \"78f929edfbd95057c37ce421449a9742\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-117.ec2.internal"
Jul 30 03:34:55.218179 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.217983    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/050b224397a0390fc072543beebab8c7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal\" (UID: \"050b224397a0390fc072543beebab8c7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal"
Jul 30 03:34:55.239353 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:55.239302    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-117.ec2.internal\" not found"
Jul 30 03:34:55.319142 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.319007    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/050b224397a0390fc072543beebab8c7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal\" (UID: \"050b224397a0390fc072543beebab8c7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal"
Jul 30 03:34:55.319142 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.319127    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/050b224397a0390fc072543beebab8c7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal\" (UID: \"050b224397a0390fc072543beebab8c7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal"
Jul 30 03:34:55.319317 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.319145    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/78f929edfbd95057c37ce421449a9742-config\") pod \"kube-apiserver-proxy-ip-10-0-143-117.ec2.internal\" (UID: \"78f929edfbd95057c37ce421449a9742\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-117.ec2.internal"
Jul 30 03:34:55.319317 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.319074    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/050b224397a0390fc072543beebab8c7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal\" (UID: \"050b224397a0390fc072543beebab8c7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal"
Jul 30 03:34:55.319317 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.319186    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/78f929edfbd95057c37ce421449a9742-config\") pod \"kube-apiserver-proxy-ip-10-0-143-117.ec2.internal\" (UID: \"78f929edfbd95057c37ce421449a9742\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-117.ec2.internal"
Jul 30 03:34:55.319317 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.319216    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/050b224397a0390fc072543beebab8c7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal\" (UID: \"050b224397a0390fc072543beebab8c7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal"
Jul 30 03:34:55.340297 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:55.340249    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-117.ec2.internal\" not found"
Jul 30 03:34:55.441428 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:55.441378    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-117.ec2.internal\" not found"
Jul 30 03:34:55.484907 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.484852    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal"
Jul 30 03:34:55.488322 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.488306    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-117.ec2.internal"
Jul 30 03:34:55.542445 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:55.542406    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-117.ec2.internal\" not found"
Jul 30 03:34:55.643606 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:55.643519    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-117.ec2.internal\" not found"
Jul 30 03:34:55.743292 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.743228    2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 30 03:34:55.743797 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.743443    2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 30 03:34:55.744403 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:55.744381    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-117.ec2.internal\" not found"
Jul 30 03:34:55.812084 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.812053    2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 03:34:55.814018 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.813988    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal"
Jul 30 03:34:55.814144 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.814024    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 30 03:34:55.826128 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.826101    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 30 03:34:55.828029 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.828014    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-117.ec2.internal"
Jul 30 03:34:55.828537 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.828520    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 30 03:34:55.837322 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.837298    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 30 03:34:55.850238 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.850209    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-dh9mx"
Jul 30 03:34:55.858327 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.858305    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-dh9mx"
Jul 30 03:34:55.923038 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.922922    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-29 03:29:55 +0000 UTC" deadline="2028-04-13 06:29:39.289414442 +0000 UTC"
Jul 30 03:34:55.923038 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:55.922985    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14954h54m43.366433183s"
Jul 30 03:34:56.049717 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.049506    2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 03:34:56.086445 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.086412    2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 03:34:56.106576 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:56.106528    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78f929edfbd95057c37ce421449a9742.slice/crio-8ed62c63c7bcd9ee92dab8bbf469431e678f8d075028fef8ba16b3f194b1f4f1 WatchSource:0}: Error finding container 8ed62c63c7bcd9ee92dab8bbf469431e678f8d075028fef8ba16b3f194b1f4f1: Status 404 returned error can't find the container with id 8ed62c63c7bcd9ee92dab8bbf469431e678f8d075028fef8ba16b3f194b1f4f1
Jul 30 03:34:56.106928 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:56.106907    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod050b224397a0390fc072543beebab8c7.slice/crio-7cad6e65b4079c6d91fe708cb8912b55a47b16133e115e0453c7703909e253b8 WatchSource:0}: Error finding container 7cad6e65b4079c6d91fe708cb8912b55a47b16133e115e0453c7703909e253b8: Status 404 returned error can't find the container with id 7cad6e65b4079c6d91fe708cb8912b55a47b16133e115e0453c7703909e253b8
Jul 30 03:34:56.111835 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.111815    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 03:34:56.567454 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.567415    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 03:34:56.793441 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.793403    2576 apiserver.go:52] "Watching apiserver"
Jul 30 03:34:56.800475 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.800440    2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 30 03:34:56.801072 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.801042    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-nhwmf","kube-system/global-pull-secret-syncer-pzlrv","kube-system/kube-apiserver-proxy-ip-10-0-143-117.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7bhs","openshift-cluster-node-tuning-operator/tuned-fszmc","openshift-image-registry/node-ca-tl4qv","openshift-multus/multus-additional-cni-plugins-7n5z9","openshift-multus/multus-th2nr","openshift-multus/network-metrics-daemon-rqrtn","kube-system/konnectivity-agent-9jdph","openshift-dns/node-resolver-mfgtq","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal","openshift-network-diagnostics/network-check-target-fzz5w","openshift-network-operator/iptables-alerter-mmh5j"]
Jul 30 03:34:56.802959 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.802855    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7n5z9"
Jul 30 03:34:56.804330 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.804090    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fzz5w"
Jul 30 03:34:56.804330 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:56.804179    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fzz5w" podUID="c8447d93-b20e-44aa-a11e-3c1aec4d0ddd"
Jul 30 03:34:56.805683 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.805564    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mmh5j"
Jul 30 03:34:56.806370 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.806351    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 30 03:34:56.806676 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.806656    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 30 03:34:56.806821 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.806794    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 30 03:34:56.806895 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.806853    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 30 03:34:56.806974 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.806923    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-5pq5v\""
Jul 30 03:34:56.807115 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.807097    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 30 03:34:56.807258 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.807235    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7bhs"
Jul 30 03:34:56.808611 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.808564    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 30 03:34:56.808742 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.808678    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 30 03:34:56.809740 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.809717    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 30 03:34:56.810101 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.810077    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wt5kt\""
Jul 30 03:34:56.810609 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.810587    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 30 03:34:56.810839 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.810818    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-zllnd\""
Jul 30 03:34:56.811210 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.811194    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 30 03:34:56.811419 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.811397    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 30 03:34:56.812540 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.812517    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.814723 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.814489    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tl4qv"
Jul 30 03:34:56.816022 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.815995    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqrtn"
Jul 30 03:34:56.816135 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:56.816087    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqrtn" podUID="890cb2cc-a9ca-4b13-bef5-f2282e486612"
Jul 30 03:34:56.817488 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.817227    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 30 03:34:56.818226 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.818191    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 30 03:34:56.818438 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.818418    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-th2nr"
Jul 30 03:34:56.818633 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.818615    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 30 03:34:56.818709 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.818665    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 30 03:34:56.818908 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.818886    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-rkfsc\""
Jul 30 03:34:56.821281 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.821255    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-8dk44\""
Jul 30 03:34:56.821476 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.821458    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 30 03:34:56.822867 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.822818    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mfgtq"
Jul 30 03:34:56.824499 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.824471    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-knrzr\""
Jul 30 03:34:56.824796 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.824777    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 30 03:34:56.825288 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.825262    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9jdph"
Jul 30 03:34:56.825924 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.825903    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-nl4nf\""
Jul 30 03:34:56.826144 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.826121    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 30 03:34:56.826373 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.826360    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 30 03:34:56.827264 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.827240    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/04cb8a74-98ed-4fe2-9c4f-189783dce4da-cnibin\") pod \"multus-additional-cni-plugins-7n5z9\" (UID: \"04cb8a74-98ed-4fe2-9c4f-189783dce4da\") " pod="openshift-multus/multus-additional-cni-plugins-7n5z9"
Jul 30 03:34:56.827376 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.827274    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b90028ac-a40a-4d9c-816a-8fec579a4b93-host\") pod \"node-ca-tl4qv\" (UID: \"b90028ac-a40a-4d9c-816a-8fec579a4b93\") " pod="openshift-image-registry/node-ca-tl4qv"
Jul 30 03:34:56.827376 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.827324    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v86vs\" (UniqueName: \"kubernetes.io/projected/b90028ac-a40a-4d9c-816a-8fec579a4b93-kube-api-access-v86vs\") pod \"node-ca-tl4qv\" (UID: \"b90028ac-a40a-4d9c-816a-8fec579a4b93\") " pod="openshift-image-registry/node-ca-tl4qv"
Jul 30 03:34:56.827376 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.827356    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4fc32317-f442-41da-8068-42fd29b51d26-iptables-alerter-script\") pod \"iptables-alerter-mmh5j\" (UID: \"4fc32317-f442-41da-8068-42fd29b51d26\") " pod="openshift-network-operator/iptables-alerter-mmh5j"
Jul 30 03:34:56.827530 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.827386    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4fc32317-f442-41da-8068-42fd29b51d26-host-slash\") pod \"iptables-alerter-mmh5j\" (UID: \"4fc32317-f442-41da-8068-42fd29b51d26\") " pod="openshift-network-operator/iptables-alerter-mmh5j"
Jul 30 03:34:56.827530 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.827411    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/04cb8a74-98ed-4fe2-9c4f-189783dce4da-os-release\") pod \"multus-additional-cni-plugins-7n5z9\" (UID: \"04cb8a74-98ed-4fe2-9c4f-189783dce4da\") " pod="openshift-multus/multus-additional-cni-plugins-7n5z9"
Jul 30 03:34:56.827530 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.827451    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pz6fq\" (UniqueName: \"kubernetes.io/projected/c8447d93-b20e-44aa-a11e-3c1aec4d0ddd-kube-api-access-pz6fq\") pod \"network-check-target-fzz5w\" (UID: \"c8447d93-b20e-44aa-a11e-3c1aec4d0ddd\") " pod="openshift-network-diagnostics/network-check-target-fzz5w"
Jul 30 03:34:56.827530 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.827476    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2ce5d0b8-bd3d-458a-a201-c2633e1af196-sys-fs\") pod \"aws-ebs-csi-driver-node-h7bhs\" (UID: \"2ce5d0b8-bd3d-458a-a201-c2633e1af196\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7bhs"
Jul 30 03:34:56.827661 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.827531    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-etc-modprobe-d\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.827661 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.827556    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-etc-kubernetes\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.827661 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.827581    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-etc-tuned\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.827661 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.827645    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/04cb8a74-98ed-4fe2-9c4f-189783dce4da-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7n5z9\" (UID: \"04cb8a74-98ed-4fe2-9c4f-189783dce4da\") " pod="openshift-multus/multus-additional-cni-plugins-7n5z9"
Jul 30 03:34:56.827924 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.827735    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/04cb8a74-98ed-4fe2-9c4f-189783dce4da-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7n5z9\" (UID: \"04cb8a74-98ed-4fe2-9c4f-189783dce4da\") " pod="openshift-multus/multus-additional-cni-plugins-7n5z9"
Jul 30 03:34:56.827924 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.827784    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2ce5d0b8-bd3d-458a-a201-c2633e1af196-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h7bhs\" (UID: \"2ce5d0b8-bd3d-458a-a201-c2633e1af196\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7bhs"
Jul 30 03:34:56.827924 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.827814    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-etc-sysctl-conf\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.827924 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.827827    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 30 03:34:56.827924 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.827842    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-run\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.827924 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.827879    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-lib-modules\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.827924 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.827910    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2ce5d0b8-bd3d-458a-a201-c2633e1af196-socket-dir\") pod \"aws-ebs-csi-driver-node-h7bhs\" (UID: \"2ce5d0b8-bd3d-458a-a201-c2633e1af196\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7bhs"
Jul 30 03:34:56.828292 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.827931    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-vx6rm\""
Jul 30 03:34:56.828292 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.827937    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvrr6\" (UniqueName: \"kubernetes.io/projected/890cb2cc-a9ca-4b13-bef5-f2282e486612-kube-api-access-vvrr6\") pod \"network-metrics-daemon-rqrtn\" (UID: \"890cb2cc-a9ca-4b13-bef5-f2282e486612\") " pod="openshift-multus/network-metrics-daemon-rqrtn"
Jul 30 03:34:56.828292 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.827994    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z7tz\" (UniqueName: \"kubernetes.io/projected/04cb8a74-98ed-4fe2-9c4f-189783dce4da-kube-api-access-7z7tz\") pod \"multus-additional-cni-plugins-7n5z9\" (UID: \"04cb8a74-98ed-4fe2-9c4f-189783dce4da\") " pod="openshift-multus/multus-additional-cni-plugins-7n5z9"
Jul 30 03:34:56.828292 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.828080    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/04cb8a74-98ed-4fe2-9c4f-189783dce4da-system-cni-dir\") pod \"multus-additional-cni-plugins-7n5z9\" (UID: \"04cb8a74-98ed-4fe2-9c4f-189783dce4da\") " pod="openshift-multus/multus-additional-cni-plugins-7n5z9"
Jul 30 03:34:56.828292 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.828106    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/04cb8a74-98ed-4fe2-9c4f-189783dce4da-cni-binary-copy\") pod \"multus-additional-cni-plugins-7n5z9\" (UID: \"04cb8a74-98ed-4fe2-9c4f-189783dce4da\") " pod="openshift-multus/multus-additional-cni-plugins-7n5z9"
Jul 30 03:34:56.828292 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.828131    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9rkg\" (UniqueName: \"kubernetes.io/projected/2ce5d0b8-bd3d-458a-a201-c2633e1af196-kube-api-access-z9rkg\") pod \"aws-ebs-csi-driver-node-h7bhs\" (UID: \"2ce5d0b8-bd3d-458a-a201-c2633e1af196\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7bhs"
Jul 30 03:34:56.828292 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.828170    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 30 03:34:56.828292 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.828191    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-tmp\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.828292 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.828227    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkwb6\" (UniqueName: \"kubernetes.io/projected/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-kube-api-access-vkwb6\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.828292 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.828272    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5t44\" (UniqueName: \"kubernetes.io/projected/4fc32317-f442-41da-8068-42fd29b51d26-kube-api-access-j5t44\") pod \"iptables-alerter-mmh5j\" (UID: \"4fc32317-f442-41da-8068-42fd29b51d26\") " pod="openshift-network-operator/iptables-alerter-mmh5j"
Jul 30 03:34:56.828795 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.828304    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2ce5d0b8-bd3d-458a-a201-c2633e1af196-registration-dir\") pod \"aws-ebs-csi-driver-node-h7bhs\" (UID: \"2ce5d0b8-bd3d-458a-a201-c2633e1af196\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7bhs"
Jul 30 03:34:56.828795 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.828326    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2ce5d0b8-bd3d-458a-a201-c2633e1af196-etc-selinux\") pod \"aws-ebs-csi-driver-node-h7bhs\" (UID: \"2ce5d0b8-bd3d-458a-a201-c2633e1af196\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7bhs"
Jul 30 03:34:56.828795 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.828390    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-etc-sysconfig\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.828795 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.828444    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-etc-sysctl-d\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.828795 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.828480    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-etc-systemd\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.828795 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.828502    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-sys\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.828795 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.828537    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/04cb8a74-98ed-4fe2-9c4f-189783dce4da-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7n5z9\" (UID: \"04cb8a74-98ed-4fe2-9c4f-189783dce4da\") " pod="openshift-multus/multus-additional-cni-plugins-7n5z9"
Jul 30 03:34:56.828795 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.828566    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2ce5d0b8-bd3d-458a-a201-c2633e1af196-device-dir\") pod \"aws-ebs-csi-driver-node-h7bhs\" (UID: \"2ce5d0b8-bd3d-458a-a201-c2633e1af196\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7bhs"
Jul 30 03:34:56.828795 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.828591    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-var-lib-kubelet\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.828795 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.828623    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-host\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.828795 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.828651    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b90028ac-a40a-4d9c-816a-8fec579a4b93-serviceca\") pod \"node-ca-tl4qv\" (UID: \"b90028ac-a40a-4d9c-816a-8fec579a4b93\") " pod="openshift-image-registry/node-ca-tl4qv"
Jul 30 03:34:56.828795 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.828685    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/890cb2cc-a9ca-4b13-bef5-f2282e486612-metrics-certs\") pod \"network-metrics-daemon-rqrtn\" (UID: \"890cb2cc-a9ca-4b13-bef5-f2282e486612\") " pod="openshift-multus/network-metrics-daemon-rqrtn"
Jul 30 03:34:56.829544 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.829309    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzlrv"
Jul 30 03:34:56.829544 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:56.829448    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzlrv" podUID="1c2622ca-8052-489f-8d23-999618145c5a"
Jul 30 03:34:56.831213 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.831190    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:56.833673 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.833651    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 30 03:34:56.834040 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.834018    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 30 03:34:56.834341 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.834321    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-spxkq\""
Jul 30 03:34:56.834431 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.834341    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 30 03:34:56.834807 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.834625    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 30 03:34:56.834807 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.834635    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 30 03:34:56.834807 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.834663    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 30 03:34:56.860323 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.860256    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 03:29:56 +0000 UTC" deadline="2028-05-12 22:10:14.263174328 +0000 UTC"
Jul 30 03:34:56.860323 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.860318    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15666h35m17.402861236s"
Jul 30 03:34:56.916003 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.915963    2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 30 03:34:56.929921 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.929887    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/62d78bc8-364a-44c0-ac71-f00f0a84623f-env-overrides\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:56.930133 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.929953    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvrr6\" (UniqueName: \"kubernetes.io/projected/890cb2cc-a9ca-4b13-bef5-f2282e486612-kube-api-access-vvrr6\") pod \"network-metrics-daemon-rqrtn\" (UID: \"890cb2cc-a9ca-4b13-bef5-f2282e486612\") " pod="openshift-multus/network-metrics-daemon-rqrtn"
Jul 30 03:34:56.930133 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.930053    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-os-release\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:56.930133 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.930093    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-host-var-lib-cni-bin\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:56.930133 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.930124    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-multus-conf-dir\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:56.930348 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.930153    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/62d78bc8-364a-44c0-ac71-f00f0a84623f-ovnkube-config\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:56.930348 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.930188    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/04cb8a74-98ed-4fe2-9c4f-189783dce4da-cni-binary-copy\") pod \"multus-additional-cni-plugins-7n5z9\" (UID: \"04cb8a74-98ed-4fe2-9c4f-189783dce4da\") " pod="openshift-multus/multus-additional-cni-plugins-7n5z9"
Jul 30 03:34:56.930348 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.930231    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9rkg\" (UniqueName: \"kubernetes.io/projected/2ce5d0b8-bd3d-458a-a201-c2633e1af196-kube-api-access-z9rkg\") pod \"aws-ebs-csi-driver-node-h7bhs\" (UID: \"2ce5d0b8-bd3d-458a-a201-c2633e1af196\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7bhs"
Jul 30 03:34:56.930348 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.930264    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-tmp\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.930348 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.930298    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49852e08-c471-4251-9b75-5e3e7fcc189b-tmp-dir\") pod \"node-resolver-mfgtq\" (UID: \"49852e08-c471-4251-9b75-5e3e7fcc189b\") " pod="openshift-dns/node-resolver-mfgtq"
Jul 30 03:34:56.930348 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.930326    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppx95\" (UniqueName: \"kubernetes.io/projected/62d78bc8-364a-44c0-ac71-f00f0a84623f-kube-api-access-ppx95\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:56.930681 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.930364    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2ce5d0b8-bd3d-458a-a201-c2633e1af196-etc-selinux\") pod \"aws-ebs-csi-driver-node-h7bhs\" (UID: \"2ce5d0b8-bd3d-458a-a201-c2633e1af196\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7bhs"
Jul 30 03:34:56.930681 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.930400    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-etc-sysconfig\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.930681 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.930443    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-etc-systemd\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.930681 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.930471    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-host\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.930681 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.930501    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-host-kubelet\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:56.930681 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.930533    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-var-lib-kubelet\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.930681 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.930566    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-host-run-k8s-cni-cncf-io\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:56.930681 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.930607    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2e09e50f-f869-4451-9504-a0909a29ae2a-multus-daemon-config\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:56.930681 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.930639    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-host-cni-bin\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:56.930681 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.930670    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v86vs\" (UniqueName: \"kubernetes.io/projected/b90028ac-a40a-4d9c-816a-8fec579a4b93-kube-api-access-v86vs\") pod \"node-ca-tl4qv\" (UID: \"b90028ac-a40a-4d9c-816a-8fec579a4b93\") " pod="openshift-image-registry/node-ca-tl4qv"
Jul 30 03:34:56.931253 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.930695    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-etc-sysconfig\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.931253 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.930707    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-hostroot\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:56.931253 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.930738    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/49852e08-c471-4251-9b75-5e3e7fcc189b-hosts-file\") pod \"node-resolver-mfgtq\" (UID: \"49852e08-c471-4251-9b75-5e3e7fcc189b\") " pod="openshift-dns/node-resolver-mfgtq"
Jul 30 03:34:56.931253 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.930819    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-etc-systemd\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.931253 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.930888    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2ce5d0b8-bd3d-458a-a201-c2633e1af196-etc-selinux\") pod \"aws-ebs-csi-driver-node-h7bhs\" (UID: \"2ce5d0b8-bd3d-458a-a201-c2633e1af196\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7bhs"
Jul 30 03:34:56.931253 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.930924    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-host-run-netns\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:56.931253 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.930979    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4fc32317-f442-41da-8068-42fd29b51d26-iptables-alerter-script\") pod \"iptables-alerter-mmh5j\" (UID: \"4fc32317-f442-41da-8068-42fd29b51d26\") " pod="openshift-network-operator/iptables-alerter-mmh5j"
Jul 30 03:34:56.931253 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.931029    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/04cb8a74-98ed-4fe2-9c4f-189783dce4da-os-release\") pod \"multus-additional-cni-plugins-7n5z9\" (UID: \"04cb8a74-98ed-4fe2-9c4f-189783dce4da\") " pod="openshift-multus/multus-additional-cni-plugins-7n5z9"
Jul 30 03:34:56.931253 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.931064    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz6fq\" (UniqueName: \"kubernetes.io/projected/c8447d93-b20e-44aa-a11e-3c1aec4d0ddd-kube-api-access-pz6fq\") pod \"network-check-target-fzz5w\" (UID: \"c8447d93-b20e-44aa-a11e-3c1aec4d0ddd\") " pod="openshift-network-diagnostics/network-check-target-fzz5w"
Jul 30 03:34:56.931253 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.931099    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1c2622ca-8052-489f-8d23-999618145c5a-dbus\") pod \"global-pull-secret-syncer-pzlrv\" (UID: \"1c2622ca-8052-489f-8d23-999618145c5a\") " pod="kube-system/global-pull-secret-syncer-pzlrv"
Jul 30 03:34:56.931253 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.931127    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/04cb8a74-98ed-4fe2-9c4f-189783dce4da-cni-binary-copy\") pod \"multus-additional-cni-plugins-7n5z9\" (UID: \"04cb8a74-98ed-4fe2-9c4f-189783dce4da\") " pod="openshift-multus/multus-additional-cni-plugins-7n5z9"
Jul 30 03:34:56.931253 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.931129    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-systemd-units\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:56.931253 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.931173    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-etc-openvswitch\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:56.931253 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.931176    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-host\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.932047 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.931646    2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 30 03:34:56.932047 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.931669    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-var-lib-kubelet\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.932047 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.931204    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/62d78bc8-364a-44c0-ac71-f00f0a84623f-ovnkube-script-lib\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:56.932047 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.931935    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2ce5d0b8-bd3d-458a-a201-c2633e1af196-sys-fs\") pod \"aws-ebs-csi-driver-node-h7bhs\" (UID: \"2ce5d0b8-bd3d-458a-a201-c2633e1af196\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7bhs"
Jul 30 03:34:56.932047 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.931967    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-etc-kubernetes\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.932047 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.931998    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-host-run-netns\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:56.932047 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.932038    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-etc-kubernetes\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:56.932390 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.932096    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwwhc\" (UniqueName: \"kubernetes.io/projected/2e09e50f-f869-4451-9504-a0909a29ae2a-kube-api-access-jwwhc\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:56.932390 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.932176    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/04cb8a74-98ed-4fe2-9c4f-189783dce4da-os-release\") pod \"multus-additional-cni-plugins-7n5z9\" (UID: \"04cb8a74-98ed-4fe2-9c4f-189783dce4da\") " pod="openshift-multus/multus-additional-cni-plugins-7n5z9"
Jul 30 03:34:56.932390 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.932197    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-etc-kubernetes\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.932390 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.932240    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4fc32317-f442-41da-8068-42fd29b51d26-iptables-alerter-script\") pod \"iptables-alerter-mmh5j\" (UID: \"4fc32317-f442-41da-8068-42fd29b51d26\") " pod="openshift-network-operator/iptables-alerter-mmh5j"
Jul 30 03:34:56.932390 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.932253    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2ce5d0b8-bd3d-458a-a201-c2633e1af196-sys-fs\") pod \"aws-ebs-csi-driver-node-h7bhs\" (UID: \"2ce5d0b8-bd3d-458a-a201-c2633e1af196\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7bhs"
Jul 30 03:34:56.932390 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.932259    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/04cb8a74-98ed-4fe2-9c4f-189783dce4da-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7n5z9\" (UID: \"04cb8a74-98ed-4fe2-9c4f-189783dce4da\") " pod="openshift-multus/multus-additional-cni-plugins-7n5z9"
Jul 30 03:34:56.932390 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.932326    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2ce5d0b8-bd3d-458a-a201-c2633e1af196-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h7bhs\" (UID: \"2ce5d0b8-bd3d-458a-a201-c2633e1af196\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7bhs"
Jul 30 03:34:56.932390 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.932357    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-etc-sysctl-conf\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.932793 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.932402    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-run\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.932793 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.932429    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-lib-modules\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.932793 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.932399    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2ce5d0b8-bd3d-458a-a201-c2633e1af196-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h7bhs\" (UID: \"2ce5d0b8-bd3d-458a-a201-c2633e1af196\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7bhs"
Jul 30 03:34:56.932793 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.932471    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58w4w\" (UniqueName: \"kubernetes.io/projected/49852e08-c471-4251-9b75-5e3e7fcc189b-kube-api-access-58w4w\") pod \"node-resolver-mfgtq\" (UID: \"49852e08-c471-4251-9b75-5e3e7fcc189b\") " pod="openshift-dns/node-resolver-mfgtq"
Jul 30 03:34:56.932793 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.932513    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-etc-sysctl-conf\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.932793 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.932519    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c50df0e7-6b07-47da-9c3c-d4328dcdcfa5-agent-certs\") pod \"konnectivity-agent-9jdph\" (UID: \"c50df0e7-6b07-47da-9c3c-d4328dcdcfa5\") " pod="kube-system/konnectivity-agent-9jdph"
Jul 30 03:34:56.932793 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.932535    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-run\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.932793 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.932553    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2ce5d0b8-bd3d-458a-a201-c2633e1af196-socket-dir\") pod \"aws-ebs-csi-driver-node-h7bhs\" (UID: \"2ce5d0b8-bd3d-458a-a201-c2633e1af196\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7bhs"
Jul 30 03:34:56.932793 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.932579    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2e09e50f-f869-4451-9504-a0909a29ae2a-cni-binary-copy\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:56.932793 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.932578    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-lib-modules\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.932793 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.932604    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-host-var-lib-kubelet\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:56.932793 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.932628    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1c2622ca-8052-489f-8d23-999618145c5a-kubelet-config\") pod \"global-pull-secret-syncer-pzlrv\" (UID: \"1c2622ca-8052-489f-8d23-999618145c5a\") " pod="kube-system/global-pull-secret-syncer-pzlrv"
Jul 30 03:34:56.932793 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.932653    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-host-slash\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:56.932793 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.932675    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-host-run-ovn-kubernetes\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:56.932793 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.932690    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2ce5d0b8-bd3d-458a-a201-c2633e1af196-socket-dir\") pod \"aws-ebs-csi-driver-node-h7bhs\" (UID: \"2ce5d0b8-bd3d-458a-a201-c2633e1af196\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7bhs"
Jul 30 03:34:56.932793 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.932690    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/04cb8a74-98ed-4fe2-9c4f-189783dce4da-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7n5z9\" (UID: \"04cb8a74-98ed-4fe2-9c4f-189783dce4da\") " pod="openshift-multus/multus-additional-cni-plugins-7n5z9"
Jul 30 03:34:56.932793 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.932701    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:56.933563 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.932728    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z7tz\" (UniqueName: \"kubernetes.io/projected/04cb8a74-98ed-4fe2-9c4f-189783dce4da-kube-api-access-7z7tz\") pod \"multus-additional-cni-plugins-7n5z9\" (UID: \"04cb8a74-98ed-4fe2-9c4f-189783dce4da\") " pod="openshift-multus/multus-additional-cni-plugins-7n5z9"
Jul 30 03:34:56.933563 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.932752    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/04cb8a74-98ed-4fe2-9c4f-189783dce4da-system-cni-dir\") pod \"multus-additional-cni-plugins-7n5z9\" (UID: \"04cb8a74-98ed-4fe2-9c4f-189783dce4da\") " pod="openshift-multus/multus-additional-cni-plugins-7n5z9"
Jul 30 03:34:56.933563 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.932775    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkwb6\" (UniqueName: \"kubernetes.io/projected/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-kube-api-access-vkwb6\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.933563 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.932813    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-multus-cni-dir\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:56.933563 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.932831    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/04cb8a74-98ed-4fe2-9c4f-189783dce4da-system-cni-dir\") pod \"multus-additional-cni-plugins-7n5z9\" (UID: \"04cb8a74-98ed-4fe2-9c4f-189783dce4da\") " pod="openshift-multus/multus-additional-cni-plugins-7n5z9"
Jul 30 03:34:56.933563 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.932846    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-cnibin\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:56.933563 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.932875    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-host-run-multus-certs\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:56.933563 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.932908    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c50df0e7-6b07-47da-9c3c-d4328dcdcfa5-konnectivity-ca\") pod \"konnectivity-agent-9jdph\" (UID: \"c50df0e7-6b07-47da-9c3c-d4328dcdcfa5\") " pod="kube-system/konnectivity-agent-9jdph"
Jul 30 03:34:56.933563 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.932945    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5t44\" (UniqueName: \"kubernetes.io/projected/4fc32317-f442-41da-8068-42fd29b51d26-kube-api-access-j5t44\") pod \"iptables-alerter-mmh5j\" (UID: \"4fc32317-f442-41da-8068-42fd29b51d26\") " pod="openshift-network-operator/iptables-alerter-mmh5j"
Jul 30 03:34:56.933563 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.932992    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2ce5d0b8-bd3d-458a-a201-c2633e1af196-registration-dir\") pod \"aws-ebs-csi-driver-node-h7bhs\" (UID: \"2ce5d0b8-bd3d-458a-a201-c2633e1af196\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7bhs"
Jul 30 03:34:56.933563 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.933053    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-etc-sysctl-d\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.933563 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.933079    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-sys\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.933563 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.933107    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2ce5d0b8-bd3d-458a-a201-c2633e1af196-registration-dir\") pod \"aws-ebs-csi-driver-node-h7bhs\" (UID: \"2ce5d0b8-bd3d-458a-a201-c2633e1af196\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7bhs"
Jul 30 03:34:56.933563 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.933111    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-host-var-lib-cni-multus\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:56.933563 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.933152    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1c2622ca-8052-489f-8d23-999618145c5a-original-pull-secret\") pod \"global-pull-secret-syncer-pzlrv\" (UID: \"1c2622ca-8052-489f-8d23-999618145c5a\") " pod="kube-system/global-pull-secret-syncer-pzlrv"
Jul 30 03:34:56.933563 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.933181    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-run-systemd\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:56.933563 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.933195    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-sys\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.934433 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.933212    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-etc-sysctl-d\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.934433 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.933228    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-var-lib-openvswitch\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:56.934433 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.933262    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/04cb8a74-98ed-4fe2-9c4f-189783dce4da-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7n5z9\" (UID: \"04cb8a74-98ed-4fe2-9c4f-189783dce4da\") " pod="openshift-multus/multus-additional-cni-plugins-7n5z9"
Jul 30 03:34:56.934433 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.933289    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2ce5d0b8-bd3d-458a-a201-c2633e1af196-device-dir\") pod \"aws-ebs-csi-driver-node-h7bhs\" (UID: \"2ce5d0b8-bd3d-458a-a201-c2633e1af196\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7bhs"
Jul 30 03:34:56.934433 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.933315    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b90028ac-a40a-4d9c-816a-8fec579a4b93-serviceca\") pod \"node-ca-tl4qv\" (UID: \"b90028ac-a40a-4d9c-816a-8fec579a4b93\") " pod="openshift-image-registry/node-ca-tl4qv"
Jul 30 03:34:56.934433 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.933340    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/890cb2cc-a9ca-4b13-bef5-f2282e486612-metrics-certs\") pod \"network-metrics-daemon-rqrtn\" (UID: \"890cb2cc-a9ca-4b13-bef5-f2282e486612\") " pod="openshift-multus/network-metrics-daemon-rqrtn"
Jul 30 03:34:56.934433 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.933367    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-system-cni-dir\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:56.934433 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.933394    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-run-ovn\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:56.934433 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.933403    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2ce5d0b8-bd3d-458a-a201-c2633e1af196-device-dir\") pod \"aws-ebs-csi-driver-node-h7bhs\" (UID: \"2ce5d0b8-bd3d-458a-a201-c2633e1af196\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7bhs"
Jul 30 03:34:56.934433 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.933427    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-log-socket\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:56.934433 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.933434    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/04cb8a74-98ed-4fe2-9c4f-189783dce4da-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7n5z9\" (UID: \"04cb8a74-98ed-4fe2-9c4f-189783dce4da\") " pod="openshift-multus/multus-additional-cni-plugins-7n5z9"
Jul 30 03:34:56.934433 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.933472    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/04cb8a74-98ed-4fe2-9c4f-189783dce4da-cnibin\") pod \"multus-additional-cni-plugins-7n5z9\" (UID: \"04cb8a74-98ed-4fe2-9c4f-189783dce4da\") " pod="openshift-multus/multus-additional-cni-plugins-7n5z9"
Jul 30 03:34:56.934433 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.933510    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b90028ac-a40a-4d9c-816a-8fec579a4b93-host\") pod \"node-ca-tl4qv\" (UID: \"b90028ac-a40a-4d9c-816a-8fec579a4b93\") " pod="openshift-image-registry/node-ca-tl4qv"
Jul 30 03:34:56.934433 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.933558    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/04cb8a74-98ed-4fe2-9c4f-189783dce4da-cnibin\") pod \"multus-additional-cni-plugins-7n5z9\" (UID: \"04cb8a74-98ed-4fe2-9c4f-189783dce4da\") " pod="openshift-multus/multus-additional-cni-plugins-7n5z9"
Jul 30 03:34:56.934433 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:56.933568    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 03:34:56.934433 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.933605    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b90028ac-a40a-4d9c-816a-8fec579a4b93-host\") pod \"node-ca-tl4qv\" (UID: \"b90028ac-a40a-4d9c-816a-8fec579a4b93\") " pod="openshift-image-registry/node-ca-tl4qv"
Jul 30 03:34:56.934433 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.933571    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-multus-socket-dir-parent\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:56.935078 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:56.933664    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/890cb2cc-a9ca-4b13-bef5-f2282e486612-metrics-certs podName:890cb2cc-a9ca-4b13-bef5-f2282e486612 nodeName:}" failed. No retries permitted until 2026-07-30 03:34:57.433627599 +0000 UTC m=+3.051859862 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/890cb2cc-a9ca-4b13-bef5-f2282e486612-metrics-certs") pod "network-metrics-daemon-rqrtn" (UID: "890cb2cc-a9ca-4b13-bef5-f2282e486612") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 03:34:56.935078 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.933699    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-run-openvswitch\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:56.935078 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.933730    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/62d78bc8-364a-44c0-ac71-f00f0a84623f-ovn-node-metrics-cert\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:56.935078 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.933761    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4fc32317-f442-41da-8068-42fd29b51d26-host-slash\") pod \"iptables-alerter-mmh5j\" (UID: \"4fc32317-f442-41da-8068-42fd29b51d26\") " pod="openshift-network-operator/iptables-alerter-mmh5j"
Jul 30 03:34:56.935078 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.933789    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-etc-modprobe-d\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.935078 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.933808    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b90028ac-a40a-4d9c-816a-8fec579a4b93-serviceca\") pod \"node-ca-tl4qv\" (UID: \"b90028ac-a40a-4d9c-816a-8fec579a4b93\") " pod="openshift-image-registry/node-ca-tl4qv"
Jul 30 03:34:56.935078 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.933815    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-etc-tuned\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.935078 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.933825    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4fc32317-f442-41da-8068-42fd29b51d26-host-slash\") pod \"iptables-alerter-mmh5j\" (UID: \"4fc32317-f442-41da-8068-42fd29b51d26\") " pod="openshift-network-operator/iptables-alerter-mmh5j"
Jul 30 03:34:56.935078 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.933839    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-node-log\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:56.935078 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.933870    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/04cb8a74-98ed-4fe2-9c4f-189783dce4da-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7n5z9\" (UID: \"04cb8a74-98ed-4fe2-9c4f-189783dce4da\") " pod="openshift-multus/multus-additional-cni-plugins-7n5z9"
Jul 30 03:34:56.935078 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.933897    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-host-cni-netd\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:56.935078 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.933926    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-etc-modprobe-d\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.935078 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.934459    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/04cb8a74-98ed-4fe2-9c4f-189783dce4da-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7n5z9\" (UID: \"04cb8a74-98ed-4fe2-9c4f-189783dce4da\") " pod="openshift-multus/multus-additional-cni-plugins-7n5z9"
Jul 30 03:34:56.937795 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.937767    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-etc-tuned\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.937929 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.937767    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-tmp\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.938022 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:56.937950    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 03:34:56.938022 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:56.937967    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 03:34:56.938022 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:56.937980    2576 projected.go:196] Error preparing data for projected volume kube-api-access-pz6fq for pod openshift-network-diagnostics/network-check-target-fzz5w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 03:34:56.938022 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:56.938048    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c8447d93-b20e-44aa-a11e-3c1aec4d0ddd-kube-api-access-pz6fq podName:c8447d93-b20e-44aa-a11e-3c1aec4d0ddd nodeName:}" failed. No retries permitted until 2026-07-30 03:34:57.438028188 +0000 UTC m=+3.056260427 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-pz6fq" (UniqueName: "kubernetes.io/projected/c8447d93-b20e-44aa-a11e-3c1aec4d0ddd-kube-api-access-pz6fq") pod "network-check-target-fzz5w" (UID: "c8447d93-b20e-44aa-a11e-3c1aec4d0ddd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 03:34:56.940594 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.940561    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvrr6\" (UniqueName: \"kubernetes.io/projected/890cb2cc-a9ca-4b13-bef5-f2282e486612-kube-api-access-vvrr6\") pod \"network-metrics-daemon-rqrtn\" (UID: \"890cb2cc-a9ca-4b13-bef5-f2282e486612\") " pod="openshift-multus/network-metrics-daemon-rqrtn"
Jul 30 03:34:56.941951 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.941926    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v86vs\" (UniqueName: \"kubernetes.io/projected/b90028ac-a40a-4d9c-816a-8fec579a4b93-kube-api-access-v86vs\") pod \"node-ca-tl4qv\" (UID: \"b90028ac-a40a-4d9c-816a-8fec579a4b93\") " pod="openshift-image-registry/node-ca-tl4qv"
Jul 30 03:34:56.942089 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.942048    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9rkg\" (UniqueName: \"kubernetes.io/projected/2ce5d0b8-bd3d-458a-a201-c2633e1af196-kube-api-access-z9rkg\") pod \"aws-ebs-csi-driver-node-h7bhs\" (UID: \"2ce5d0b8-bd3d-458a-a201-c2633e1af196\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7bhs"
Jul 30 03:34:56.944158 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.944130    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z7tz\" (UniqueName: \"kubernetes.io/projected/04cb8a74-98ed-4fe2-9c4f-189783dce4da-kube-api-access-7z7tz\") pod \"multus-additional-cni-plugins-7n5z9\" (UID: \"04cb8a74-98ed-4fe2-9c4f-189783dce4da\") " pod="openshift-multus/multus-additional-cni-plugins-7n5z9"
Jul 30 03:34:56.945015 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.944981    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkwb6\" (UniqueName: \"kubernetes.io/projected/ef1fe8ef-7af9-4567-8694-5d5e9b204a70-kube-api-access-vkwb6\") pod \"tuned-fszmc\" (UID: \"ef1fe8ef-7af9-4567-8694-5d5e9b204a70\") " pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:56.945125 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:56.945065    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5t44\" (UniqueName: \"kubernetes.io/projected/4fc32317-f442-41da-8068-42fd29b51d26-kube-api-access-j5t44\") pod \"iptables-alerter-mmh5j\" (UID: \"4fc32317-f442-41da-8068-42fd29b51d26\") " pod="openshift-network-operator/iptables-alerter-mmh5j"
Jul 30 03:34:57.034718 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.034678    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-node-log\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.034895 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.034727    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-host-cni-netd\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.034895 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.034754    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/62d78bc8-364a-44c0-ac71-f00f0a84623f-env-overrides\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.034895 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.034781    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-os-release\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:57.034895 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.034807    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-host-var-lib-cni-bin\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:57.034895 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.034823    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-node-log\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.034895 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.034835    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-multus-conf-dir\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:57.034895 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.034882    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-multus-conf-dir\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:57.035323 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.034899    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/62d78bc8-364a-44c0-ac71-f00f0a84623f-ovnkube-config\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.035323 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.034928    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-host-cni-netd\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.035323 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.034933    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49852e08-c471-4251-9b75-5e3e7fcc189b-tmp-dir\") pod \"node-resolver-mfgtq\" (UID: \"49852e08-c471-4251-9b75-5e3e7fcc189b\") " pod="openshift-dns/node-resolver-mfgtq"
Jul 30 03:34:57.035323 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.034985    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppx95\" (UniqueName: \"kubernetes.io/projected/62d78bc8-364a-44c0-ac71-f00f0a84623f-kube-api-access-ppx95\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.035323 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.035014    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-host-kubelet\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.035323 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.035040    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-host-run-k8s-cni-cncf-io\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:57.035323 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.035066    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2e09e50f-f869-4451-9504-a0909a29ae2a-multus-daemon-config\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:57.035323 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.035091    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-host-cni-bin\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.035323 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.035118    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-hostroot\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:57.035323 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.035143    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/49852e08-c471-4251-9b75-5e3e7fcc189b-hosts-file\") pod \"node-resolver-mfgtq\" (UID: \"49852e08-c471-4251-9b75-5e3e7fcc189b\") " pod="openshift-dns/node-resolver-mfgtq"
Jul 30 03:34:57.035323 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.035170    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-host-run-netns\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.035323 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.035226    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1c2622ca-8052-489f-8d23-999618145c5a-dbus\") pod \"global-pull-secret-syncer-pzlrv\" (UID: \"1c2622ca-8052-489f-8d23-999618145c5a\") " pod="kube-system/global-pull-secret-syncer-pzlrv"
Jul 30 03:34:57.035323 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.035255    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-systemd-units\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.035323 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.035284    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-etc-openvswitch\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.035323 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.035314    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/62d78bc8-364a-44c0-ac71-f00f0a84623f-ovnkube-script-lib\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.036123 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.035368    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-host-run-netns\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:57.036123 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.035398    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-etc-kubernetes\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:57.036123 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.035429    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwwhc\" (UniqueName: \"kubernetes.io/projected/2e09e50f-f869-4451-9504-a0909a29ae2a-kube-api-access-jwwhc\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:57.036123 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.035470    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58w4w\" (UniqueName: \"kubernetes.io/projected/49852e08-c471-4251-9b75-5e3e7fcc189b-kube-api-access-58w4w\") pod \"node-resolver-mfgtq\" (UID: \"49852e08-c471-4251-9b75-5e3e7fcc189b\") " pod="openshift-dns/node-resolver-mfgtq"
Jul 30 03:34:57.036123 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.035478    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/62d78bc8-364a-44c0-ac71-f00f0a84623f-env-overrides\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.036123 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.035499    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c50df0e7-6b07-47da-9c3c-d4328dcdcfa5-agent-certs\") pod \"konnectivity-agent-9jdph\" (UID: \"c50df0e7-6b07-47da-9c3c-d4328dcdcfa5\") " pod="kube-system/konnectivity-agent-9jdph"
Jul 30 03:34:57.036123 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.035531    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2e09e50f-f869-4451-9504-a0909a29ae2a-cni-binary-copy\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:57.036123 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.035555    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-os-release\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:57.036123 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.035567    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-host-var-lib-kubelet\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:57.036123 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.035583    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-host-var-lib-cni-bin\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:57.036123 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.035598    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1c2622ca-8052-489f-8d23-999618145c5a-kubelet-config\") pod \"global-pull-secret-syncer-pzlrv\" (UID: \"1c2622ca-8052-489f-8d23-999618145c5a\") " pod="kube-system/global-pull-secret-syncer-pzlrv"
Jul 30 03:34:57.036123 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.035630    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-host-slash\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.036123 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.035681    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1c2622ca-8052-489f-8d23-999618145c5a-dbus\") pod \"global-pull-secret-syncer-pzlrv\" (UID: \"1c2622ca-8052-489f-8d23-999618145c5a\") " pod="kube-system/global-pull-secret-syncer-pzlrv"
Jul 30 03:34:57.036123 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.035661    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-host-run-ovn-kubernetes\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.036123 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.035732    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-host-run-ovn-kubernetes\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.036123 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.035773    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.036123 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.035813    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-multus-cni-dir\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:57.036979 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.036173    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/62d78bc8-364a-44c0-ac71-f00f0a84623f-ovnkube-config\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.036979 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.036228    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-systemd-units\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.036979 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.036286    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-host-run-netns\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.036979 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.036299    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/49852e08-c471-4251-9b75-5e3e7fcc189b-hosts-file\") pod \"node-resolver-mfgtq\" (UID: \"49852e08-c471-4251-9b75-5e3e7fcc189b\") " pod="openshift-dns/node-resolver-mfgtq"
Jul 30 03:34:57.036979 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.036351    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1c2622ca-8052-489f-8d23-999618145c5a-kubelet-config\") pod \"global-pull-secret-syncer-pzlrv\" (UID: \"1c2622ca-8052-489f-8d23-999618145c5a\") " pod="kube-system/global-pull-secret-syncer-pzlrv"
Jul 30 03:34:57.036979 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.036365    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2e09e50f-f869-4451-9504-a0909a29ae2a-multus-daemon-config\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:57.036979 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.036401    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-etc-openvswitch\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.036979 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.036439    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-multus-cni-dir\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:57.036979 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.036470    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-host-slash\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.036979 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.036548    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-host-var-lib-kubelet\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:57.036979 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.036592    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.036979 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.036639    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-host-kubelet\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.036979 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.036733    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49852e08-c471-4251-9b75-5e3e7fcc189b-tmp-dir\") pod \"node-resolver-mfgtq\" (UID: \"49852e08-c471-4251-9b75-5e3e7fcc189b\") " pod="openshift-dns/node-resolver-mfgtq"
Jul 30 03:34:57.036979 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.036764    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2e09e50f-f869-4451-9504-a0909a29ae2a-cni-binary-copy\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:57.036979 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.036791    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-host-run-k8s-cni-cncf-io\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:57.036979 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.036807    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-cnibin\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:57.036979 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.036841    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-host-run-netns\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:57.036979 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.036869    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-host-run-multus-certs\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:57.037964 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.036890    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-etc-kubernetes\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:57.037964 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.036901    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c50df0e7-6b07-47da-9c3c-d4328dcdcfa5-konnectivity-ca\") pod \"konnectivity-agent-9jdph\" (UID: \"c50df0e7-6b07-47da-9c3c-d4328dcdcfa5\") " pod="kube-system/konnectivity-agent-9jdph"
Jul 30 03:34:57.037964 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.036229    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-host-cni-bin\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.037964 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.036925    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-cnibin\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:57.037964 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.036948    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-host-run-multus-certs\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:57.037964 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.036953    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/62d78bc8-364a-44c0-ac71-f00f0a84623f-ovnkube-script-lib\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.037964 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.036997    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-hostroot\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:57.037964 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.037023    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-host-var-lib-cni-multus\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:57.037964 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.037061    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-host-var-lib-cni-multus\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:57.037964 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.037066    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1c2622ca-8052-489f-8d23-999618145c5a-original-pull-secret\") pod \"global-pull-secret-syncer-pzlrv\" (UID: \"1c2622ca-8052-489f-8d23-999618145c5a\") " pod="kube-system/global-pull-secret-syncer-pzlrv"
Jul 30 03:34:57.037964 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.037101    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-run-systemd\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.037964 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.037128    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-var-lib-openvswitch\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.037964 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:57.037145    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 03:34:57.037964 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.037173    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-system-cni-dir\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:57.037964 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:57.037206    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c2622ca-8052-489f-8d23-999618145c5a-original-pull-secret podName:1c2622ca-8052-489f-8d23-999618145c5a nodeName:}" failed. No retries permitted until 2026-07-30 03:34:57.537186576 +0000 UTC m=+3.155418819 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1c2622ca-8052-489f-8d23-999618145c5a-original-pull-secret") pod "global-pull-secret-syncer-pzlrv" (UID: "1c2622ca-8052-489f-8d23-999618145c5a") : object "kube-system"/"original-pull-secret" not registered
Jul 30 03:34:57.037964 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.037230    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-system-cni-dir\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:57.037964 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.037237    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-run-systemd\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.037964 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.037272    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-var-lib-openvswitch\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.038780 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.037286    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-run-ovn\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.038780 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.037317    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-log-socket\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.038780 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.037351    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-multus-socket-dir-parent\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:57.038780 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.037381    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-run-openvswitch\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.038780 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.037401    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-run-ovn\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.038780 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.037409    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/62d78bc8-364a-44c0-ac71-f00f0a84623f-ovn-node-metrics-cert\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.038780 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.037415    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-log-socket\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.038780 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.037408    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c50df0e7-6b07-47da-9c3c-d4328dcdcfa5-konnectivity-ca\") pod \"konnectivity-agent-9jdph\" (UID: \"c50df0e7-6b07-47da-9c3c-d4328dcdcfa5\") " pod="kube-system/konnectivity-agent-9jdph"
Jul 30 03:34:57.038780 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.037448    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/62d78bc8-364a-44c0-ac71-f00f0a84623f-run-openvswitch\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.038780 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.037454    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2e09e50f-f869-4451-9504-a0909a29ae2a-multus-socket-dir-parent\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:57.039488 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.039464    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c50df0e7-6b07-47da-9c3c-d4328dcdcfa5-agent-certs\") pod \"konnectivity-agent-9jdph\" (UID: \"c50df0e7-6b07-47da-9c3c-d4328dcdcfa5\") " pod="kube-system/konnectivity-agent-9jdph"
Jul 30 03:34:57.039818 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.039796    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/62d78bc8-364a-44c0-ac71-f00f0a84623f-ovn-node-metrics-cert\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.051784 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.051720    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-117.ec2.internal" event={"ID":"78f929edfbd95057c37ce421449a9742","Type":"ContainerStarted","Data":"8ed62c63c7bcd9ee92dab8bbf469431e678f8d075028fef8ba16b3f194b1f4f1"}
Jul 30 03:34:57.053010 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.052975    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal" event={"ID":"050b224397a0390fc072543beebab8c7","Type":"ContainerStarted","Data":"7cad6e65b4079c6d91fe708cb8912b55a47b16133e115e0453c7703909e253b8"}
Jul 30 03:34:57.056088 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.056056    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwwhc\" (UniqueName: \"kubernetes.io/projected/2e09e50f-f869-4451-9504-a0909a29ae2a-kube-api-access-jwwhc\") pod \"multus-th2nr\" (UID: \"2e09e50f-f869-4451-9504-a0909a29ae2a\") " pod="openshift-multus/multus-th2nr"
Jul 30 03:34:57.057119 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.057094    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58w4w\" (UniqueName: \"kubernetes.io/projected/49852e08-c471-4251-9b75-5e3e7fcc189b-kube-api-access-58w4w\") pod \"node-resolver-mfgtq\" (UID: \"49852e08-c471-4251-9b75-5e3e7fcc189b\") " pod="openshift-dns/node-resolver-mfgtq"
Jul 30 03:34:57.063618 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.063558    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppx95\" (UniqueName: \"kubernetes.io/projected/62d78bc8-364a-44c0-ac71-f00f0a84623f-kube-api-access-ppx95\") pod \"ovnkube-node-nhwmf\" (UID: \"62d78bc8-364a-44c0-ac71-f00f0a84623f\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.119796 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.119699    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7n5z9"
Jul 30 03:34:57.129139 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.129100    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mmh5j"
Jul 30 03:34:57.142377 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.142267    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7bhs"
Jul 30 03:34:57.149575 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.149536    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fszmc"
Jul 30 03:34:57.157939 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.157894    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tl4qv"
Jul 30 03:34:57.168150 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.168116    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-th2nr"
Jul 30 03:34:57.178409 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.178376    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mfgtq"
Jul 30 03:34:57.185847 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.185808    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9jdph"
Jul 30 03:34:57.192940 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.192904    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:34:57.441168 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.441077    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz6fq\" (UniqueName: \"kubernetes.io/projected/c8447d93-b20e-44aa-a11e-3c1aec4d0ddd-kube-api-access-pz6fq\") pod \"network-check-target-fzz5w\" (UID: \"c8447d93-b20e-44aa-a11e-3c1aec4d0ddd\") " pod="openshift-network-diagnostics/network-check-target-fzz5w"
Jul 30 03:34:57.441332 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.441174    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/890cb2cc-a9ca-4b13-bef5-f2282e486612-metrics-certs\") pod \"network-metrics-daemon-rqrtn\" (UID: \"890cb2cc-a9ca-4b13-bef5-f2282e486612\") " pod="openshift-multus/network-metrics-daemon-rqrtn"
Jul 30 03:34:57.441332 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:57.441255    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 03:34:57.441332 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:57.441282    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 03:34:57.441332 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:57.441297    2576 projected.go:196] Error preparing data for projected volume kube-api-access-pz6fq for pod openshift-network-diagnostics/network-check-target-fzz5w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 03:34:57.441332 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:57.441300    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 03:34:57.441525 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:57.441357    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c8447d93-b20e-44aa-a11e-3c1aec4d0ddd-kube-api-access-pz6fq podName:c8447d93-b20e-44aa-a11e-3c1aec4d0ddd nodeName:}" failed. No retries permitted until 2026-07-30 03:34:58.441341955 +0000 UTC m=+4.059574176 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-pz6fq" (UniqueName: "kubernetes.io/projected/c8447d93-b20e-44aa-a11e-3c1aec4d0ddd-kube-api-access-pz6fq") pod "network-check-target-fzz5w" (UID: "c8447d93-b20e-44aa-a11e-3c1aec4d0ddd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 03:34:57.441525 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:57.441373    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/890cb2cc-a9ca-4b13-bef5-f2282e486612-metrics-certs podName:890cb2cc-a9ca-4b13-bef5-f2282e486612 nodeName:}" failed. No retries permitted until 2026-07-30 03:34:58.441366212 +0000 UTC m=+4.059598434 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/890cb2cc-a9ca-4b13-bef5-f2282e486612-metrics-certs") pod "network-metrics-daemon-rqrtn" (UID: "890cb2cc-a9ca-4b13-bef5-f2282e486612") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 03:34:57.542607 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.542570    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1c2622ca-8052-489f-8d23-999618145c5a-original-pull-secret\") pod \"global-pull-secret-syncer-pzlrv\" (UID: \"1c2622ca-8052-489f-8d23-999618145c5a\") " pod="kube-system/global-pull-secret-syncer-pzlrv"
Jul 30 03:34:57.542800 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:57.542748    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 03:34:57.542877 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:57.542829    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c2622ca-8052-489f-8d23-999618145c5a-original-pull-secret podName:1c2622ca-8052-489f-8d23-999618145c5a nodeName:}" failed. No retries permitted until 2026-07-30 03:34:58.542810033 +0000 UTC m=+4.161042257 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1c2622ca-8052-489f-8d23-999618145c5a-original-pull-secret") pod "global-pull-secret-syncer-pzlrv" (UID: "1c2622ca-8052-489f-8d23-999618145c5a") : object "kube-system"/"original-pull-secret" not registered
Jul 30 03:34:57.861666 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.861510    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 03:29:56 +0000 UTC" deadline="2028-04-04 04:40:42.085316083 +0000 UTC"
Jul 30 03:34:57.861666 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:57.861599    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14737h5m44.22372216s"
Jul 30 03:34:58.045451 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:58.045409    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fzz5w"
Jul 30 03:34:58.045653 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:58.045555    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fzz5w" podUID="c8447d93-b20e-44aa-a11e-3c1aec4d0ddd"
Jul 30 03:34:58.145384 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:58.145226    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb90028ac_a40a_4d9c_816a_8fec579a4b93.slice/crio-900d5455e666c46cfdee64572d44a1ef20e47717d9c018a59cdd4551f3ffdbb2 WatchSource:0}: Error finding container 900d5455e666c46cfdee64572d44a1ef20e47717d9c018a59cdd4551f3ffdbb2: Status 404 returned error can't find the container with id 900d5455e666c46cfdee64572d44a1ef20e47717d9c018a59cdd4551f3ffdbb2
Jul 30 03:34:58.147064 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:58.146905    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc50df0e7_6b07_47da_9c3c_d4328dcdcfa5.slice/crio-21e357180508136fa4db12b7021dd5235f663b8a3b8a46de00985e26ba131d32 WatchSource:0}: Error finding container 21e357180508136fa4db12b7021dd5235f663b8a3b8a46de00985e26ba131d32: Status 404 returned error can't find the container with id 21e357180508136fa4db12b7021dd5235f663b8a3b8a46de00985e26ba131d32
Jul 30 03:34:58.147740 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:58.147627    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fc32317_f442_41da_8068_42fd29b51d26.slice/crio-27ccdc87cecf4f73bf2d75550dc04bdf88e3d5f53982f5a2b1ea6ea1d1ecfabd WatchSource:0}: Error finding container 27ccdc87cecf4f73bf2d75550dc04bdf88e3d5f53982f5a2b1ea6ea1d1ecfabd: Status 404 returned error can't find the container with id 27ccdc87cecf4f73bf2d75550dc04bdf88e3d5f53982f5a2b1ea6ea1d1ecfabd
Jul 30 03:34:58.148888 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:58.148858    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e09e50f_f869_4451_9504_a0909a29ae2a.slice/crio-886a151e808b5311e8a68f4846ee949d2edc84e8f0fad6d76d1ac63d10dec91b WatchSource:0}: Error finding container 886a151e808b5311e8a68f4846ee949d2edc84e8f0fad6d76d1ac63d10dec91b: Status 404 returned error can't find the container with id 886a151e808b5311e8a68f4846ee949d2edc84e8f0fad6d76d1ac63d10dec91b
Jul 30 03:34:58.151035 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:58.150874    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04cb8a74_98ed_4fe2_9c4f_189783dce4da.slice/crio-6565c7c57b85a1892de6b0960221f05bd152b069f472f7efd043407ed0e9f01c WatchSource:0}: Error finding container 6565c7c57b85a1892de6b0960221f05bd152b069f472f7efd043407ed0e9f01c: Status 404 returned error can't find the container with id 6565c7c57b85a1892de6b0960221f05bd152b069f472f7efd043407ed0e9f01c
Jul 30 03:34:58.151704 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:34:58.151678    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49852e08_c471_4251_9b75_5e3e7fcc189b.slice/crio-4661e8d19ca57169a85b23649a36025519ede1ef90bb745c6a082e3028107774 WatchSource:0}: Error finding container 4661e8d19ca57169a85b23649a36025519ede1ef90bb745c6a082e3028107774: Status 404 returned error can't find the container with id 4661e8d19ca57169a85b23649a36025519ede1ef90bb745c6a082e3028107774
Jul 30 03:34:58.450912 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:58.450704    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz6fq\" (UniqueName: \"kubernetes.io/projected/c8447d93-b20e-44aa-a11e-3c1aec4d0ddd-kube-api-access-pz6fq\") pod \"network-check-target-fzz5w\" (UID: \"c8447d93-b20e-44aa-a11e-3c1aec4d0ddd\") " pod="openshift-network-diagnostics/network-check-target-fzz5w"
Jul 30 03:34:58.450912 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:58.450896    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 03:34:58.450912 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:58.450921    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 03:34:58.451204 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:58.450949    2576 projected.go:196] Error preparing data for projected volume kube-api-access-pz6fq for pod openshift-network-diagnostics/network-check-target-fzz5w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 03:34:58.451204 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:58.451004    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c8447d93-b20e-44aa-a11e-3c1aec4d0ddd-kube-api-access-pz6fq podName:c8447d93-b20e-44aa-a11e-3c1aec4d0ddd nodeName:}" failed. No retries permitted until 2026-07-30 03:35:00.450989963 +0000 UTC m=+6.069222206 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-pz6fq" (UniqueName: "kubernetes.io/projected/c8447d93-b20e-44aa-a11e-3c1aec4d0ddd-kube-api-access-pz6fq") pod "network-check-target-fzz5w" (UID: "c8447d93-b20e-44aa-a11e-3c1aec4d0ddd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 03:34:58.451204 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:58.450945    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/890cb2cc-a9ca-4b13-bef5-f2282e486612-metrics-certs\") pod \"network-metrics-daemon-rqrtn\" (UID: \"890cb2cc-a9ca-4b13-bef5-f2282e486612\") " pod="openshift-multus/network-metrics-daemon-rqrtn"
Jul 30 03:34:58.451204 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:58.451038    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 03:34:58.451204 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:58.451087    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/890cb2cc-a9ca-4b13-bef5-f2282e486612-metrics-certs podName:890cb2cc-a9ca-4b13-bef5-f2282e486612 nodeName:}" failed. No retries permitted until 2026-07-30 03:35:00.451074402 +0000 UTC m=+6.069306624 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/890cb2cc-a9ca-4b13-bef5-f2282e486612-metrics-certs") pod "network-metrics-daemon-rqrtn" (UID: "890cb2cc-a9ca-4b13-bef5-f2282e486612") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 03:34:58.551820 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:58.551775    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1c2622ca-8052-489f-8d23-999618145c5a-original-pull-secret\") pod \"global-pull-secret-syncer-pzlrv\" (UID: \"1c2622ca-8052-489f-8d23-999618145c5a\") " pod="kube-system/global-pull-secret-syncer-pzlrv"
Jul 30 03:34:58.552040 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:58.551955    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 03:34:58.552136 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:58.552053    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c2622ca-8052-489f-8d23-999618145c5a-original-pull-secret podName:1c2622ca-8052-489f-8d23-999618145c5a nodeName:}" failed. No retries permitted until 2026-07-30 03:35:00.55203285 +0000 UTC m=+6.170265090 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1c2622ca-8052-489f-8d23-999618145c5a-original-pull-secret") pod "global-pull-secret-syncer-pzlrv" (UID: "1c2622ca-8052-489f-8d23-999618145c5a") : object "kube-system"/"original-pull-secret" not registered
Jul 30 03:34:59.046313 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:59.046146    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqrtn"
Jul 30 03:34:59.046313 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:59.046172    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzlrv"
Jul 30 03:34:59.046313 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:59.046291    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzlrv" podUID="1c2622ca-8052-489f-8d23-999618145c5a"
Jul 30 03:34:59.047352 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:34:59.046739    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqrtn" podUID="890cb2cc-a9ca-4b13-bef5-f2282e486612"
Jul 30 03:34:59.065778 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:59.065731    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-th2nr" event={"ID":"2e09e50f-f869-4451-9504-a0909a29ae2a","Type":"ContainerStarted","Data":"886a151e808b5311e8a68f4846ee949d2edc84e8f0fad6d76d1ac63d10dec91b"}
Jul 30 03:34:59.089078 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:59.088973    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mmh5j" event={"ID":"4fc32317-f442-41da-8068-42fd29b51d26","Type":"ContainerStarted","Data":"27ccdc87cecf4f73bf2d75550dc04bdf88e3d5f53982f5a2b1ea6ea1d1ecfabd"}
Jul 30 03:34:59.094343 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:59.093351    2576 generic.go:356] "Generic (PLEG): container finished" podID="050b224397a0390fc072543beebab8c7" containerID="f355cc89322c2401d418d67337855979eef4c933b36578af3711b7a4f19eba41" exitCode=0
Jul 30 03:34:59.094343 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:59.093449    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal" event={"ID":"050b224397a0390fc072543beebab8c7","Type":"ContainerDied","Data":"f355cc89322c2401d418d67337855979eef4c933b36578af3711b7a4f19eba41"}
Jul 30 03:34:59.128649 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:59.128610    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fszmc" event={"ID":"ef1fe8ef-7af9-4567-8694-5d5e9b204a70","Type":"ContainerStarted","Data":"5d46e54f8b8cc2361c171f40b5b85625a7aaa76d947312f74ed6486707ffc0f8"}
Jul 30 03:34:59.145947 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:59.145835    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7bhs" event={"ID":"2ce5d0b8-bd3d-458a-a201-c2633e1af196","Type":"ContainerStarted","Data":"9a61d38ba5f1fe0334ec877ffdf73ccbc04ead776a5bad6398f077b91a2819ff"}
Jul 30 03:34:59.150627 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:59.150490    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf" event={"ID":"62d78bc8-364a-44c0-ac71-f00f0a84623f","Type":"ContainerStarted","Data":"e245d8d6d0638b60c49a679433280dea47b6df08a330da45696df838310a2449"}
Jul 30 03:34:59.172861 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:59.172701    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7n5z9" event={"ID":"04cb8a74-98ed-4fe2-9c4f-189783dce4da","Type":"ContainerStarted","Data":"6565c7c57b85a1892de6b0960221f05bd152b069f472f7efd043407ed0e9f01c"}
Jul 30 03:34:59.186249 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:59.186140    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9jdph" event={"ID":"c50df0e7-6b07-47da-9c3c-d4328dcdcfa5","Type":"ContainerStarted","Data":"21e357180508136fa4db12b7021dd5235f663b8a3b8a46de00985e26ba131d32"}
Jul 30 03:34:59.188654 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:59.188556    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tl4qv" event={"ID":"b90028ac-a40a-4d9c-816a-8fec579a4b93","Type":"ContainerStarted","Data":"900d5455e666c46cfdee64572d44a1ef20e47717d9c018a59cdd4551f3ffdbb2"}
Jul 30 03:34:59.212692 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:59.211411    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-117.ec2.internal" event={"ID":"78f929edfbd95057c37ce421449a9742","Type":"ContainerStarted","Data":"8e38c410bb2edde2f775d5f6b19d37b916a52143eea5788b7cdc41c011e11e61"}
Jul 30 03:34:59.229029 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:59.228852    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mfgtq" event={"ID":"49852e08-c471-4251-9b75-5e3e7fcc189b","Type":"ContainerStarted","Data":"4661e8d19ca57169a85b23649a36025519ede1ef90bb745c6a082e3028107774"}
Jul 30 03:34:59.230352 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:34:59.230201    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-117.ec2.internal" podStartSLOduration=3.230179579 podStartE2EDuration="3.230179579s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 03:34:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:34:59.228980364 +0000 UTC m=+4.847212648" watchObservedRunningTime="2026-07-30 03:34:59.230179579 +0000 UTC m=+4.848411825"
Jul 30 03:35:00.046362 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:00.045782    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fzz5w"
Jul 30 03:35:00.046362 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:00.045933    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fzz5w" podUID="c8447d93-b20e-44aa-a11e-3c1aec4d0ddd"
Jul 30 03:35:00.263744 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:00.263699    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal" event={"ID":"050b224397a0390fc072543beebab8c7","Type":"ContainerStarted","Data":"54241444adaa7ec230ead079aec8640c4271e6c670c7dbd69fa8cad5a6847e8b"}
Jul 30 03:35:00.470831 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:00.470733    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz6fq\" (UniqueName: \"kubernetes.io/projected/c8447d93-b20e-44aa-a11e-3c1aec4d0ddd-kube-api-access-pz6fq\") pod \"network-check-target-fzz5w\" (UID: \"c8447d93-b20e-44aa-a11e-3c1aec4d0ddd\") " pod="openshift-network-diagnostics/network-check-target-fzz5w"
Jul 30 03:35:00.471051 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:00.470836    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/890cb2cc-a9ca-4b13-bef5-f2282e486612-metrics-certs\") pod \"network-metrics-daemon-rqrtn\" (UID: \"890cb2cc-a9ca-4b13-bef5-f2282e486612\") " pod="openshift-multus/network-metrics-daemon-rqrtn"
Jul 30 03:35:00.471051 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:00.471014    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 03:35:00.471183 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:00.471086    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/890cb2cc-a9ca-4b13-bef5-f2282e486612-metrics-certs podName:890cb2cc-a9ca-4b13-bef5-f2282e486612 nodeName:}" failed. No retries permitted until 2026-07-30 03:35:04.47106418 +0000 UTC m=+10.089296406 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/890cb2cc-a9ca-4b13-bef5-f2282e486612-metrics-certs") pod "network-metrics-daemon-rqrtn" (UID: "890cb2cc-a9ca-4b13-bef5-f2282e486612") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 03:35:00.471602 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:00.471577    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 03:35:00.471706 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:00.471610    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 03:35:00.471706 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:00.471628    2576 projected.go:196] Error preparing data for projected volume kube-api-access-pz6fq for pod openshift-network-diagnostics/network-check-target-fzz5w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 03:35:00.471706 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:00.471682    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c8447d93-b20e-44aa-a11e-3c1aec4d0ddd-kube-api-access-pz6fq podName:c8447d93-b20e-44aa-a11e-3c1aec4d0ddd nodeName:}" failed. No retries permitted until 2026-07-30 03:35:04.471664758 +0000 UTC m=+10.089896993 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-pz6fq" (UniqueName: "kubernetes.io/projected/c8447d93-b20e-44aa-a11e-3c1aec4d0ddd-kube-api-access-pz6fq") pod "network-check-target-fzz5w" (UID: "c8447d93-b20e-44aa-a11e-3c1aec4d0ddd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 03:35:00.572229 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:00.572183    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1c2622ca-8052-489f-8d23-999618145c5a-original-pull-secret\") pod \"global-pull-secret-syncer-pzlrv\" (UID: \"1c2622ca-8052-489f-8d23-999618145c5a\") " pod="kube-system/global-pull-secret-syncer-pzlrv"
Jul 30 03:35:00.572414 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:00.572395    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 03:35:00.572496 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:00.572460    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c2622ca-8052-489f-8d23-999618145c5a-original-pull-secret podName:1c2622ca-8052-489f-8d23-999618145c5a nodeName:}" failed. No retries permitted until 2026-07-30 03:35:04.572442833 +0000 UTC m=+10.190675057 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1c2622ca-8052-489f-8d23-999618145c5a-original-pull-secret") pod "global-pull-secret-syncer-pzlrv" (UID: "1c2622ca-8052-489f-8d23-999618145c5a") : object "kube-system"/"original-pull-secret" not registered
Jul 30 03:35:01.047832 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:01.047795    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqrtn"
Jul 30 03:35:01.048376 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:01.047963    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqrtn" podUID="890cb2cc-a9ca-4b13-bef5-f2282e486612"
Jul 30 03:35:01.048447 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:01.048391    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzlrv"
Jul 30 03:35:01.048612 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:01.048494    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzlrv" podUID="1c2622ca-8052-489f-8d23-999618145c5a"
Jul 30 03:35:02.045572 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:02.045280    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fzz5w"
Jul 30 03:35:02.045572 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:02.045450    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fzz5w" podUID="c8447d93-b20e-44aa-a11e-3c1aec4d0ddd"
Jul 30 03:35:03.045350 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:03.045313    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqrtn"
Jul 30 03:35:03.045851 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:03.045313    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzlrv"
Jul 30 03:35:03.045851 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:03.045558    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqrtn" podUID="890cb2cc-a9ca-4b13-bef5-f2282e486612"
Jul 30 03:35:03.046023 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:03.045975    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzlrv" podUID="1c2622ca-8052-489f-8d23-999618145c5a"
Jul 30 03:35:04.045381 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:04.045293    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fzz5w"
Jul 30 03:35:04.045762 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:04.045442    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fzz5w" podUID="c8447d93-b20e-44aa-a11e-3c1aec4d0ddd"
Jul 30 03:35:04.513766 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:04.513607    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz6fq\" (UniqueName: \"kubernetes.io/projected/c8447d93-b20e-44aa-a11e-3c1aec4d0ddd-kube-api-access-pz6fq\") pod \"network-check-target-fzz5w\" (UID: \"c8447d93-b20e-44aa-a11e-3c1aec4d0ddd\") " pod="openshift-network-diagnostics/network-check-target-fzz5w"
Jul 30 03:35:04.513766 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:04.513699    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/890cb2cc-a9ca-4b13-bef5-f2282e486612-metrics-certs\") pod \"network-metrics-daemon-rqrtn\" (UID: \"890cb2cc-a9ca-4b13-bef5-f2282e486612\") " pod="openshift-multus/network-metrics-daemon-rqrtn"
Jul 30 03:35:04.514092 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:04.513857    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 03:35:04.514092 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:04.513860    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 03:35:04.514092 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:04.513886    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 03:35:04.514092 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:04.513900    2576 projected.go:196] Error preparing data for projected volume kube-api-access-pz6fq for pod openshift-network-diagnostics/network-check-target-fzz5w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 03:35:04.514092 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:04.513928    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/890cb2cc-a9ca-4b13-bef5-f2282e486612-metrics-certs podName:890cb2cc-a9ca-4b13-bef5-f2282e486612 nodeName:}" failed. No retries permitted until 2026-07-30 03:35:12.513906539 +0000 UTC m=+18.132138762 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/890cb2cc-a9ca-4b13-bef5-f2282e486612-metrics-certs") pod "network-metrics-daemon-rqrtn" (UID: "890cb2cc-a9ca-4b13-bef5-f2282e486612") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 03:35:04.514092 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:04.513969    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c8447d93-b20e-44aa-a11e-3c1aec4d0ddd-kube-api-access-pz6fq podName:c8447d93-b20e-44aa-a11e-3c1aec4d0ddd nodeName:}" failed. No retries permitted until 2026-07-30 03:35:12.513935794 +0000 UTC m=+18.132168031 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-pz6fq" (UniqueName: "kubernetes.io/projected/c8447d93-b20e-44aa-a11e-3c1aec4d0ddd-kube-api-access-pz6fq") pod "network-check-target-fzz5w" (UID: "c8447d93-b20e-44aa-a11e-3c1aec4d0ddd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 03:35:04.614940 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:04.614892    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1c2622ca-8052-489f-8d23-999618145c5a-original-pull-secret\") pod \"global-pull-secret-syncer-pzlrv\" (UID: \"1c2622ca-8052-489f-8d23-999618145c5a\") " pod="kube-system/global-pull-secret-syncer-pzlrv"
Jul 30 03:35:04.615134 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:04.615060    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 03:35:04.615222 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:04.615146    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c2622ca-8052-489f-8d23-999618145c5a-original-pull-secret podName:1c2622ca-8052-489f-8d23-999618145c5a nodeName:}" failed. No retries permitted until 2026-07-30 03:35:12.615122592 +0000 UTC m=+18.233354816 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1c2622ca-8052-489f-8d23-999618145c5a-original-pull-secret") pod "global-pull-secret-syncer-pzlrv" (UID: "1c2622ca-8052-489f-8d23-999618145c5a") : object "kube-system"/"original-pull-secret" not registered
Jul 30 03:35:05.047221 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:05.046467    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqrtn"
Jul 30 03:35:05.047221 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:05.046609    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqrtn" podUID="890cb2cc-a9ca-4b13-bef5-f2282e486612"
Jul 30 03:35:05.047221 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:05.047075    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzlrv"
Jul 30 03:35:05.047221 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:05.047176    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzlrv" podUID="1c2622ca-8052-489f-8d23-999618145c5a"
Jul 30 03:35:06.045430 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:06.045397    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fzz5w"
Jul 30 03:35:06.045639 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:06.045533    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fzz5w" podUID="c8447d93-b20e-44aa-a11e-3c1aec4d0ddd"
Jul 30 03:35:07.046139 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:07.045992    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqrtn"
Jul 30 03:35:07.046609 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:07.046145    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqrtn" podUID="890cb2cc-a9ca-4b13-bef5-f2282e486612"
Jul 30 03:35:07.046609 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:07.046521    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzlrv"
Jul 30 03:35:07.046729 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:07.046619    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzlrv" podUID="1c2622ca-8052-489f-8d23-999618145c5a"
Jul 30 03:35:08.045092 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:08.044976    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fzz5w"
Jul 30 03:35:08.045267 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:08.045117    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fzz5w" podUID="c8447d93-b20e-44aa-a11e-3c1aec4d0ddd"
Jul 30 03:35:09.045027 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:09.044981    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqrtn"
Jul 30 03:35:09.045479 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:09.045003    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzlrv"
Jul 30 03:35:09.045479 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:09.045119    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqrtn" podUID="890cb2cc-a9ca-4b13-bef5-f2282e486612"
Jul 30 03:35:09.045479 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:09.045213    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzlrv" podUID="1c2622ca-8052-489f-8d23-999618145c5a"
Jul 30 03:35:10.045896 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:10.045815    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fzz5w"
Jul 30 03:35:10.046288 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:10.045942    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fzz5w" podUID="c8447d93-b20e-44aa-a11e-3c1aec4d0ddd"
Jul 30 03:35:11.045015 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:11.044974    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqrtn"
Jul 30 03:35:11.045193 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:11.044974    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzlrv"
Jul 30 03:35:11.045193 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:11.045115    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqrtn" podUID="890cb2cc-a9ca-4b13-bef5-f2282e486612"
Jul 30 03:35:11.045322 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:11.045210    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzlrv" podUID="1c2622ca-8052-489f-8d23-999618145c5a"
Jul 30 03:35:12.045625 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:12.045587    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fzz5w"
Jul 30 03:35:12.046165 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:12.045722    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fzz5w" podUID="c8447d93-b20e-44aa-a11e-3c1aec4d0ddd"
Jul 30 03:35:12.579139 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:12.579085    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz6fq\" (UniqueName: \"kubernetes.io/projected/c8447d93-b20e-44aa-a11e-3c1aec4d0ddd-kube-api-access-pz6fq\") pod \"network-check-target-fzz5w\" (UID: \"c8447d93-b20e-44aa-a11e-3c1aec4d0ddd\") " pod="openshift-network-diagnostics/network-check-target-fzz5w"
Jul 30 03:35:12.579437 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:12.579168    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/890cb2cc-a9ca-4b13-bef5-f2282e486612-metrics-certs\") pod \"network-metrics-daemon-rqrtn\" (UID: \"890cb2cc-a9ca-4b13-bef5-f2282e486612\") " pod="openshift-multus/network-metrics-daemon-rqrtn"
Jul 30 03:35:12.579437 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:12.579271    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 03:35:12.579437 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:12.579281    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 03:35:12.579437 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:12.579311    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 03:35:12.579437 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:12.579323    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/890cb2cc-a9ca-4b13-bef5-f2282e486612-metrics-certs podName:890cb2cc-a9ca-4b13-bef5-f2282e486612 nodeName:}" failed. No retries permitted until 2026-07-30 03:35:28.579309959 +0000 UTC m=+34.197542178 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/890cb2cc-a9ca-4b13-bef5-f2282e486612-metrics-certs") pod "network-metrics-daemon-rqrtn" (UID: "890cb2cc-a9ca-4b13-bef5-f2282e486612") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 03:35:12.579437 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:12.579325    2576 projected.go:196] Error preparing data for projected volume kube-api-access-pz6fq for pod openshift-network-diagnostics/network-check-target-fzz5w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 03:35:12.579437 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:12.579388    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c8447d93-b20e-44aa-a11e-3c1aec4d0ddd-kube-api-access-pz6fq podName:c8447d93-b20e-44aa-a11e-3c1aec4d0ddd nodeName:}" failed. No retries permitted until 2026-07-30 03:35:28.579368007 +0000 UTC m=+34.197600246 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-pz6fq" (UniqueName: "kubernetes.io/projected/c8447d93-b20e-44aa-a11e-3c1aec4d0ddd-kube-api-access-pz6fq") pod "network-check-target-fzz5w" (UID: "c8447d93-b20e-44aa-a11e-3c1aec4d0ddd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 03:35:12.680231 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:12.680192    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1c2622ca-8052-489f-8d23-999618145c5a-original-pull-secret\") pod \"global-pull-secret-syncer-pzlrv\" (UID: \"1c2622ca-8052-489f-8d23-999618145c5a\") " pod="kube-system/global-pull-secret-syncer-pzlrv"
Jul 30 03:35:12.680420 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:12.680348    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 03:35:12.680420 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:12.680418    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c2622ca-8052-489f-8d23-999618145c5a-original-pull-secret podName:1c2622ca-8052-489f-8d23-999618145c5a nodeName:}" failed. No retries permitted until 2026-07-30 03:35:28.680401364 +0000 UTC m=+34.298633606 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1c2622ca-8052-489f-8d23-999618145c5a-original-pull-secret") pod "global-pull-secret-syncer-pzlrv" (UID: "1c2622ca-8052-489f-8d23-999618145c5a") : object "kube-system"/"original-pull-secret" not registered
Jul 30 03:35:13.045118 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:13.045074    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqrtn"
Jul 30 03:35:13.045296 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:13.045234    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqrtn" podUID="890cb2cc-a9ca-4b13-bef5-f2282e486612"
Jul 30 03:35:13.045347 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:13.045296    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzlrv"
Jul 30 03:35:13.045444 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:13.045411    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzlrv" podUID="1c2622ca-8052-489f-8d23-999618145c5a"
Jul 30 03:35:14.045691 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:14.045653    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fzz5w"
Jul 30 03:35:14.046175 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:14.045787    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fzz5w" podUID="c8447d93-b20e-44aa-a11e-3c1aec4d0ddd"
Jul 30 03:35:15.046249 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:15.046215    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqrtn"
Jul 30 03:35:15.046728 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:15.046316    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqrtn" podUID="890cb2cc-a9ca-4b13-bef5-f2282e486612"
Jul 30 03:35:15.046728 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:15.046351    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzlrv"
Jul 30 03:35:15.046728 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:15.046420    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzlrv" podUID="1c2622ca-8052-489f-8d23-999618145c5a"
Jul 30 03:35:16.046057 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:16.045877    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fzz5w"
Jul 30 03:35:16.046057 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:16.046024    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fzz5w" podUID="c8447d93-b20e-44aa-a11e-3c1aec4d0ddd"
Jul 30 03:35:17.045883 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:17.045844    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqrtn"
Jul 30 03:35:17.046365 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:17.045844    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzlrv"
Jul 30 03:35:17.046365 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:17.046021    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqrtn" podUID="890cb2cc-a9ca-4b13-bef5-f2282e486612"
Jul 30 03:35:17.046365 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:17.046068    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzlrv" podUID="1c2622ca-8052-489f-8d23-999618145c5a"
Jul 30 03:35:18.045518 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:18.045088    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fzz5w"
Jul 30 03:35:18.045663 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:18.045618    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fzz5w" podUID="c8447d93-b20e-44aa-a11e-3c1aec4d0ddd"
Jul 30 03:35:18.302213 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:18.302181    2576 generic.go:356] "Generic (PLEG): container finished" podID="04cb8a74-98ed-4fe2-9c4f-189783dce4da" containerID="b0b31476620ff34eb63be264fe2e1f77c7755834a6d365c541386ebcade62838" exitCode=0
Jul 30 03:35:18.303180 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:18.302267    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7n5z9" event={"ID":"04cb8a74-98ed-4fe2-9c4f-189783dce4da","Type":"ContainerDied","Data":"b0b31476620ff34eb63be264fe2e1f77c7755834a6d365c541386ebcade62838"}
Jul 30 03:35:18.307367 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:18.304356    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9jdph" event={"ID":"c50df0e7-6b07-47da-9c3c-d4328dcdcfa5","Type":"ContainerStarted","Data":"a3e6e18a1d744cefc266a2fd221aea357f2b38ebcb6d9965e0cdceec1359772d"}
Jul 30 03:35:18.311006 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:18.310967    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tl4qv" event={"ID":"b90028ac-a40a-4d9c-816a-8fec579a4b93","Type":"ContainerStarted","Data":"4aa0104e6bf7081795c81afc260a956a13a35c1d662be811483ebe964c3512a2"}
Jul 30 03:35:18.312490 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:18.312464    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mfgtq" event={"ID":"49852e08-c471-4251-9b75-5e3e7fcc189b","Type":"ContainerStarted","Data":"ab6dd9367fb618e49502937ebf09b9e765cf6a07d4d5d92da66d1b502cdd0f79"}
Jul 30 03:35:18.314030 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:18.314003    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-th2nr" event={"ID":"2e09e50f-f869-4451-9504-a0909a29ae2a","Type":"ContainerStarted","Data":"6385d2b10fd950c83820420f8563d0f03662f1ce9a96571f86423a3c7a7a35a7"}
Jul 30 03:35:18.315549 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:18.315466    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fszmc" event={"ID":"ef1fe8ef-7af9-4567-8694-5d5e9b204a70","Type":"ContainerStarted","Data":"bdf70ab8351c56b31f3418946d34a01ef42f4bc58f0fdf771d69847fb35587f7"}
Jul 30 03:35:18.317101 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:18.317081    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7bhs" event={"ID":"2ce5d0b8-bd3d-458a-a201-c2633e1af196","Type":"ContainerStarted","Data":"ead2b15db47d29537e427042f4a7d52266ed6cefc7e9a446c7f09fd9e138c673"}
Jul 30 03:35:18.320379 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:18.320361    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf" event={"ID":"62d78bc8-364a-44c0-ac71-f00f0a84623f","Type":"ContainerStarted","Data":"bb896a2f7c0a9be5abd36921f3a62c6b5f88c0580f7ad3d1d61214bee889cb88"}
Jul 30 03:35:18.320496 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:18.320455    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf" event={"ID":"62d78bc8-364a-44c0-ac71-f00f0a84623f","Type":"ContainerStarted","Data":"c72211403cd00e63ae863dd6db06e2cea6357340b4c5f69084645d3ce7344908"}
Jul 30 03:35:18.320559 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:18.320508    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf" event={"ID":"62d78bc8-364a-44c0-ac71-f00f0a84623f","Type":"ContainerStarted","Data":"d56ca38d81a7671e6933720c8b246e7a0cf5008dd0946224511af4b94c049de7"}
Jul 30 03:35:18.320612 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:18.320558    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf" event={"ID":"62d78bc8-364a-44c0-ac71-f00f0a84623f","Type":"ContainerStarted","Data":"7ec065a1a67a52f9a09bce66b45a1ad3821004c00ff329d2616d7a916e4b9034"}
Jul 30 03:35:18.320672 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:18.320621    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf" event={"ID":"62d78bc8-364a-44c0-ac71-f00f0a84623f","Type":"ContainerStarted","Data":"43d8ac53aa83c7cf3a65b81808780c99ee7e9315fb96fb0dc5d0c385e0dca7d5"}
Jul 30 03:35:18.320725 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:18.320671    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf" event={"ID":"62d78bc8-364a-44c0-ac71-f00f0a84623f","Type":"ContainerStarted","Data":"2a4a7af2bbe8f40e616f51903a7f474cbfb348f67ce1ebe2a193d46282c0248b"}
Jul 30 03:35:18.328291 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:18.328238    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal" podStartSLOduration=22.328220497 podStartE2EDuration="22.328220497s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 03:34:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:35:00.281857238 +0000 UTC m=+5.900089481" watchObservedRunningTime="2026-07-30 03:35:18.328220497 +0000 UTC m=+23.946452739"
Jul 30 03:35:18.344605 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:18.344544    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fszmc" podStartSLOduration=4.076115239 podStartE2EDuration="23.344522663s" totalImagesPullingTime="19.268407424s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 03:34:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:35:18.344299178 +0000 UTC m=+23.962531419" watchObservedRunningTime="2026-07-30 03:35:18.344522663 +0000 UTC m=+23.962754907"
Jul 30 03:35:18.361134 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:18.361072    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-th2nr" podStartSLOduration=4.053590748 podStartE2EDuration="23.361051004s" totalImagesPullingTime="19.307460256s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 03:34:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:35:18.360829858 +0000 UTC m=+23.979062124" watchObservedRunningTime="2026-07-30 03:35:18.361051004 +0000 UTC m=+23.979283246"
Jul 30 03:35:18.376001 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:18.375940    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mfgtq" podStartSLOduration=4.124945995 podStartE2EDuration="23.375905656s" totalImagesPullingTime="19.250959661s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 03:34:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:35:18.375419524 +0000 UTC m=+23.993651766" watchObservedRunningTime="2026-07-30 03:35:18.375905656 +0000 UTC m=+23.994137898"
Jul 30 03:35:18.429108 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:18.429068    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9jdph" podStartSLOduration=8.140775901 podStartE2EDuration="23.429049084s" totalImagesPullingTime="15.288273183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:34:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:35:18.398610428 +0000 UTC m=+24.016842670" watchObservedRunningTime="2026-07-30 03:35:18.429049084 +0000 UTC m=+24.047281413"
Jul 30 03:35:18.917658 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:18.917548    2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 30 03:35:19.046067 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:19.046029    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzlrv"
Jul 30 03:35:19.046287 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:19.046240    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzlrv" podUID="1c2622ca-8052-489f-8d23-999618145c5a"
Jul 30 03:35:19.046287 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:19.046261    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqrtn"
Jul 30 03:35:19.046403 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:19.046376    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqrtn" podUID="890cb2cc-a9ca-4b13-bef5-f2282e486612"
Jul 30 03:35:19.325250 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:19.325208    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mmh5j" event={"ID":"4fc32317-f442-41da-8068-42fd29b51d26","Type":"ContainerStarted","Data":"1c3c60fb0f2756e3f0175046bcf0d6f1afa5416ce4c153b521f55cc0e1c455f8"}
Jul 30 03:35:19.328110 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:19.328081    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7bhs" event={"ID":"2ce5d0b8-bd3d-458a-a201-c2633e1af196","Type":"ContainerStarted","Data":"243a4fb0d95b86284ef42fa77d6709eb8cabc56631b68a3523c96c0642be4889"}
Jul 30 03:35:19.341898 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:19.341824    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-mmh5j" podStartSLOduration=5.085657977 podStartE2EDuration="24.341806263s" totalImagesPullingTime="19.256148286s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 03:34:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:35:19.341491542 +0000 UTC m=+24.959723784" watchObservedRunningTime="2026-07-30 03:35:19.341806263 +0000 UTC m=+24.960038503"
Jul 30 03:35:19.342272 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:19.342240    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-tl4qv" podStartSLOduration=5.08142153 podStartE2EDuration="24.342228799s" totalImagesPullingTime="19.260807269s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 03:34:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:35:18.428905732 +0000 UTC m=+24.047137976" watchObservedRunningTime="2026-07-30 03:35:19.342228799 +0000 UTC m=+24.960461041"
Jul 30 03:35:19.895691 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:19.895579    2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-30T03:35:18.917573215Z","UUID":"db0c8917-bd31-43d1-bd61-972d1dd43a1b","Handler":null,"Name":"","Endpoint":""}
Jul 30 03:35:19.899777 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:19.899654    2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 30 03:35:19.899777 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:19.899709    2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 30 03:35:20.012994 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:20.012948    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9jdph"
Jul 30 03:35:20.013681 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:20.013630    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9jdph"
Jul 30 03:35:20.044975 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:20.044945    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fzz5w"
Jul 30 03:35:20.045170 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:20.045062    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fzz5w" podUID="c8447d93-b20e-44aa-a11e-3c1aec4d0ddd"
Jul 30 03:35:20.333253 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:20.332949    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7bhs" event={"ID":"2ce5d0b8-bd3d-458a-a201-c2633e1af196","Type":"ContainerStarted","Data":"a7f8db7bab4e8a1646e2c991f7bc682224a309fabf0ae56cbc4b5ed861363c3c"}
Jul 30 03:35:20.337646 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:20.337611    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf" event={"ID":"62d78bc8-364a-44c0-ac71-f00f0a84623f","Type":"ContainerStarted","Data":"2787c76d5df5558cc2f1f4a9843e5f5285ce345218a8a79f89b57a3805c4b835"}
Jul 30 03:35:20.338167 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:20.338065    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9jdph"
Jul 30 03:35:20.338533 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:20.338515    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9jdph"
Jul 30 03:35:20.351915 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:20.351860    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7bhs" podStartSLOduration=3.9514630779999997 podStartE2EDuration="25.351830423s" totalImagesPullingTime="21.400367345s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 03:34:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:35:20.351707319 +0000 UTC m=+25.969939560" watchObservedRunningTime="2026-07-30 03:35:20.351830423 +0000 UTC m=+25.970062667"
Jul 30 03:35:21.045120 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:21.045089    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqrtn"
Jul 30 03:35:21.045403 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:21.045227    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqrtn" podUID="890cb2cc-a9ca-4b13-bef5-f2282e486612"
Jul 30 03:35:21.045403 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:21.045256    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzlrv"
Jul 30 03:35:21.045403 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:21.045369    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzlrv" podUID="1c2622ca-8052-489f-8d23-999618145c5a"
Jul 30 03:35:22.045116 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:22.045085    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fzz5w"
Jul 30 03:35:22.045610 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:22.045196    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fzz5w" podUID="c8447d93-b20e-44aa-a11e-3c1aec4d0ddd"
Jul 30 03:35:23.045545 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:23.045351    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzlrv"
Jul 30 03:35:23.046177 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:23.045421    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqrtn"
Jul 30 03:35:23.046177 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:23.045610    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzlrv" podUID="1c2622ca-8052-489f-8d23-999618145c5a"
Jul 30 03:35:23.046177 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:23.045734    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqrtn" podUID="890cb2cc-a9ca-4b13-bef5-f2282e486612"
Jul 30 03:35:23.348253 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:23.348166    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf" event={"ID":"62d78bc8-364a-44c0-ac71-f00f0a84623f","Type":"ContainerStarted","Data":"7afc0197b70fa638e54b08b119d9e73f20410ec9bfeb19c3f6863ffdf6beac5c"}
Jul 30 03:35:23.348448 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:23.348422    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:35:23.348509 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:23.348502    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:35:23.348565 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:23.348552    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:35:23.350090 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:23.350061    2576 generic.go:356] "Generic (PLEG): container finished" podID="04cb8a74-98ed-4fe2-9c4f-189783dce4da" containerID="8c8e72f662d3d23aaf33d773ff4d5946f40d4cfead973c3658ae923994b5408b" exitCode=0
Jul 30 03:35:23.350208 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:23.350107    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7n5z9" event={"ID":"04cb8a74-98ed-4fe2-9c4f-189783dce4da","Type":"ContainerDied","Data":"8c8e72f662d3d23aaf33d773ff4d5946f40d4cfead973c3658ae923994b5408b"}
Jul 30 03:35:23.364749 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:23.364724    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:35:23.364890 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:23.364824    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:35:23.382214 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:23.382160    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf" podStartSLOduration=8.051173401 podStartE2EDuration="27.382145412s" totalImagesPullingTime="19.330972011s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 03:34:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:35:23.381535475 +0000 UTC m=+28.999767715" watchObservedRunningTime="2026-07-30 03:35:23.382145412 +0000 UTC m=+29.000377652"
Jul 30 03:35:24.045730 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:24.045540    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fzz5w"
Jul 30 03:35:24.046213 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:24.045825    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fzz5w" podUID="c8447d93-b20e-44aa-a11e-3c1aec4d0ddd"
Jul 30 03:35:24.760125 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:24.760035    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rqrtn"]
Jul 30 03:35:24.760282 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:24.760228    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqrtn"
Jul 30 03:35:24.760381 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:24.760357    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqrtn" podUID="890cb2cc-a9ca-4b13-bef5-f2282e486612"
Jul 30 03:35:24.762353 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:24.762330    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pzlrv"]
Jul 30 03:35:24.762483 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:24.762425    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzlrv"
Jul 30 03:35:24.762545 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:24.762515    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzlrv" podUID="1c2622ca-8052-489f-8d23-999618145c5a"
Jul 30 03:35:24.763127 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:24.763094    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fzz5w"]
Jul 30 03:35:24.763245 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:24.763179    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fzz5w"
Jul 30 03:35:24.763295 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:24.763249    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fzz5w" podUID="c8447d93-b20e-44aa-a11e-3c1aec4d0ddd"
Jul 30 03:35:24.998478 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:24.998429    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:35:25.091938 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:25.091902    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:35:25.357701 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:25.357604    2576 generic.go:356] "Generic (PLEG): container finished" podID="04cb8a74-98ed-4fe2-9c4f-189783dce4da" containerID="eb91f39c4401abfe4a10d21587eb76fc085c925ff4fe8cfad9f9fe4143f900f5" exitCode=0
Jul 30 03:35:25.357859 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:25.357692    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7n5z9" event={"ID":"04cb8a74-98ed-4fe2-9c4f-189783dce4da","Type":"ContainerDied","Data":"eb91f39c4401abfe4a10d21587eb76fc085c925ff4fe8cfad9f9fe4143f900f5"}
Jul 30 03:35:25.617701 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:25.617613    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:35:25.711358 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:25.711323    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:35:27.045829 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:27.045644    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqrtn"
Jul 30 03:35:27.046250 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:27.045644    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fzz5w"
Jul 30 03:35:27.046250 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:27.045908    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqrtn" podUID="890cb2cc-a9ca-4b13-bef5-f2282e486612"
Jul 30 03:35:27.046250 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:27.046006    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fzz5w" podUID="c8447d93-b20e-44aa-a11e-3c1aec4d0ddd"
Jul 30 03:35:27.046250 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:27.045686    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzlrv"
Jul 30 03:35:27.046250 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:27.046138    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzlrv" podUID="1c2622ca-8052-489f-8d23-999618145c5a"
Jul 30 03:35:27.363833 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:27.363743    2576 generic.go:356] "Generic (PLEG): container finished" podID="04cb8a74-98ed-4fe2-9c4f-189783dce4da" containerID="92dd47a6a9a33724a8f0989a0cf0f9195efb69f1648443503a3ce0b10c519d0f" exitCode=0
Jul 30 03:35:27.363833 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:27.363805    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7n5z9" event={"ID":"04cb8a74-98ed-4fe2-9c4f-189783dce4da","Type":"ContainerDied","Data":"92dd47a6a9a33724a8f0989a0cf0f9195efb69f1648443503a3ce0b10c519d0f"}
Jul 30 03:35:28.602029 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:28.601943    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/890cb2cc-a9ca-4b13-bef5-f2282e486612-metrics-certs\") pod \"network-metrics-daemon-rqrtn\" (UID: \"890cb2cc-a9ca-4b13-bef5-f2282e486612\") " pod="openshift-multus/network-metrics-daemon-rqrtn"
Jul 30 03:35:28.602507 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:28.602078    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz6fq\" (UniqueName: \"kubernetes.io/projected/c8447d93-b20e-44aa-a11e-3c1aec4d0ddd-kube-api-access-pz6fq\") pod \"network-check-target-fzz5w\" (UID: \"c8447d93-b20e-44aa-a11e-3c1aec4d0ddd\") " pod="openshift-network-diagnostics/network-check-target-fzz5w"
Jul 30 03:35:28.602507 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:28.602103    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 03:35:28.602507 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:28.602172    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/890cb2cc-a9ca-4b13-bef5-f2282e486612-metrics-certs podName:890cb2cc-a9ca-4b13-bef5-f2282e486612 nodeName:}" failed. No retries permitted until 2026-07-30 03:36:00.602152672 +0000 UTC m=+66.220384894 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/890cb2cc-a9ca-4b13-bef5-f2282e486612-metrics-certs") pod "network-metrics-daemon-rqrtn" (UID: "890cb2cc-a9ca-4b13-bef5-f2282e486612") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 03:35:28.602507 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:28.602235    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 03:35:28.602507 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:28.602255    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 03:35:28.602507 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:28.602266    2576 projected.go:196] Error preparing data for projected volume kube-api-access-pz6fq for pod openshift-network-diagnostics/network-check-target-fzz5w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 03:35:28.602507 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:28.602323    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c8447d93-b20e-44aa-a11e-3c1aec4d0ddd-kube-api-access-pz6fq podName:c8447d93-b20e-44aa-a11e-3c1aec4d0ddd nodeName:}" failed. No retries permitted until 2026-07-30 03:36:00.602307501 +0000 UTC m=+66.220539731 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-pz6fq" (UniqueName: "kubernetes.io/projected/c8447d93-b20e-44aa-a11e-3c1aec4d0ddd-kube-api-access-pz6fq") pod "network-check-target-fzz5w" (UID: "c8447d93-b20e-44aa-a11e-3c1aec4d0ddd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 03:35:28.702625 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:28.702583    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1c2622ca-8052-489f-8d23-999618145c5a-original-pull-secret\") pod \"global-pull-secret-syncer-pzlrv\" (UID: \"1c2622ca-8052-489f-8d23-999618145c5a\") " pod="kube-system/global-pull-secret-syncer-pzlrv"
Jul 30 03:35:28.702817 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:28.702705    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 03:35:28.702817 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:28.702780    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c2622ca-8052-489f-8d23-999618145c5a-original-pull-secret podName:1c2622ca-8052-489f-8d23-999618145c5a nodeName:}" failed. No retries permitted until 2026-07-30 03:36:00.702761289 +0000 UTC m=+66.320993508 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1c2622ca-8052-489f-8d23-999618145c5a-original-pull-secret") pod "global-pull-secret-syncer-pzlrv" (UID: "1c2622ca-8052-489f-8d23-999618145c5a") : object "kube-system"/"original-pull-secret" not registered
Jul 30 03:35:29.045423 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:29.045381    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fzz5w"
Jul 30 03:35:29.045423 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:29.045412    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqrtn"
Jul 30 03:35:29.045668 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:29.045532    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzlrv"
Jul 30 03:35:29.045668 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:29.045548    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqrtn" podUID="890cb2cc-a9ca-4b13-bef5-f2282e486612"
Jul 30 03:35:29.045668 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:29.045643    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fzz5w" podUID="c8447d93-b20e-44aa-a11e-3c1aec4d0ddd"
Jul 30 03:35:29.045790 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:29.045731    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzlrv" podUID="1c2622ca-8052-489f-8d23-999618145c5a"
Jul 30 03:35:30.179325 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:30.179232    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeReady"
Jul 30 03:35:30.179865 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:30.179393    2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 30 03:35:30.226242 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:30.226207    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-rg4t8"]
Jul 30 03:35:30.228894 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:30.228867    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rg4t8"
Jul 30 03:35:30.231548 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:30.231498    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 30 03:35:30.231713 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:30.231645    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 30 03:35:30.231713 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:30.231657    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-nmkwv\""
Jul 30 03:35:30.231821 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:30.231711    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qkjrz"]
Jul 30 03:35:30.233706 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:30.233684    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qkjrz"
Jul 30 03:35:30.235979 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:30.235953    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 30 03:35:30.236117 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:30.235998    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 30 03:35:30.236117 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:30.236029    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-pbdqz\""
Jul 30 03:35:30.236411 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:30.236253    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 30 03:35:30.241190 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:30.240103    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rg4t8"]
Jul 30 03:35:30.243323 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:30.243264    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qkjrz"]
Jul 30 03:35:30.314524 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:30.314471    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f75cd36-d556-4abe-88d6-e83b63fb3799-tmp-dir\") pod \"dns-default-rg4t8\" (UID: \"1f75cd36-d556-4abe-88d6-e83b63fb3799\") " pod="openshift-dns/dns-default-rg4t8"
Jul 30 03:35:30.314524 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:30.314528    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1f75cd36-d556-4abe-88d6-e83b63fb3799-config-volume\") pod \"dns-default-rg4t8\" (UID: \"1f75cd36-d556-4abe-88d6-e83b63fb3799\") " pod="openshift-dns/dns-default-rg4t8"
Jul 30 03:35:30.314763 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:30.314565    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46hvj\" (UniqueName: \"kubernetes.io/projected/1f75cd36-d556-4abe-88d6-e83b63fb3799-kube-api-access-46hvj\") pod \"dns-default-rg4t8\" (UID: \"1f75cd36-d556-4abe-88d6-e83b63fb3799\") " pod="openshift-dns/dns-default-rg4t8"
Jul 30 03:35:30.314763 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:30.314606    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1f75cd36-d556-4abe-88d6-e83b63fb3799-metrics-tls\") pod \"dns-default-rg4t8\" (UID: \"1f75cd36-d556-4abe-88d6-e83b63fb3799\") " pod="openshift-dns/dns-default-rg4t8"
Jul 30 03:35:30.415500 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:30.415461    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1f75cd36-d556-4abe-88d6-e83b63fb3799-metrics-tls\") pod \"dns-default-rg4t8\" (UID: \"1f75cd36-d556-4abe-88d6-e83b63fb3799\") " pod="openshift-dns/dns-default-rg4t8"
Jul 30 03:35:30.415500 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:30.415502    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2de4f068-3e70-41d6-8c64-5b6620269083-cert\") pod \"ingress-canary-qkjrz\" (UID: \"2de4f068-3e70-41d6-8c64-5b6620269083\") " pod="openshift-ingress-canary/ingress-canary-qkjrz"
Jul 30 03:35:30.415746 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:30.415562    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4kkc\" (UniqueName: \"kubernetes.io/projected/2de4f068-3e70-41d6-8c64-5b6620269083-kube-api-access-p4kkc\") pod \"ingress-canary-qkjrz\" (UID: \"2de4f068-3e70-41d6-8c64-5b6620269083\") " pod="openshift-ingress-canary/ingress-canary-qkjrz"
Jul 30 03:35:30.415746 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:30.415618    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f75cd36-d556-4abe-88d6-e83b63fb3799-tmp-dir\") pod \"dns-default-rg4t8\" (UID: \"1f75cd36-d556-4abe-88d6-e83b63fb3799\") " pod="openshift-dns/dns-default-rg4t8"
Jul 30 03:35:30.415746 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:30.415636    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 03:35:30.415746 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:30.415643    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1f75cd36-d556-4abe-88d6-e83b63fb3799-config-volume\") pod \"dns-default-rg4t8\" (UID: \"1f75cd36-d556-4abe-88d6-e83b63fb3799\") " pod="openshift-dns/dns-default-rg4t8"
Jul 30 03:35:30.415961 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:30.415799    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f75cd36-d556-4abe-88d6-e83b63fb3799-metrics-tls podName:1f75cd36-d556-4abe-88d6-e83b63fb3799 nodeName:}" failed. No retries permitted until 2026-07-30 03:35:30.915767341 +0000 UTC m=+36.533999569 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1f75cd36-d556-4abe-88d6-e83b63fb3799-metrics-tls") pod "dns-default-rg4t8" (UID: "1f75cd36-d556-4abe-88d6-e83b63fb3799") : secret "dns-default-metrics-tls" not found
Jul 30 03:35:30.416030 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:30.415998    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f75cd36-d556-4abe-88d6-e83b63fb3799-tmp-dir\") pod \"dns-default-rg4t8\" (UID: \"1f75cd36-d556-4abe-88d6-e83b63fb3799\") " pod="openshift-dns/dns-default-rg4t8"
Jul 30 03:35:30.416087 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:30.416067    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46hvj\" (UniqueName: \"kubernetes.io/projected/1f75cd36-d556-4abe-88d6-e83b63fb3799-kube-api-access-46hvj\") pod \"dns-default-rg4t8\" (UID: \"1f75cd36-d556-4abe-88d6-e83b63fb3799\") " pod="openshift-dns/dns-default-rg4t8"
Jul 30 03:35:30.416313 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:30.416291    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1f75cd36-d556-4abe-88d6-e83b63fb3799-config-volume\") pod \"dns-default-rg4t8\" (UID: \"1f75cd36-d556-4abe-88d6-e83b63fb3799\") " pod="openshift-dns/dns-default-rg4t8"
Jul 30 03:35:30.427312 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:30.427280    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46hvj\" (UniqueName: \"kubernetes.io/projected/1f75cd36-d556-4abe-88d6-e83b63fb3799-kube-api-access-46hvj\") pod \"dns-default-rg4t8\" (UID: \"1f75cd36-d556-4abe-88d6-e83b63fb3799\") " pod="openshift-dns/dns-default-rg4t8"
Jul 30 03:35:30.516462 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:30.516365    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2de4f068-3e70-41d6-8c64-5b6620269083-cert\") pod \"ingress-canary-qkjrz\" (UID: \"2de4f068-3e70-41d6-8c64-5b6620269083\") " pod="openshift-ingress-canary/ingress-canary-qkjrz"
Jul 30 03:35:30.516616 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:30.516531    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4kkc\" (UniqueName: \"kubernetes.io/projected/2de4f068-3e70-41d6-8c64-5b6620269083-kube-api-access-p4kkc\") pod \"ingress-canary-qkjrz\" (UID: \"2de4f068-3e70-41d6-8c64-5b6620269083\") " pod="openshift-ingress-canary/ingress-canary-qkjrz"
Jul 30 03:35:30.519277 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:30.519243    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2de4f068-3e70-41d6-8c64-5b6620269083-cert\") pod \"ingress-canary-qkjrz\" (UID: \"2de4f068-3e70-41d6-8c64-5b6620269083\") " pod="openshift-ingress-canary/ingress-canary-qkjrz"
Jul 30 03:35:30.527293 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:30.527263    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4kkc\" (UniqueName: \"kubernetes.io/projected/2de4f068-3e70-41d6-8c64-5b6620269083-kube-api-access-p4kkc\") pod \"ingress-canary-qkjrz\" (UID: \"2de4f068-3e70-41d6-8c64-5b6620269083\") " pod="openshift-ingress-canary/ingress-canary-qkjrz"
Jul 30 03:35:30.549352 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:30.549303    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qkjrz"
Jul 30 03:35:30.920276 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:30.920232    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1f75cd36-d556-4abe-88d6-e83b63fb3799-metrics-tls\") pod \"dns-default-rg4t8\" (UID: \"1f75cd36-d556-4abe-88d6-e83b63fb3799\") " pod="openshift-dns/dns-default-rg4t8"
Jul 30 03:35:30.920452 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:30.920391    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 03:35:30.920510 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:30.920481    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f75cd36-d556-4abe-88d6-e83b63fb3799-metrics-tls podName:1f75cd36-d556-4abe-88d6-e83b63fb3799 nodeName:}" failed. No retries permitted until 2026-07-30 03:35:31.9204583 +0000 UTC m=+37.538690524 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1f75cd36-d556-4abe-88d6-e83b63fb3799-metrics-tls") pod "dns-default-rg4t8" (UID: "1f75cd36-d556-4abe-88d6-e83b63fb3799") : secret "dns-default-metrics-tls" not found
Jul 30 03:35:31.045482 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:31.045442    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzlrv"
Jul 30 03:35:31.045670 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:31.045442    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fzz5w"
Jul 30 03:35:31.045670 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:31.045654    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqrtn"
Jul 30 03:35:31.048448 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:31.048424    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 30 03:35:31.048604 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:31.048465    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bvtgp\""
Jul 30 03:35:31.048604 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:31.048539    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 30 03:35:31.048800 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:31.048717    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 30 03:35:31.048800 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:31.048760    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jkrkk\""
Jul 30 03:35:31.049461 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:31.049440    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 30 03:35:31.076328 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:31.076287    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:35:31.176610 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:31.176509    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:35:31.504127 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:31.503961    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qkjrz"]
Jul 30 03:35:31.506993 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:35:31.506966    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2de4f068_3e70_41d6_8c64_5b6620269083.slice/crio-977213236fe2bee2438442f841ce27c368f010a9ad61c1664dbb670b3a2a0903 WatchSource:0}: Error finding container 977213236fe2bee2438442f841ce27c368f010a9ad61c1664dbb670b3a2a0903: Status 404 returned error can't find the container with id 977213236fe2bee2438442f841ce27c368f010a9ad61c1664dbb670b3a2a0903
Jul 30 03:35:31.557300 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:31.557267    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:35:31.659552 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:31.659511    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:35:31.929657 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:31.929616    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1f75cd36-d556-4abe-88d6-e83b63fb3799-metrics-tls\") pod \"dns-default-rg4t8\" (UID: \"1f75cd36-d556-4abe-88d6-e83b63fb3799\") " pod="openshift-dns/dns-default-rg4t8"
Jul 30 03:35:31.929812 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:31.929770    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 03:35:31.929876 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:31.929831    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f75cd36-d556-4abe-88d6-e83b63fb3799-metrics-tls podName:1f75cd36-d556-4abe-88d6-e83b63fb3799 nodeName:}" failed. No retries permitted until 2026-07-30 03:35:33.929815114 +0000 UTC m=+39.548047333 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1f75cd36-d556-4abe-88d6-e83b63fb3799-metrics-tls") pod "dns-default-rg4t8" (UID: "1f75cd36-d556-4abe-88d6-e83b63fb3799") : secret "dns-default-metrics-tls" not found
Jul 30 03:35:32.131423 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:32.131384    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:35:32.195611 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:32.195524    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:35:32.379205 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:32.379156    2576 generic.go:356] "Generic (PLEG): container finished" podID="04cb8a74-98ed-4fe2-9c4f-189783dce4da" containerID="77c636669e5fa7181aab63961ff4c36e30422df84786af1a630eb96da9a091e8" exitCode=0
Jul 30 03:35:32.379205 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:32.379204    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7n5z9" event={"ID":"04cb8a74-98ed-4fe2-9c4f-189783dce4da","Type":"ContainerDied","Data":"77c636669e5fa7181aab63961ff4c36e30422df84786af1a630eb96da9a091e8"}
Jul 30 03:35:32.380655 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:32.380534    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qkjrz" event={"ID":"2de4f068-3e70-41d6-8c64-5b6620269083","Type":"ContainerStarted","Data":"977213236fe2bee2438442f841ce27c368f010a9ad61c1664dbb670b3a2a0903"}
Jul 30 03:35:33.386913 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:33.386869    2576 generic.go:356] "Generic (PLEG): container finished" podID="04cb8a74-98ed-4fe2-9c4f-189783dce4da" containerID="d2f671ec0251aea37da8af55911b1255526c8641f675eb604e7087281de0a9bb" exitCode=0
Jul 30 03:35:33.387401 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:33.386946    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7n5z9" event={"ID":"04cb8a74-98ed-4fe2-9c4f-189783dce4da","Type":"ContainerDied","Data":"d2f671ec0251aea37da8af55911b1255526c8641f675eb604e7087281de0a9bb"}
Jul 30 03:35:33.944693 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:33.944580    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1f75cd36-d556-4abe-88d6-e83b63fb3799-metrics-tls\") pod \"dns-default-rg4t8\" (UID: \"1f75cd36-d556-4abe-88d6-e83b63fb3799\") " pod="openshift-dns/dns-default-rg4t8"
Jul 30 03:35:33.944860 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:33.944751    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 03:35:33.944860 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:33.944826    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f75cd36-d556-4abe-88d6-e83b63fb3799-metrics-tls podName:1f75cd36-d556-4abe-88d6-e83b63fb3799 nodeName:}" failed. No retries permitted until 2026-07-30 03:35:37.944807473 +0000 UTC m=+43.563039695 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1f75cd36-d556-4abe-88d6-e83b63fb3799-metrics-tls") pod "dns-default-rg4t8" (UID: "1f75cd36-d556-4abe-88d6-e83b63fb3799") : secret "dns-default-metrics-tls" not found
Jul 30 03:35:34.390641 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:34.390603    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qkjrz" event={"ID":"2de4f068-3e70-41d6-8c64-5b6620269083","Type":"ContainerStarted","Data":"f65c075e6aa38b9ed11c9b20a3a6e0865a22bea1f1d92f798bc125850a40574e"}
Jul 30 03:35:34.394095 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:34.394067    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7n5z9" event={"ID":"04cb8a74-98ed-4fe2-9c4f-189783dce4da","Type":"ContainerStarted","Data":"6ff6dff6c76bc418caa7a31b9fd827f35ef55822936145d2c850261881e0be30"}
Jul 30 03:35:34.408184 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:34.408103    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qkjrz" podStartSLOduration=2.172974396 podStartE2EDuration="4.408085695s" totalImagesPullingTime="2.235111299s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:35:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:35:34.407109351 +0000 UTC m=+40.025341593" watchObservedRunningTime="2026-07-30 03:35:34.408085695 +0000 UTC m=+40.026317935"
Jul 30 03:35:34.433500 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:34.433439    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7n5z9" podStartSLOduration=9.731199857 podStartE2EDuration="39.433421677s" totalImagesPullingTime="29.70222182s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 03:34:55 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:35:34.431636619 +0000 UTC m=+40.049868860" watchObservedRunningTime="2026-07-30 03:35:34.433421677 +0000 UTC m=+40.051654352"
Jul 30 03:35:34.551692 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:34.551663    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:35:34.652487 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:34.652396    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:35:34.929401 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:34.929167    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:35:34.985296 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:34.985260    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:35:37.975584 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:37.975535    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1f75cd36-d556-4abe-88d6-e83b63fb3799-metrics-tls\") pod \"dns-default-rg4t8\" (UID: \"1f75cd36-d556-4abe-88d6-e83b63fb3799\") " pod="openshift-dns/dns-default-rg4t8"
Jul 30 03:35:37.975980 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:37.975694    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 03:35:37.975980 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:37.975753    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f75cd36-d556-4abe-88d6-e83b63fb3799-metrics-tls podName:1f75cd36-d556-4abe-88d6-e83b63fb3799 nodeName:}" failed. No retries permitted until 2026-07-30 03:35:45.975739054 +0000 UTC m=+51.593971276 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1f75cd36-d556-4abe-88d6-e83b63fb3799-metrics-tls") pod "dns-default-rg4t8" (UID: "1f75cd36-d556-4abe-88d6-e83b63fb3799") : secret "dns-default-metrics-tls" not found
Jul 30 03:35:46.028091 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:46.028047    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1f75cd36-d556-4abe-88d6-e83b63fb3799-metrics-tls\") pod \"dns-default-rg4t8\" (UID: \"1f75cd36-d556-4abe-88d6-e83b63fb3799\") " pod="openshift-dns/dns-default-rg4t8"
Jul 30 03:35:46.028621 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:46.028190    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 03:35:46.028621 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:35:46.028252    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f75cd36-d556-4abe-88d6-e83b63fb3799-metrics-tls podName:1f75cd36-d556-4abe-88d6-e83b63fb3799 nodeName:}" failed. No retries permitted until 2026-07-30 03:36:02.028236459 +0000 UTC m=+67.646468681 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1f75cd36-d556-4abe-88d6-e83b63fb3799-metrics-tls") pod "dns-default-rg4t8" (UID: "1f75cd36-d556-4abe-88d6-e83b63fb3799") : secret "dns-default-metrics-tls" not found
Jul 30 03:35:46.954486 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:46.954436    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:35:47.008307 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:47.008271    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:35:55.061938 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:55.061885    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:35:55.162678 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:55.162648    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:35:55.378496 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:55.378420    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nhwmf"
Jul 30 03:35:55.497675 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:55.497641    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:35:55.599639 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:55.599602    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:35:55.702802 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:35:55.702584    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:36:00.628856 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:00.628800    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz6fq\" (UniqueName: \"kubernetes.io/projected/c8447d93-b20e-44aa-a11e-3c1aec4d0ddd-kube-api-access-pz6fq\") pod \"network-check-target-fzz5w\" (UID: \"c8447d93-b20e-44aa-a11e-3c1aec4d0ddd\") " pod="openshift-network-diagnostics/network-check-target-fzz5w"
Jul 30 03:36:00.628856 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:00.628868    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/890cb2cc-a9ca-4b13-bef5-f2282e486612-metrics-certs\") pod \"network-metrics-daemon-rqrtn\" (UID: \"890cb2cc-a9ca-4b13-bef5-f2282e486612\") " pod="openshift-multus/network-metrics-daemon-rqrtn"
Jul 30 03:36:00.631436 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:00.631419    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 30 03:36:00.631483 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:00.631460    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 30 03:36:00.639521 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:36:00.639480    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 30 03:36:00.639582 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:36:00.639566    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/890cb2cc-a9ca-4b13-bef5-f2282e486612-metrics-certs podName:890cb2cc-a9ca-4b13-bef5-f2282e486612 nodeName:}" failed. No retries permitted until 2026-07-30 03:37:04.639548907 +0000 UTC m=+130.257781125 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/890cb2cc-a9ca-4b13-bef5-f2282e486612-metrics-certs") pod "network-metrics-daemon-rqrtn" (UID: "890cb2cc-a9ca-4b13-bef5-f2282e486612") : secret "metrics-daemon-secret" not found
Jul 30 03:36:00.641573 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:00.641557    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 30 03:36:00.654372 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:00.654339    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pz6fq\" (UniqueName: \"kubernetes.io/projected/c8447d93-b20e-44aa-a11e-3c1aec4d0ddd-kube-api-access-pz6fq\") pod \"network-check-target-fzz5w\" (UID: \"c8447d93-b20e-44aa-a11e-3c1aec4d0ddd\") " pod="openshift-network-diagnostics/network-check-target-fzz5w"
Jul 30 03:36:00.729515 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:00.729470    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1c2622ca-8052-489f-8d23-999618145c5a-original-pull-secret\") pod \"global-pull-secret-syncer-pzlrv\" (UID: \"1c2622ca-8052-489f-8d23-999618145c5a\") " pod="kube-system/global-pull-secret-syncer-pzlrv"
Jul 30 03:36:00.732050 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:00.732029    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 30 03:36:00.742455 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:00.742430    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1c2622ca-8052-489f-8d23-999618145c5a-original-pull-secret\") pod \"global-pull-secret-syncer-pzlrv\" (UID: \"1c2622ca-8052-489f-8d23-999618145c5a\") " pod="kube-system/global-pull-secret-syncer-pzlrv"
Jul 30 03:36:00.758610 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:00.758570    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzlrv"
Jul 30 03:36:00.767965 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:00.767939    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jkrkk\""
Jul 30 03:36:00.776338 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:00.776301    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fzz5w"
Jul 30 03:36:00.906394 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:00.906361    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pzlrv"]
Jul 30 03:36:00.909640 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:36:00.909611    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c2622ca_8052_489f_8d23_999618145c5a.slice/crio-6f4a4ee050a6475264b021e47fc9a8e1158b36028f6dc732fd40fc57b415d112 WatchSource:0}: Error finding container 6f4a4ee050a6475264b021e47fc9a8e1158b36028f6dc732fd40fc57b415d112: Status 404 returned error can't find the container with id 6f4a4ee050a6475264b021e47fc9a8e1158b36028f6dc732fd40fc57b415d112
Jul 30 03:36:00.924390 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:00.924366    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fzz5w"]
Jul 30 03:36:00.927414 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:36:00.927391    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8447d93_b20e_44aa_a11e_3c1aec4d0ddd.slice/crio-7c4a02f85f5ff8800fc7cdff188fd3d675c0fc0562e518c9b814a4310fbbbee3 WatchSource:0}: Error finding container 7c4a02f85f5ff8800fc7cdff188fd3d675c0fc0562e518c9b814a4310fbbbee3: Status 404 returned error can't find the container with id 7c4a02f85f5ff8800fc7cdff188fd3d675c0fc0562e518c9b814a4310fbbbee3
Jul 30 03:36:01.187734 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:01.187636    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:36:01.243441 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:01.243407    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:36:01.461826 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:01.461727    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fzz5w" event={"ID":"c8447d93-b20e-44aa-a11e-3c1aec4d0ddd","Type":"ContainerStarted","Data":"7c4a02f85f5ff8800fc7cdff188fd3d675c0fc0562e518c9b814a4310fbbbee3"}
Jul 30 03:36:01.463288 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:01.463251    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pzlrv" event={"ID":"1c2622ca-8052-489f-8d23-999618145c5a","Type":"ContainerStarted","Data":"6f4a4ee050a6475264b021e47fc9a8e1158b36028f6dc732fd40fc57b415d112"}
Jul 30 03:36:02.040524 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:02.040481    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1f75cd36-d556-4abe-88d6-e83b63fb3799-metrics-tls\") pod \"dns-default-rg4t8\" (UID: \"1f75cd36-d556-4abe-88d6-e83b63fb3799\") " pod="openshift-dns/dns-default-rg4t8"
Jul 30 03:36:02.041054 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:36:02.040648    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 03:36:02.041054 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:36:02.040798    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f75cd36-d556-4abe-88d6-e83b63fb3799-metrics-tls podName:1f75cd36-d556-4abe-88d6-e83b63fb3799 nodeName:}" failed. No retries permitted until 2026-07-30 03:36:34.040776958 +0000 UTC m=+99.659009200 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1f75cd36-d556-4abe-88d6-e83b63fb3799-metrics-tls") pod "dns-default-rg4t8" (UID: "1f75cd36-d556-4abe-88d6-e83b63fb3799") : secret "dns-default-metrics-tls" not found
Jul 30 03:36:05.475305 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:05.475264    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fzz5w" event={"ID":"c8447d93-b20e-44aa-a11e-3c1aec4d0ddd","Type":"ContainerStarted","Data":"9c9f2079e33c3785a36873156479cd5464c92f32dd634f54b2c7dd46479b6281"}
Jul 30 03:36:05.475739 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:05.475367    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fzz5w"
Jul 30 03:36:05.477348 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:05.477321    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pzlrv" event={"ID":"1c2622ca-8052-489f-8d23-999618145c5a","Type":"ContainerStarted","Data":"c39445a26c0fb88ba1f1c1dc4be81434a4fcab81580e48fe3aae6e10b618dc85"}
Jul 30 03:36:05.490606 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:05.490551    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-fzz5w" podStartSLOduration=66.042204427 podStartE2EDuration="1m10.4905346s" totalImagesPullingTime="4.448330173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:34:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:36:05.490377589 +0000 UTC m=+71.108609831" watchObservedRunningTime="2026-07-30 03:36:05.4905346 +0000 UTC m=+71.108766854"
Jul 30 03:36:05.508869 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:05.508802    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-pzlrv" podStartSLOduration=65.040281908 podStartE2EDuration="1m9.508778981s" totalImagesPullingTime="4.468497073s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 03:34:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:36:05.50830856 +0000 UTC m=+71.126540800" watchObservedRunningTime="2026-07-30 03:36:05.508778981 +0000 UTC m=+71.127011222"
Jul 30 03:36:23.153201 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.153086    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f57f6567c-xxf7w"]
Jul 30 03:36:23.155609 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.155594    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-678cb85c8d-w6b2d"]
Jul 30 03:36:23.155733 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.155715    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f57f6567c-xxf7w"
Jul 30 03:36:23.157517 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.157493    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-678cb85c8d-w6b2d"
Jul 30 03:36:23.159005 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.158987    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 30 03:36:23.159121 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.159102    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 30 03:36:23.159208 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.159123    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 30 03:36:23.160456 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.160440    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 30 03:36:23.160544 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.160530    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 30 03:36:23.160601 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.160584    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 30 03:36:23.160634 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.160620    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 30 03:36:23.160670 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.160642    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-pll6d\""
Jul 30 03:36:23.160752 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.160738    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 30 03:36:23.165301 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.165281    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-678cb85c8d-w6b2d"]
Jul 30 03:36:23.171183 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.171163    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f57f6567c-xxf7w"]
Jul 30 03:36:23.183418 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.183392    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a57c996-9a44-47d1-b408-69f315b4f8f4-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5f57f6567c-xxf7w\" (UID: \"8a57c996-9a44-47d1-b408-69f315b4f8f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f57f6567c-xxf7w"
Jul 30 03:36:23.183521 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.183431    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/8a57c996-9a44-47d1-b408-69f315b4f8f4-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5f57f6567c-xxf7w\" (UID: \"8a57c996-9a44-47d1-b408-69f315b4f8f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f57f6567c-xxf7w"
Jul 30 03:36:23.183521 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.183457    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/8a57c996-9a44-47d1-b408-69f315b4f8f4-ca\") pod \"cluster-proxy-proxy-agent-5f57f6567c-xxf7w\" (UID: \"8a57c996-9a44-47d1-b408-69f315b4f8f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f57f6567c-xxf7w"
Jul 30 03:36:23.183521 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.183504    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9da28532-5a33-4def-8351-f3f70deffdcf-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-678cb85c8d-w6b2d\" (UID: \"9da28532-5a33-4def-8351-f3f70deffdcf\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-678cb85c8d-w6b2d"
Jul 30 03:36:23.183648 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.183535    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4r6b\" (UniqueName: \"kubernetes.io/projected/9da28532-5a33-4def-8351-f3f70deffdcf-kube-api-access-h4r6b\") pod \"managed-serviceaccount-addon-agent-678cb85c8d-w6b2d\" (UID: \"9da28532-5a33-4def-8351-f3f70deffdcf\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-678cb85c8d-w6b2d"
Jul 30 03:36:23.183648 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.183590    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/8a57c996-9a44-47d1-b408-69f315b4f8f4-hub\") pod \"cluster-proxy-proxy-agent-5f57f6567c-xxf7w\" (UID: \"8a57c996-9a44-47d1-b408-69f315b4f8f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f57f6567c-xxf7w"
Jul 30 03:36:23.183648 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.183611    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5z2fp\" (UniqueName: \"kubernetes.io/projected/8a57c996-9a44-47d1-b408-69f315b4f8f4-kube-api-access-5z2fp\") pod \"cluster-proxy-proxy-agent-5f57f6567c-xxf7w\" (UID: \"8a57c996-9a44-47d1-b408-69f315b4f8f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f57f6567c-xxf7w"
Jul 30 03:36:23.183766 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.183651    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8a57c996-9a44-47d1-b408-69f315b4f8f4-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5f57f6567c-xxf7w\" (UID: \"8a57c996-9a44-47d1-b408-69f315b4f8f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f57f6567c-xxf7w"
Jul 30 03:36:23.284696 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.284653    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/8a57c996-9a44-47d1-b408-69f315b4f8f4-hub\") pod \"cluster-proxy-proxy-agent-5f57f6567c-xxf7w\" (UID: \"8a57c996-9a44-47d1-b408-69f315b4f8f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f57f6567c-xxf7w"
Jul 30 03:36:23.284696 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.284697    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5z2fp\" (UniqueName: \"kubernetes.io/projected/8a57c996-9a44-47d1-b408-69f315b4f8f4-kube-api-access-5z2fp\") pod \"cluster-proxy-proxy-agent-5f57f6567c-xxf7w\" (UID: \"8a57c996-9a44-47d1-b408-69f315b4f8f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f57f6567c-xxf7w"
Jul 30 03:36:23.284910 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.284842    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8a57c996-9a44-47d1-b408-69f315b4f8f4-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5f57f6567c-xxf7w\" (UID: \"8a57c996-9a44-47d1-b408-69f315b4f8f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f57f6567c-xxf7w"
Jul 30 03:36:23.284975 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.284908    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a57c996-9a44-47d1-b408-69f315b4f8f4-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5f57f6567c-xxf7w\" (UID: \"8a57c996-9a44-47d1-b408-69f315b4f8f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f57f6567c-xxf7w"
Jul 30 03:36:23.284975 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.284957    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/8a57c996-9a44-47d1-b408-69f315b4f8f4-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5f57f6567c-xxf7w\" (UID: \"8a57c996-9a44-47d1-b408-69f315b4f8f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f57f6567c-xxf7w"
Jul 30 03:36:23.285081 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.284984    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/8a57c996-9a44-47d1-b408-69f315b4f8f4-ca\") pod \"cluster-proxy-proxy-agent-5f57f6567c-xxf7w\" (UID: \"8a57c996-9a44-47d1-b408-69f315b4f8f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f57f6567c-xxf7w"
Jul 30 03:36:23.285081 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.285012    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9da28532-5a33-4def-8351-f3f70deffdcf-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-678cb85c8d-w6b2d\" (UID: \"9da28532-5a33-4def-8351-f3f70deffdcf\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-678cb85c8d-w6b2d"
Jul 30 03:36:23.285081 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.285036    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4r6b\" (UniqueName: \"kubernetes.io/projected/9da28532-5a33-4def-8351-f3f70deffdcf-kube-api-access-h4r6b\") pod \"managed-serviceaccount-addon-agent-678cb85c8d-w6b2d\" (UID: \"9da28532-5a33-4def-8351-f3f70deffdcf\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-678cb85c8d-w6b2d"
Jul 30 03:36:23.285722 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.285692    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a57c996-9a44-47d1-b408-69f315b4f8f4-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5f57f6567c-xxf7w\" (UID: \"8a57c996-9a44-47d1-b408-69f315b4f8f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f57f6567c-xxf7w"
Jul 30 03:36:23.287406 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.287379    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/8a57c996-9a44-47d1-b408-69f315b4f8f4-ca\") pod \"cluster-proxy-proxy-agent-5f57f6567c-xxf7w\" (UID: \"8a57c996-9a44-47d1-b408-69f315b4f8f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f57f6567c-xxf7w"
Jul 30 03:36:23.287554 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.287540    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8a57c996-9a44-47d1-b408-69f315b4f8f4-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5f57f6567c-xxf7w\" (UID: \"8a57c996-9a44-47d1-b408-69f315b4f8f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f57f6567c-xxf7w"
Jul 30 03:36:23.287599 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.287564    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/8a57c996-9a44-47d1-b408-69f315b4f8f4-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5f57f6567c-xxf7w\" (UID: \"8a57c996-9a44-47d1-b408-69f315b4f8f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f57f6567c-xxf7w"
Jul 30 03:36:23.287766 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.287745    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/8a57c996-9a44-47d1-b408-69f315b4f8f4-hub\") pod \"cluster-proxy-proxy-agent-5f57f6567c-xxf7w\" (UID: \"8a57c996-9a44-47d1-b408-69f315b4f8f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f57f6567c-xxf7w"
Jul 30 03:36:23.287821 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.287808    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9da28532-5a33-4def-8351-f3f70deffdcf-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-678cb85c8d-w6b2d\" (UID: \"9da28532-5a33-4def-8351-f3f70deffdcf\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-678cb85c8d-w6b2d"
Jul 30 03:36:23.293173 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.293126    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5z2fp\" (UniqueName: \"kubernetes.io/projected/8a57c996-9a44-47d1-b408-69f315b4f8f4-kube-api-access-5z2fp\") pod \"cluster-proxy-proxy-agent-5f57f6567c-xxf7w\" (UID: \"8a57c996-9a44-47d1-b408-69f315b4f8f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f57f6567c-xxf7w"
Jul 30 03:36:23.293319 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.293224    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4r6b\" (UniqueName: \"kubernetes.io/projected/9da28532-5a33-4def-8351-f3f70deffdcf-kube-api-access-h4r6b\") pod \"managed-serviceaccount-addon-agent-678cb85c8d-w6b2d\" (UID: \"9da28532-5a33-4def-8351-f3f70deffdcf\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-678cb85c8d-w6b2d"
Jul 30 03:36:23.482857 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.482769    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f57f6567c-xxf7w"
Jul 30 03:36:23.488478 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.488453    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-678cb85c8d-w6b2d"
Jul 30 03:36:23.620025 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.619994    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f57f6567c-xxf7w"]
Jul 30 03:36:23.623473 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:36:23.623450    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8a57c996_9a44_47d1_b408_69f315b4f8f4.slice/crio-03cd3b3c598d38850ae04c8dfadddf1ba18b37cc419630865174fd6e44acaf8e WatchSource:0}: Error finding container 03cd3b3c598d38850ae04c8dfadddf1ba18b37cc419630865174fd6e44acaf8e: Status 404 returned error can't find the container with id 03cd3b3c598d38850ae04c8dfadddf1ba18b37cc419630865174fd6e44acaf8e
Jul 30 03:36:23.631439 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:23.631416    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-678cb85c8d-w6b2d"]
Jul 30 03:36:23.634047 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:36:23.634024    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9da28532_5a33_4def_8351_f3f70deffdcf.slice/crio-f9dbe524a71ccbdf11371b6295c7e450f7f70c8ee4830a7852d3ac16f4536cd4 WatchSource:0}: Error finding container f9dbe524a71ccbdf11371b6295c7e450f7f70c8ee4830a7852d3ac16f4536cd4: Status 404 returned error can't find the container with id f9dbe524a71ccbdf11371b6295c7e450f7f70c8ee4830a7852d3ac16f4536cd4
Jul 30 03:36:24.190574 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:24.190539    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:36:24.242268 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:24.242232    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:36:24.528500 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:24.528464    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-678cb85c8d-w6b2d" event={"ID":"9da28532-5a33-4def-8351-f3f70deffdcf","Type":"ContainerStarted","Data":"f9dbe524a71ccbdf11371b6295c7e450f7f70c8ee4830a7852d3ac16f4536cd4"}
Jul 30 03:36:24.530489 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:24.530435    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f57f6567c-xxf7w" event={"ID":"8a57c996-9a44-47d1-b408-69f315b4f8f4","Type":"ContainerStarted","Data":"03cd3b3c598d38850ae04c8dfadddf1ba18b37cc419630865174fd6e44acaf8e"}
Jul 30 03:36:28.542936 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:28.542890    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-678cb85c8d-w6b2d" event={"ID":"9da28532-5a33-4def-8351-f3f70deffdcf","Type":"ContainerStarted","Data":"34c0f22d08a3d9b817b559e8e26157863f716548bab3fa5aa49a30236de1b223"}
Jul 30 03:36:28.544905 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:28.544881    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f57f6567c-xxf7w" event={"ID":"8a57c996-9a44-47d1-b408-69f315b4f8f4","Type":"ContainerStarted","Data":"c5b7c80253871d4d47ec5efb022d6e63e5a06a2a340e6309baa915410480b5c3"}
Jul 30 03:36:28.545018 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:28.544916    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f57f6567c-xxf7w" event={"ID":"8a57c996-9a44-47d1-b408-69f315b4f8f4","Type":"ContainerStarted","Data":"433c03a60f23b2582cd3e951e0e192751b1f024c9b971dda0f3c9c64ad782fc9"}
Jul 30 03:36:28.545018 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:28.544942    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f57f6567c-xxf7w" event={"ID":"8a57c996-9a44-47d1-b408-69f315b4f8f4","Type":"ContainerStarted","Data":"f59f2765307a07205806c8388609c9c5ec26e3b4ae815a9925d67d7907c56559"}
Jul 30 03:36:28.557762 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:28.557719    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-678cb85c8d-w6b2d" podStartSLOduration=1.103910507 podStartE2EDuration="5.557705412s" totalImagesPullingTime="4.453794905s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:36:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:36:28.557605865 +0000 UTC m=+94.175838106" watchObservedRunningTime="2026-07-30 03:36:28.557705412 +0000 UTC m=+94.175937652"
Jul 30 03:36:28.574983 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:28.574942    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f57f6567c-xxf7w" podStartSLOduration=1.10220523 podStartE2EDuration="5.574909777s" totalImagesPullingTime="4.472704547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:36:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:36:28.573807969 +0000 UTC m=+94.192040220" watchObservedRunningTime="2026-07-30 03:36:28.574909777 +0000 UTC m=+94.193142020"
Jul 30 03:36:34.068016 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:34.067970    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1f75cd36-d556-4abe-88d6-e83b63fb3799-metrics-tls\") pod \"dns-default-rg4t8\" (UID: \"1f75cd36-d556-4abe-88d6-e83b63fb3799\") " pod="openshift-dns/dns-default-rg4t8"
Jul 30 03:36:34.068380 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:36:34.068116    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 03:36:34.068380 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:36:34.068185    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f75cd36-d556-4abe-88d6-e83b63fb3799-metrics-tls podName:1f75cd36-d556-4abe-88d6-e83b63fb3799 nodeName:}" failed. No retries permitted until 2026-07-30 03:37:38.068165263 +0000 UTC m=+163.686397492 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1f75cd36-d556-4abe-88d6-e83b63fb3799-metrics-tls") pod "dns-default-rg4t8" (UID: "1f75cd36-d556-4abe-88d6-e83b63fb3799") : secret "dns-default-metrics-tls" not found
Jul 30 03:36:36.482623 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:36:36.482589    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-fzz5w"
Jul 30 03:37:04.686080 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:37:04.686043    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/890cb2cc-a9ca-4b13-bef5-f2282e486612-metrics-certs\") pod \"network-metrics-daemon-rqrtn\" (UID: \"890cb2cc-a9ca-4b13-bef5-f2282e486612\") " pod="openshift-multus/network-metrics-daemon-rqrtn"
Jul 30 03:37:04.686651 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:37:04.686174    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 30 03:37:04.686651 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:37:04.686258    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/890cb2cc-a9ca-4b13-bef5-f2282e486612-metrics-certs podName:890cb2cc-a9ca-4b13-bef5-f2282e486612 nodeName:}" failed. No retries permitted until 2026-07-30 03:39:06.68623806 +0000 UTC m=+252.304470283 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/890cb2cc-a9ca-4b13-bef5-f2282e486612-metrics-certs") pod "network-metrics-daemon-rqrtn" (UID: "890cb2cc-a9ca-4b13-bef5-f2282e486612") : secret "metrics-daemon-secret" not found
Jul 30 03:37:33.245222 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:37:33.245179    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-rg4t8" podUID="1f75cd36-d556-4abe-88d6-e83b63fb3799"
Jul 30 03:37:33.725274 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:37:33.725249    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rg4t8"
Jul 30 03:37:34.071203 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:37:34.070811    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-rqrtn" podUID="890cb2cc-a9ca-4b13-bef5-f2282e486612"
Jul 30 03:37:38.111176 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:37:38.111137    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1f75cd36-d556-4abe-88d6-e83b63fb3799-metrics-tls\") pod \"dns-default-rg4t8\" (UID: \"1f75cd36-d556-4abe-88d6-e83b63fb3799\") " pod="openshift-dns/dns-default-rg4t8"
Jul 30 03:37:38.111579 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:37:38.111296    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 03:37:38.111579 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:37:38.111369    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f75cd36-d556-4abe-88d6-e83b63fb3799-metrics-tls podName:1f75cd36-d556-4abe-88d6-e83b63fb3799 nodeName:}" failed. No retries permitted until 2026-07-30 03:39:40.11135472 +0000 UTC m=+285.729586942 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1f75cd36-d556-4abe-88d6-e83b63fb3799-metrics-tls") pod "dns-default-rg4t8" (UID: "1f75cd36-d556-4abe-88d6-e83b63fb3799") : secret "dns-default-metrics-tls" not found
Jul 30 03:37:46.045375 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:37:46.045326    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqrtn"
Jul 30 03:38:24.743620 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:24.743594    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:38:28.876028 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:28.875995    2576 generic.go:356] "Generic (PLEG): container finished" podID="9da28532-5a33-4def-8351-f3f70deffdcf" containerID="34c0f22d08a3d9b817b559e8e26157863f716548bab3fa5aa49a30236de1b223" exitCode=255
Jul 30 03:38:28.876409 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:28.876055    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-678cb85c8d-w6b2d" event={"ID":"9da28532-5a33-4def-8351-f3f70deffdcf","Type":"ContainerDied","Data":"34c0f22d08a3d9b817b559e8e26157863f716548bab3fa5aa49a30236de1b223"}
Jul 30 03:38:28.876409 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:28.876327    2576 scope.go:117] "RemoveContainer" containerID="34c0f22d08a3d9b817b559e8e26157863f716548bab3fa5aa49a30236de1b223"
Jul 30 03:38:29.880913 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:29.880878    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-678cb85c8d-w6b2d" event={"ID":"9da28532-5a33-4def-8351-f3f70deffdcf","Type":"ContainerStarted","Data":"4fe3c0d9e211e97ec14838fc5ceb7577439ffd1a8199f0f3fcd864a3b7a5b92f"}
Jul 30 03:38:52.326489 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:52.326459    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:38:52.428011 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:52.427985    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:38:54.526303 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:54.526269    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:38:54.578081 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:54.578050    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:38:56.508198 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:56.508170    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:38:56.601009 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:56.600983    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:38:57.207098 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:57.207064    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:38:57.308736 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:57.308706    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:38:57.675539 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:57.675507    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-wvzzm"]
Jul 30 03:38:57.677405 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:57.677385    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wvzzm"
Jul 30 03:38:57.680812 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:57.680797    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 30 03:38:57.681158 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:57.681139    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 30 03:38:57.681273 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:57.681145    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 30 03:38:57.681524 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:57.681508    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 30 03:38:57.682546 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:57.682524    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-97fcc\""
Jul 30 03:38:57.711199 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:57.711167    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wvzzm"]
Jul 30 03:38:57.714111 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:57.714068    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:38:57.848575 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:57.848544    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/30a35c99-75dd-4268-8446-9caec1370557-data-volume\") pod \"insights-runtime-extractor-wvzzm\" (UID: \"30a35c99-75dd-4268-8446-9caec1370557\") " pod="openshift-insights/insights-runtime-extractor-wvzzm"
Jul 30 03:38:57.848575 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:57.848576    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/30a35c99-75dd-4268-8446-9caec1370557-crio-socket\") pod \"insights-runtime-extractor-wvzzm\" (UID: \"30a35c99-75dd-4268-8446-9caec1370557\") " pod="openshift-insights/insights-runtime-extractor-wvzzm"
Jul 30 03:38:57.848807 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:57.848594    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blxv7\" (UniqueName: \"kubernetes.io/projected/30a35c99-75dd-4268-8446-9caec1370557-kube-api-access-blxv7\") pod \"insights-runtime-extractor-wvzzm\" (UID: \"30a35c99-75dd-4268-8446-9caec1370557\") " pod="openshift-insights/insights-runtime-extractor-wvzzm"
Jul 30 03:38:57.848807 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:57.848717    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/30a35c99-75dd-4268-8446-9caec1370557-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wvzzm\" (UID: \"30a35c99-75dd-4268-8446-9caec1370557\") " pod="openshift-insights/insights-runtime-extractor-wvzzm"
Jul 30 03:38:57.848807 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:57.848786    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/30a35c99-75dd-4268-8446-9caec1370557-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wvzzm\" (UID: \"30a35c99-75dd-4268-8446-9caec1370557\") " pod="openshift-insights/insights-runtime-extractor-wvzzm"
Jul 30 03:38:57.949326 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:57.949253    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/30a35c99-75dd-4268-8446-9caec1370557-data-volume\") pod \"insights-runtime-extractor-wvzzm\" (UID: \"30a35c99-75dd-4268-8446-9caec1370557\") " pod="openshift-insights/insights-runtime-extractor-wvzzm"
Jul 30 03:38:57.949326 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:57.949292    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/30a35c99-75dd-4268-8446-9caec1370557-crio-socket\") pod \"insights-runtime-extractor-wvzzm\" (UID: \"30a35c99-75dd-4268-8446-9caec1370557\") " pod="openshift-insights/insights-runtime-extractor-wvzzm"
Jul 30 03:38:57.949326 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:57.949316    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blxv7\" (UniqueName: \"kubernetes.io/projected/30a35c99-75dd-4268-8446-9caec1370557-kube-api-access-blxv7\") pod \"insights-runtime-extractor-wvzzm\" (UID: \"30a35c99-75dd-4268-8446-9caec1370557\") " pod="openshift-insights/insights-runtime-extractor-wvzzm"
Jul 30 03:38:57.949623 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:57.949382    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/30a35c99-75dd-4268-8446-9caec1370557-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wvzzm\" (UID: \"30a35c99-75dd-4268-8446-9caec1370557\") " pod="openshift-insights/insights-runtime-extractor-wvzzm"
Jul 30 03:38:57.949623 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:57.949387    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/30a35c99-75dd-4268-8446-9caec1370557-crio-socket\") pod \"insights-runtime-extractor-wvzzm\" (UID: \"30a35c99-75dd-4268-8446-9caec1370557\") " pod="openshift-insights/insights-runtime-extractor-wvzzm"
Jul 30 03:38:57.949623 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:57.949434    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/30a35c99-75dd-4268-8446-9caec1370557-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wvzzm\" (UID: \"30a35c99-75dd-4268-8446-9caec1370557\") " pod="openshift-insights/insights-runtime-extractor-wvzzm"
Jul 30 03:38:57.949623 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:38:57.949495    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 03:38:57.949623 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:38:57.949526    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 03:38:57.949623 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:38:57.949569    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/30a35c99-75dd-4268-8446-9caec1370557-kube-rbac-proxy-cm podName:30a35c99-75dd-4268-8446-9caec1370557 nodeName:}" failed. No retries permitted until 2026-07-30 03:38:58.449550229 +0000 UTC m=+244.067782467 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/30a35c99-75dd-4268-8446-9caec1370557-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wvzzm" (UID: "30a35c99-75dd-4268-8446-9caec1370557") : configmap "kube-rbac-proxy" not found
Jul 30 03:38:57.949623 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:38:57.949595    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30a35c99-75dd-4268-8446-9caec1370557-insights-runtime-extractor-tls podName:30a35c99-75dd-4268-8446-9caec1370557 nodeName:}" failed. No retries permitted until 2026-07-30 03:38:58.449576746 +0000 UTC m=+244.067808976 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/30a35c99-75dd-4268-8446-9caec1370557-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wvzzm" (UID: "30a35c99-75dd-4268-8446-9caec1370557") : secret "insights-runtime-extractor-tls" not found
Jul 30 03:38:57.949623 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:57.949603    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/30a35c99-75dd-4268-8446-9caec1370557-data-volume\") pod \"insights-runtime-extractor-wvzzm\" (UID: \"30a35c99-75dd-4268-8446-9caec1370557\") " pod="openshift-insights/insights-runtime-extractor-wvzzm"
Jul 30 03:38:57.965021 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:57.964998    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blxv7\" (UniqueName: \"kubernetes.io/projected/30a35c99-75dd-4268-8446-9caec1370557-kube-api-access-blxv7\") pod \"insights-runtime-extractor-wvzzm\" (UID: \"30a35c99-75dd-4268-8446-9caec1370557\") " pod="openshift-insights/insights-runtime-extractor-wvzzm"
Jul 30 03:38:57.978284 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:57.978262    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:38:58.078386 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:58.078366    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:38:58.452755 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:58.452725    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/30a35c99-75dd-4268-8446-9caec1370557-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wvzzm\" (UID: \"30a35c99-75dd-4268-8446-9caec1370557\") " pod="openshift-insights/insights-runtime-extractor-wvzzm"
Jul 30 03:38:58.452755 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:58.452760    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/30a35c99-75dd-4268-8446-9caec1370557-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wvzzm\" (UID: \"30a35c99-75dd-4268-8446-9caec1370557\") " pod="openshift-insights/insights-runtime-extractor-wvzzm"
Jul 30 03:38:58.452970 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:38:58.452838    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 03:38:58.452970 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:38:58.452854    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 03:38:58.452970 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:38:58.452897    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30a35c99-75dd-4268-8446-9caec1370557-insights-runtime-extractor-tls podName:30a35c99-75dd-4268-8446-9caec1370557 nodeName:}" failed. No retries permitted until 2026-07-30 03:38:59.452882773 +0000 UTC m=+245.071114992 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/30a35c99-75dd-4268-8446-9caec1370557-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wvzzm" (UID: "30a35c99-75dd-4268-8446-9caec1370557") : secret "insights-runtime-extractor-tls" not found
Jul 30 03:38:58.452970 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:38:58.452909    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/30a35c99-75dd-4268-8446-9caec1370557-kube-rbac-proxy-cm podName:30a35c99-75dd-4268-8446-9caec1370557 nodeName:}" failed. No retries permitted until 2026-07-30 03:38:59.452902308 +0000 UTC m=+245.071134527 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/30a35c99-75dd-4268-8446-9caec1370557-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wvzzm" (UID: "30a35c99-75dd-4268-8446-9caec1370557") : configmap "kube-rbac-proxy" not found
Jul 30 03:38:58.842170 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:58.842136    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:38:58.907646 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:58.907618    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:38:59.008792 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:59.008749    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:38:59.061943 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:59.061901    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:38:59.459045 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:59.459017    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/30a35c99-75dd-4268-8446-9caec1370557-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wvzzm\" (UID: \"30a35c99-75dd-4268-8446-9caec1370557\") " pod="openshift-insights/insights-runtime-extractor-wvzzm"
Jul 30 03:38:59.459249 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:59.459053    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/30a35c99-75dd-4268-8446-9caec1370557-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wvzzm\" (UID: \"30a35c99-75dd-4268-8446-9caec1370557\") " pod="openshift-insights/insights-runtime-extractor-wvzzm"
Jul 30 03:38:59.459249 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:38:59.459144    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 03:38:59.459249 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:38:59.459152    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 03:38:59.459249 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:38:59.459194    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30a35c99-75dd-4268-8446-9caec1370557-insights-runtime-extractor-tls podName:30a35c99-75dd-4268-8446-9caec1370557 nodeName:}" failed. No retries permitted until 2026-07-30 03:39:01.459181538 +0000 UTC m=+247.077413756 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/30a35c99-75dd-4268-8446-9caec1370557-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wvzzm" (UID: "30a35c99-75dd-4268-8446-9caec1370557") : secret "insights-runtime-extractor-tls" not found
Jul 30 03:38:59.459249 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:38:59.459207    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/30a35c99-75dd-4268-8446-9caec1370557-kube-rbac-proxy-cm podName:30a35c99-75dd-4268-8446-9caec1370557 nodeName:}" failed. No retries permitted until 2026-07-30 03:39:01.459201191 +0000 UTC m=+247.077433409 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/30a35c99-75dd-4268-8446-9caec1370557-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wvzzm" (UID: "30a35c99-75dd-4268-8446-9caec1370557") : configmap "kube-rbac-proxy" not found
Jul 30 03:38:59.659025 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:59.658994    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mfgtq_49852e08-c471-4251-9b75-5e3e7fcc189b/dns-node-resolver/0.log"
Jul 30 03:38:59.731309 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:38:59.731217    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:00.857688 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:00.857661    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tl4qv_b90028ac-a40a-4d9c-816a-8fec579a4b93/node-ca/0.log"
Jul 30 03:39:01.086307 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:01.086276    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:01.205085 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:01.204995    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:01.304896 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:01.304867    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:01.390630 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:01.390602    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:01.474809 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:01.474739    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/30a35c99-75dd-4268-8446-9caec1370557-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wvzzm\" (UID: \"30a35c99-75dd-4268-8446-9caec1370557\") " pod="openshift-insights/insights-runtime-extractor-wvzzm"
Jul 30 03:39:01.474809 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:01.474774    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/30a35c99-75dd-4268-8446-9caec1370557-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wvzzm\" (UID: \"30a35c99-75dd-4268-8446-9caec1370557\") " pod="openshift-insights/insights-runtime-extractor-wvzzm"
Jul 30 03:39:01.474993 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:39:01.474857    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 03:39:01.474993 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:39:01.474886    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 03:39:01.474993 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:39:01.474949    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/30a35c99-75dd-4268-8446-9caec1370557-kube-rbac-proxy-cm podName:30a35c99-75dd-4268-8446-9caec1370557 nodeName:}" failed. No retries permitted until 2026-07-30 03:39:05.47491239 +0000 UTC m=+251.093144623 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/30a35c99-75dd-4268-8446-9caec1370557-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wvzzm" (UID: "30a35c99-75dd-4268-8446-9caec1370557") : configmap "kube-rbac-proxy" not found
Jul 30 03:39:01.474993 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:39:01.474967    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30a35c99-75dd-4268-8446-9caec1370557-insights-runtime-extractor-tls podName:30a35c99-75dd-4268-8446-9caec1370557 nodeName:}" failed. No retries permitted until 2026-07-30 03:39:05.474958449 +0000 UTC m=+251.093190667 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/30a35c99-75dd-4268-8446-9caec1370557-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wvzzm" (UID: "30a35c99-75dd-4268-8446-9caec1370557") : secret "insights-runtime-extractor-tls" not found
Jul 30 03:39:01.496075 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:01.496036    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:01.592801 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:01.592771    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:01.656524 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:01.656505    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qkjrz_2de4f068-3e70-41d6-8c64-5b6620269083/serve-healthcheck-canary/0.log"
Jul 30 03:39:01.778807 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:01.778787    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:02.279548 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:02.279518    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:02.773823 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:02.773784    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:03.174086 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:03.174015    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:03.376464 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:03.376438    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:03.577672 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:03.577642    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:03.779939 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:03.779892    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:04.684542 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:04.684512    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:04.966843 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:04.966748    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:05.019111 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:05.019080    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:05.180674 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:05.180645    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:05.401671 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:05.401643    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:05.506864 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:05.506825    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/30a35c99-75dd-4268-8446-9caec1370557-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wvzzm\" (UID: \"30a35c99-75dd-4268-8446-9caec1370557\") " pod="openshift-insights/insights-runtime-extractor-wvzzm"
Jul 30 03:39:05.506864 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:05.506870    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/30a35c99-75dd-4268-8446-9caec1370557-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wvzzm\" (UID: \"30a35c99-75dd-4268-8446-9caec1370557\") " pod="openshift-insights/insights-runtime-extractor-wvzzm"
Jul 30 03:39:05.507071 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:39:05.506972    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 03:39:05.507071 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:39:05.507042    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/30a35c99-75dd-4268-8446-9caec1370557-kube-rbac-proxy-cm podName:30a35c99-75dd-4268-8446-9caec1370557 nodeName:}" failed. No retries permitted until 2026-07-30 03:39:13.507023248 +0000 UTC m=+259.125255484 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/30a35c99-75dd-4268-8446-9caec1370557-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wvzzm" (UID: "30a35c99-75dd-4268-8446-9caec1370557") : configmap "kube-rbac-proxy" not found
Jul 30 03:39:05.507071 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:39:05.507048    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 03:39:05.507177 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:39:05.507087    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30a35c99-75dd-4268-8446-9caec1370557-insights-runtime-extractor-tls podName:30a35c99-75dd-4268-8446-9caec1370557 nodeName:}" failed. No retries permitted until 2026-07-30 03:39:13.507075843 +0000 UTC m=+259.125308069 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/30a35c99-75dd-4268-8446-9caec1370557-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wvzzm" (UID: "30a35c99-75dd-4268-8446-9caec1370557") : secret "insights-runtime-extractor-tls" not found
Jul 30 03:39:05.577162 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:05.577125    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:05.778490 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:05.778454    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:06.720194 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:06.720158    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/890cb2cc-a9ca-4b13-bef5-f2282e486612-metrics-certs\") pod \"network-metrics-daemon-rqrtn\" (UID: \"890cb2cc-a9ca-4b13-bef5-f2282e486612\") " pod="openshift-multus/network-metrics-daemon-rqrtn"
Jul 30 03:39:06.720367 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:39:06.720301    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 30 03:39:06.720367 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:39:06.720360    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/890cb2cc-a9ca-4b13-bef5-f2282e486612-metrics-certs podName:890cb2cc-a9ca-4b13-bef5-f2282e486612 nodeName:}" failed. No retries permitted until 2026-07-30 03:41:08.720344558 +0000 UTC m=+374.338576777 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/890cb2cc-a9ca-4b13-bef5-f2282e486612-metrics-certs") pod "network-metrics-daemon-rqrtn" (UID: "890cb2cc-a9ca-4b13-bef5-f2282e486612") : secret "metrics-daemon-secret" not found
Jul 30 03:39:06.980640 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:06.980555    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:07.175534 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:07.175494    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:07.375726 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:07.375688    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:07.578721 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:07.578690    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:07.632783 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:07.632711    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:07.780881 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:07.780843    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:08.576637 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:08.576612    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:08.974568 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:08.974484    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:09.376375 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:09.376339    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:09.575731 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:09.575702    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:09.767439 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:09.767362    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:09.971166 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:09.971134    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:10.772519 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:10.772492    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:10.989424 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:10.989370    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:11.369633 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:11.369608    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:11.574568 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:11.574541    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:11.687970 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:11.687905    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:11.780968 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:11.780939    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:13.564442 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:13.564397    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/30a35c99-75dd-4268-8446-9caec1370557-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wvzzm\" (UID: \"30a35c99-75dd-4268-8446-9caec1370557\") " pod="openshift-insights/insights-runtime-extractor-wvzzm"
Jul 30 03:39:13.564442 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:13.564446    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/30a35c99-75dd-4268-8446-9caec1370557-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wvzzm\" (UID: \"30a35c99-75dd-4268-8446-9caec1370557\") " pod="openshift-insights/insights-runtime-extractor-wvzzm"
Jul 30 03:39:13.565010 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:13.564994    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/30a35c99-75dd-4268-8446-9caec1370557-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wvzzm\" (UID: \"30a35c99-75dd-4268-8446-9caec1370557\") " pod="openshift-insights/insights-runtime-extractor-wvzzm"
Jul 30 03:39:13.566690 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:13.566671    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/30a35c99-75dd-4268-8446-9caec1370557-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wvzzm\" (UID: \"30a35c99-75dd-4268-8446-9caec1370557\") " pod="openshift-insights/insights-runtime-extractor-wvzzm"
Jul 30 03:39:13.586678 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:13.586654    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wvzzm"
Jul 30 03:39:13.710521 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:13.710490    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wvzzm"]
Jul 30 03:39:13.714589 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:39:13.714564    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30a35c99_75dd_4268_8446_9caec1370557.slice/crio-010b44007660102f7fc8a9504522736ae5d22d9999b617f4f33f2a1539f0a4a3 WatchSource:0}: Error finding container 010b44007660102f7fc8a9504522736ae5d22d9999b617f4f33f2a1539f0a4a3: Status 404 returned error can't find the container with id 010b44007660102f7fc8a9504522736ae5d22d9999b617f4f33f2a1539f0a4a3
Jul 30 03:39:14.002750 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:14.002673    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wvzzm" event={"ID":"30a35c99-75dd-4268-8446-9caec1370557","Type":"ContainerStarted","Data":"010b44007660102f7fc8a9504522736ae5d22d9999b617f4f33f2a1539f0a4a3"}
Jul 30 03:39:16.553008 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:16.552879    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:16.610474 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:16.610442    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:17.029735 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:17.029702    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:17.127881 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:17.127837    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:19.020347 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:19.020312    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wvzzm" event={"ID":"30a35c99-75dd-4268-8446-9caec1370557","Type":"ContainerStarted","Data":"bf506beafaade4e1d6843d82d825beb6b339691b0c68afcb2492947d2258fa40"}
Jul 30 03:39:22.030311 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:22.030280    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wvzzm" event={"ID":"30a35c99-75dd-4268-8446-9caec1370557","Type":"ContainerStarted","Data":"7a381468a1710e5fa0fabeba43ab150cdb633cf036c87e3bbeee26502b742657"}
Jul 30 03:39:26.043996 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:26.043956    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wvzzm" event={"ID":"30a35c99-75dd-4268-8446-9caec1370557","Type":"ContainerStarted","Data":"a8b34d0b164195b24accfd229d8ec8c33e318b857130233121c6e3662b763850"}
Jul 30 03:39:26.062133 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:26.062093    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-wvzzm" podStartSLOduration=17.660883808 podStartE2EDuration="29.062078495s" totalImagesPullingTime="11.401194687s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 03:38:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:39:26.060707347 +0000 UTC m=+271.678939587" watchObservedRunningTime="2026-07-30 03:39:26.062078495 +0000 UTC m=+271.680310735"
Jul 30 03:39:27.781854 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:27.781824    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:27.880139 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:27.880111    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:27.951450 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:27.951418    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:28.126856 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:28.126779    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:30.974159 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:30.974125    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:31.044506 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:31.044474    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:31.318796 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:31.318764    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:31.551024 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:31.550995    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:31.600732 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:31.600662    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:33.605042 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:33.605011    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:33.705144 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:33.705111    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:33.919132 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:33.919022    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:35.641857 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:35.641827    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:35.739769 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:35.739741    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:35.810613 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:35.810572    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:35.920364 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:35.920298    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:36.138608 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.138580    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:36.238896 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.238802    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:36.368357 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.368315    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:36.425688 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.425664    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mxscm"]
Jul 30 03:39:36.428591 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.428559    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mxscm"
Jul 30 03:39:36.431045 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.431027    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 30 03:39:36.432615 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.432602    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-6j2hf\""
Jul 30 03:39:36.433341 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.432802    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 30 03:39:36.433341 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.433106    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 30 03:39:36.433341 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.433228    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 30 03:39:36.434793 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.433614    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 30 03:39:36.434793 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.433742    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 30 03:39:36.455539 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.455513    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:36.521109 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.521078    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:36.533546 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.533521    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/00de1d13-af77-4a4d-b18a-d94bd36f78b7-node-exporter-textfile\") pod \"node-exporter-mxscm\" (UID: \"00de1d13-af77-4a4d-b18a-d94bd36f78b7\") " pod="openshift-monitoring/node-exporter-mxscm"
Jul 30 03:39:36.533629 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.533566    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00de1d13-af77-4a4d-b18a-d94bd36f78b7-metrics-client-ca\") pod \"node-exporter-mxscm\" (UID: \"00de1d13-af77-4a4d-b18a-d94bd36f78b7\") " pod="openshift-monitoring/node-exporter-mxscm"
Jul 30 03:39:36.533678 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.533622    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/00de1d13-af77-4a4d-b18a-d94bd36f78b7-sys\") pod \"node-exporter-mxscm\" (UID: \"00de1d13-af77-4a4d-b18a-d94bd36f78b7\") " pod="openshift-monitoring/node-exporter-mxscm"
Jul 30 03:39:36.533678 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.533655    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/00de1d13-af77-4a4d-b18a-d94bd36f78b7-node-exporter-accelerators-collector-config\") pod \"node-exporter-mxscm\" (UID: \"00de1d13-af77-4a4d-b18a-d94bd36f78b7\") " pod="openshift-monitoring/node-exporter-mxscm"
Jul 30 03:39:36.533784 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.533733    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/00de1d13-af77-4a4d-b18a-d94bd36f78b7-node-exporter-tls\") pod \"node-exporter-mxscm\" (UID: \"00de1d13-af77-4a4d-b18a-d94bd36f78b7\") " pod="openshift-monitoring/node-exporter-mxscm"
Jul 30 03:39:36.533784 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.533764    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/00de1d13-af77-4a4d-b18a-d94bd36f78b7-root\") pod \"node-exporter-mxscm\" (UID: \"00de1d13-af77-4a4d-b18a-d94bd36f78b7\") " pod="openshift-monitoring/node-exporter-mxscm"
Jul 30 03:39:36.533881 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.533791    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/00de1d13-af77-4a4d-b18a-d94bd36f78b7-node-exporter-wtmp\") pod \"node-exporter-mxscm\" (UID: \"00de1d13-af77-4a4d-b18a-d94bd36f78b7\") " pod="openshift-monitoring/node-exporter-mxscm"
Jul 30 03:39:36.533881 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.533819    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcwgx\" (UniqueName: \"kubernetes.io/projected/00de1d13-af77-4a4d-b18a-d94bd36f78b7-kube-api-access-pcwgx\") pod \"node-exporter-mxscm\" (UID: \"00de1d13-af77-4a4d-b18a-d94bd36f78b7\") " pod="openshift-monitoring/node-exporter-mxscm"
Jul 30 03:39:36.533881 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.533858    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/00de1d13-af77-4a4d-b18a-d94bd36f78b7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mxscm\" (UID: \"00de1d13-af77-4a4d-b18a-d94bd36f78b7\") " pod="openshift-monitoring/node-exporter-mxscm"
Jul 30 03:39:36.620746 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.620715    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:36.634877 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.634850    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/00de1d13-af77-4a4d-b18a-d94bd36f78b7-sys\") pod \"node-exporter-mxscm\" (UID: \"00de1d13-af77-4a4d-b18a-d94bd36f78b7\") " pod="openshift-monitoring/node-exporter-mxscm"
Jul 30 03:39:36.635035 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.634883    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/00de1d13-af77-4a4d-b18a-d94bd36f78b7-node-exporter-accelerators-collector-config\") pod \"node-exporter-mxscm\" (UID: \"00de1d13-af77-4a4d-b18a-d94bd36f78b7\") " pod="openshift-monitoring/node-exporter-mxscm"
Jul 30 03:39:36.635035 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.634943    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/00de1d13-af77-4a4d-b18a-d94bd36f78b7-node-exporter-tls\") pod \"node-exporter-mxscm\" (UID: \"00de1d13-af77-4a4d-b18a-d94bd36f78b7\") " pod="openshift-monitoring/node-exporter-mxscm"
Jul 30 03:39:36.635035 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.634965    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/00de1d13-af77-4a4d-b18a-d94bd36f78b7-root\") pod \"node-exporter-mxscm\" (UID: \"00de1d13-af77-4a4d-b18a-d94bd36f78b7\") " pod="openshift-monitoring/node-exporter-mxscm"
Jul 30 03:39:36.635035 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.634969    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/00de1d13-af77-4a4d-b18a-d94bd36f78b7-sys\") pod \"node-exporter-mxscm\" (UID: \"00de1d13-af77-4a4d-b18a-d94bd36f78b7\") " pod="openshift-monitoring/node-exporter-mxscm"
Jul 30 03:39:36.635035 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.634983    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/00de1d13-af77-4a4d-b18a-d94bd36f78b7-node-exporter-wtmp\") pod \"node-exporter-mxscm\" (UID: \"00de1d13-af77-4a4d-b18a-d94bd36f78b7\") " pod="openshift-monitoring/node-exporter-mxscm"
Jul 30 03:39:36.635035 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.634999    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcwgx\" (UniqueName: \"kubernetes.io/projected/00de1d13-af77-4a4d-b18a-d94bd36f78b7-kube-api-access-pcwgx\") pod \"node-exporter-mxscm\" (UID: \"00de1d13-af77-4a4d-b18a-d94bd36f78b7\") " pod="openshift-monitoring/node-exporter-mxscm"
Jul 30 03:39:36.635035 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.635017    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/00de1d13-af77-4a4d-b18a-d94bd36f78b7-root\") pod \"node-exporter-mxscm\" (UID: \"00de1d13-af77-4a4d-b18a-d94bd36f78b7\") " pod="openshift-monitoring/node-exporter-mxscm"
Jul 30 03:39:36.635035 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.635029    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/00de1d13-af77-4a4d-b18a-d94bd36f78b7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mxscm\" (UID: \"00de1d13-af77-4a4d-b18a-d94bd36f78b7\") " pod="openshift-monitoring/node-exporter-mxscm"
Jul 30 03:39:36.635462 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.635063    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/00de1d13-af77-4a4d-b18a-d94bd36f78b7-node-exporter-textfile\") pod \"node-exporter-mxscm\" (UID: \"00de1d13-af77-4a4d-b18a-d94bd36f78b7\") " pod="openshift-monitoring/node-exporter-mxscm"
Jul 30 03:39:36.635462 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.635105    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00de1d13-af77-4a4d-b18a-d94bd36f78b7-metrics-client-ca\") pod \"node-exporter-mxscm\" (UID: \"00de1d13-af77-4a4d-b18a-d94bd36f78b7\") " pod="openshift-monitoring/node-exporter-mxscm"
Jul 30 03:39:36.635462 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.635130    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/00de1d13-af77-4a4d-b18a-d94bd36f78b7-node-exporter-wtmp\") pod \"node-exporter-mxscm\" (UID: \"00de1d13-af77-4a4d-b18a-d94bd36f78b7\") " pod="openshift-monitoring/node-exporter-mxscm"
Jul 30 03:39:36.635620 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.635498    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/00de1d13-af77-4a4d-b18a-d94bd36f78b7-node-exporter-textfile\") pod \"node-exporter-mxscm\" (UID: \"00de1d13-af77-4a4d-b18a-d94bd36f78b7\") " pod="openshift-monitoring/node-exporter-mxscm"
Jul 30 03:39:36.635677 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.635624    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/00de1d13-af77-4a4d-b18a-d94bd36f78b7-node-exporter-accelerators-collector-config\") pod \"node-exporter-mxscm\" (UID: \"00de1d13-af77-4a4d-b18a-d94bd36f78b7\") " pod="openshift-monitoring/node-exporter-mxscm"
Jul 30 03:39:36.635735 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.635716    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00de1d13-af77-4a4d-b18a-d94bd36f78b7-metrics-client-ca\") pod \"node-exporter-mxscm\" (UID: \"00de1d13-af77-4a4d-b18a-d94bd36f78b7\") " pod="openshift-monitoring/node-exporter-mxscm"
Jul 30 03:39:36.637315 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.637295    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/00de1d13-af77-4a4d-b18a-d94bd36f78b7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mxscm\" (UID: \"00de1d13-af77-4a4d-b18a-d94bd36f78b7\") " pod="openshift-monitoring/node-exporter-mxscm"
Jul 30 03:39:36.637523 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.637506    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/00de1d13-af77-4a4d-b18a-d94bd36f78b7-node-exporter-tls\") pod \"node-exporter-mxscm\" (UID: \"00de1d13-af77-4a4d-b18a-d94bd36f78b7\") " pod="openshift-monitoring/node-exporter-mxscm"
Jul 30 03:39:36.643217 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.643195    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcwgx\" (UniqueName: \"kubernetes.io/projected/00de1d13-af77-4a4d-b18a-d94bd36f78b7-kube-api-access-pcwgx\") pod \"node-exporter-mxscm\" (UID: \"00de1d13-af77-4a4d-b18a-d94bd36f78b7\") " pod="openshift-monitoring/node-exporter-mxscm"
Jul 30 03:39:36.721911 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.721876    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:36.725933 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:39:36.725898    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-rg4t8" podUID="1f75cd36-d556-4abe-88d6-e83b63fb3799"
Jul 30 03:39:36.739094 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.739074    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mxscm"
Jul 30 03:39:36.748191 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:39:36.748164    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00de1d13_af77_4a4d_b18a_d94bd36f78b7.slice/crio-289d61008f27144d43c326c129c6f2fe4069fa73277d7f58e5a897180cb9bb63 WatchSource:0}: Error finding container 289d61008f27144d43c326c129c6f2fe4069fa73277d7f58e5a897180cb9bb63: Status 404 returned error can't find the container with id 289d61008f27144d43c326c129c6f2fe4069fa73277d7f58e5a897180cb9bb63
Jul 30 03:39:36.822703 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.822646    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:36.938972 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:36.938947    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:37.075089 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.075006    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mxscm" event={"ID":"00de1d13-af77-4a4d-b18a-d94bd36f78b7","Type":"ContainerStarted","Data":"289d61008f27144d43c326c129c6f2fe4069fa73277d7f58e5a897180cb9bb63"}
Jul 30 03:39:37.075089 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.075042    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rg4t8"
Jul 30 03:39:37.102212 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.102186    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:37.323632 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.323600    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:37.420396 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.420309    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:37.472575 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.472542    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 03:39:37.475843 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.475822    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:37.478565 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.478537    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 30 03:39:37.478691 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.478565    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 30 03:39:37.478691 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.478547    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 30 03:39:37.478808 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.478722    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 30 03:39:37.478886 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.478866    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 30 03:39:37.479242 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.479028    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 30 03:39:37.479242 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.479089    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 30 03:39:37.479242 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.479133    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 30 03:39:37.479242 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.479088    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-qqkzl\""
Jul 30 03:39:37.479242 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.479211    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 30 03:39:37.492080 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.492058    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 03:39:37.498125 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.498102    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:37.541913 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.541884    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:37.542057 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.541943    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:37.542057 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.541965    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:37.542057 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.541988    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdq9t\" (UniqueName: \"kubernetes.io/projected/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-kube-api-access-kdq9t\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:37.542057 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.542026    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:37.542266 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.542092    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:37.542266 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.542110    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:37.542266 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.542130    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:37.542266 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.542174    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-web-config\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:37.542266 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.542213    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:37.542266 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.542251    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-config-volume\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:37.542538 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.542279    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-config-out\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:37.542538 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.542305    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:37.592240 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.592207    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:37.642768 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.642747    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:37.642861 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.642774    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:37.642861 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.642793    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:37.642861 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.642844    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-web-config\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:37.642994 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.642869    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:37.642994 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.642898    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-config-volume\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:37.642994 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.642958    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-config-out\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:37.642994 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.642989    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:37.643181 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.643032    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:37.643181 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.643068    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:37.643181 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.643099    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:37.643181 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.643129    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdq9t\" (UniqueName: \"kubernetes.io/projected/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-kube-api-access-kdq9t\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:37.643181 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.643167    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:37.643709 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:39:37.642960    2576 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 30 03:39:37.643709 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.643674    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:37.643816 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:39:37.643433    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-alertmanager-trusted-ca-bundle podName:c79d975f-9d98-44f1-8cb8-5703a3e08d9b nodeName:}" failed. No retries permitted until 2026-07-30 03:39:38.143413238 +0000 UTC m=+283.761645461 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "c79d975f-9d98-44f1-8cb8-5703a3e08d9b") : configmap references non-existent config key: ca-bundle.crt
Jul 30 03:39:37.643816 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:39:37.643797    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-secret-alertmanager-main-tls podName:c79d975f-9d98-44f1-8cb8-5703a3e08d9b nodeName:}" failed. No retries permitted until 2026-07-30 03:39:38.143776599 +0000 UTC m=+283.762008833 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "c79d975f-9d98-44f1-8cb8-5703a3e08d9b") : secret "alertmanager-main-tls" not found
Jul 30 03:39:37.644239 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.644181    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:37.645781 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.645754    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-config-volume\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:37.645864 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.645813    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:37.645864 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.645821    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-web-config\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:37.645986 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.645968    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-config-out\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:37.646396 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.646373    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:37.646396 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.646392    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:37.646565 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.646543    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:37.647553 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.647535    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:37.656464 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.656444    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdq9t\" (UniqueName: \"kubernetes.io/projected/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-kube-api-access-kdq9t\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:37.721102 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.721078    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:37.855572 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:37.855540    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:38.079273 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:38.079242    2576 generic.go:356] "Generic (PLEG): container finished" podID="00de1d13-af77-4a4d-b18a-d94bd36f78b7" containerID="ddc41d5aed44ff5949d6ed86156526194747c174cb1bb4c4524075e69ded9f6a" exitCode=0
Jul 30 03:39:38.079436 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:38.079303    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mxscm" event={"ID":"00de1d13-af77-4a4d-b18a-d94bd36f78b7","Type":"ContainerDied","Data":"ddc41d5aed44ff5949d6ed86156526194747c174cb1bb4c4524075e69ded9f6a"}
Jul 30 03:39:38.146452 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:38.146427    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:38.146551 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:38.146533    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:38.147327 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:38.147302    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:38.148533 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:38.148516    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:38.387351 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:38.387265    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:39:38.430138 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:38.430106    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:38.529900 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:38.529860    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 03:39:38.532731 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:39:38.532707    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc79d975f_9d98_44f1_8cb8_5703a3e08d9b.slice/crio-8773110c6b75b06c5dd079f2462c7100af07ab957bd38436fd5096107deb686d WatchSource:0}: Error finding container 8773110c6b75b06c5dd079f2462c7100af07ab957bd38436fd5096107deb686d: Status 404 returned error can't find the container with id 8773110c6b75b06c5dd079f2462c7100af07ab957bd38436fd5096107deb686d
Jul 30 03:39:38.536452 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:38.536431    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:39.083791 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:39.083756    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mxscm" event={"ID":"00de1d13-af77-4a4d-b18a-d94bd36f78b7","Type":"ContainerStarted","Data":"a2e240b09b05adb3eb8380ad444a54a151f76d546150d3358f20a30ce16dd07c"}
Jul 30 03:39:39.084241 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:39.083812    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mxscm" event={"ID":"00de1d13-af77-4a4d-b18a-d94bd36f78b7","Type":"ContainerStarted","Data":"cac2bc0ba8d1abe067f307802c36914c604d28707ce1da5a037919443b4cb2c4"}
Jul 30 03:39:39.084829 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:39.084807    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c79d975f-9d98-44f1-8cb8-5703a3e08d9b","Type":"ContainerStarted","Data":"8773110c6b75b06c5dd079f2462c7100af07ab957bd38436fd5096107deb686d"}
Jul 30 03:39:39.146515 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:39.146480    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:39.243559 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:39.243515    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:39.404904 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:39.404811    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:39.470294 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:39.470241    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:39.656563 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:39.656451    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:39.885125 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:39.885088    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:40.089093 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:40.089060    2576 generic.go:356] "Generic (PLEG): container finished" podID="c79d975f-9d98-44f1-8cb8-5703a3e08d9b" containerID="d9a1fc1446fd616e5f8b7738ec1925dbd0f09567eb5154b9d929fc90f4d27dee" exitCode=0
Jul 30 03:39:40.089484 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:40.089140    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c79d975f-9d98-44f1-8cb8-5703a3e08d9b","Type":"ContainerDied","Data":"d9a1fc1446fd616e5f8b7738ec1925dbd0f09567eb5154b9d929fc90f4d27dee"}
Jul 30 03:39:40.115454 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:40.115408    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mxscm" podStartSLOduration=3.181566842 podStartE2EDuration="4.115392346s" totalImagesPullingTime="872.046175ms" totalInitContainerRuntime="61.779329ms" isStatefulPod=true podCreationTimestamp="2026-07-30 03:39:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:39:39.112187545 +0000 UTC m=+284.730419786" watchObservedRunningTime="2026-07-30 03:39:40.115392346 +0000 UTC m=+285.733624587"
Jul 30 03:39:40.165035 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:40.164948    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1f75cd36-d556-4abe-88d6-e83b63fb3799-metrics-tls\") pod \"dns-default-rg4t8\" (UID: \"1f75cd36-d556-4abe-88d6-e83b63fb3799\") " pod="openshift-dns/dns-default-rg4t8"
Jul 30 03:39:40.167341 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:40.167316    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1f75cd36-d556-4abe-88d6-e83b63fb3799-metrics-tls\") pod \"dns-default-rg4t8\" (UID: \"1f75cd36-d556-4abe-88d6-e83b63fb3799\") " pod="openshift-dns/dns-default-rg4t8"
Jul 30 03:39:40.378364 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:40.378339    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-nmkwv\""
Jul 30 03:39:40.386376 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:40.386358    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rg4t8"
Jul 30 03:39:40.538594 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:40.538562    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rg4t8"]
Jul 30 03:39:40.541267 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:39:40.541239    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f75cd36_d556_4abe_88d6_e83b63fb3799.slice/crio-d0863ee854f6be189db4e03d3cedbd0386f4fdf4aed384817c31152b9b6656ef WatchSource:0}: Error finding container d0863ee854f6be189db4e03d3cedbd0386f4fdf4aed384817c31152b9b6656ef: Status 404 returned error can't find the container with id d0863ee854f6be189db4e03d3cedbd0386f4fdf4aed384817c31152b9b6656ef
Jul 30 03:39:41.094661 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:41.094604    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rg4t8" event={"ID":"1f75cd36-d556-4abe-88d6-e83b63fb3799","Type":"ContainerStarted","Data":"d0863ee854f6be189db4e03d3cedbd0386f4fdf4aed384817c31152b9b6656ef"}
Jul 30 03:39:41.154086 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:41.154053    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7dbd669765-v87zk"]
Jul 30 03:39:41.171540 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:41.171491    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7dbd669765-v87zk"]
Jul 30 03:39:41.171703 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:41.171595    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dbd669765-v87zk"
Jul 30 03:39:41.173961 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:41.173910    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:41.174414 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:41.174393    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-qwnxn\""
Jul 30 03:39:41.175448 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:41.175426    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 30 03:39:41.175690 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:41.175673    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 30 03:39:41.176211 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:41.175909    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 30 03:39:41.176211 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:41.175948    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 30 03:39:41.176211 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:41.176165    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 30 03:39:41.176420 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:41.176228    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 30 03:39:41.176420 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:41.176344    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 30 03:39:41.242719 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:41.242680    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:41.276332 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:41.276299    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5b538fbc-9674-462e-9a2c-b6d5f6d589b8-console-serving-cert\") pod \"console-7dbd669765-v87zk\" (UID: \"5b538fbc-9674-462e-9a2c-b6d5f6d589b8\") " pod="openshift-console/console-7dbd669765-v87zk"
Jul 30 03:39:41.276505 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:41.276352    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5b538fbc-9674-462e-9a2c-b6d5f6d589b8-console-oauth-config\") pod \"console-7dbd669765-v87zk\" (UID: \"5b538fbc-9674-462e-9a2c-b6d5f6d589b8\") " pod="openshift-console/console-7dbd669765-v87zk"
Jul 30 03:39:41.276505 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:41.276427    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5b538fbc-9674-462e-9a2c-b6d5f6d589b8-oauth-serving-cert\") pod \"console-7dbd669765-v87zk\" (UID: \"5b538fbc-9674-462e-9a2c-b6d5f6d589b8\") " pod="openshift-console/console-7dbd669765-v87zk"
Jul 30 03:39:41.276632 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:41.276499    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5b538fbc-9674-462e-9a2c-b6d5f6d589b8-console-config\") pod \"console-7dbd669765-v87zk\" (UID: \"5b538fbc-9674-462e-9a2c-b6d5f6d589b8\") " pod="openshift-console/console-7dbd669765-v87zk"
Jul 30 03:39:41.276632 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:41.276531    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5b538fbc-9674-462e-9a2c-b6d5f6d589b8-service-ca\") pod \"console-7dbd669765-v87zk\" (UID: \"5b538fbc-9674-462e-9a2c-b6d5f6d589b8\") " pod="openshift-console/console-7dbd669765-v87zk"
Jul 30 03:39:41.276632 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:41.276554    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pj6d5\" (UniqueName: \"kubernetes.io/projected/5b538fbc-9674-462e-9a2c-b6d5f6d589b8-kube-api-access-pj6d5\") pod \"console-7dbd669765-v87zk\" (UID: \"5b538fbc-9674-462e-9a2c-b6d5f6d589b8\") " pod="openshift-console/console-7dbd669765-v87zk"
Jul 30 03:39:41.378545 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:41.377889    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5b538fbc-9674-462e-9a2c-b6d5f6d589b8-console-config\") pod \"console-7dbd669765-v87zk\" (UID: \"5b538fbc-9674-462e-9a2c-b6d5f6d589b8\") " pod="openshift-console/console-7dbd669765-v87zk"
Jul 30 03:39:41.378545 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:41.377955    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5b538fbc-9674-462e-9a2c-b6d5f6d589b8-service-ca\") pod \"console-7dbd669765-v87zk\" (UID: \"5b538fbc-9674-462e-9a2c-b6d5f6d589b8\") " pod="openshift-console/console-7dbd669765-v87zk"
Jul 30 03:39:41.378545 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:41.377981    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pj6d5\" (UniqueName: \"kubernetes.io/projected/5b538fbc-9674-462e-9a2c-b6d5f6d589b8-kube-api-access-pj6d5\") pod \"console-7dbd669765-v87zk\" (UID: \"5b538fbc-9674-462e-9a2c-b6d5f6d589b8\") " pod="openshift-console/console-7dbd669765-v87zk"
Jul 30 03:39:41.378545 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:41.378077    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5b538fbc-9674-462e-9a2c-b6d5f6d589b8-console-serving-cert\") pod \"console-7dbd669765-v87zk\" (UID: \"5b538fbc-9674-462e-9a2c-b6d5f6d589b8\") " pod="openshift-console/console-7dbd669765-v87zk"
Jul 30 03:39:41.378545 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:41.378102    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5b538fbc-9674-462e-9a2c-b6d5f6d589b8-console-oauth-config\") pod \"console-7dbd669765-v87zk\" (UID: \"5b538fbc-9674-462e-9a2c-b6d5f6d589b8\") " pod="openshift-console/console-7dbd669765-v87zk"
Jul 30 03:39:41.378545 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:41.378161    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5b538fbc-9674-462e-9a2c-b6d5f6d589b8-oauth-serving-cert\") pod \"console-7dbd669765-v87zk\" (UID: \"5b538fbc-9674-462e-9a2c-b6d5f6d589b8\") " pod="openshift-console/console-7dbd669765-v87zk"
Jul 30 03:39:41.378957 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:41.378636    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5b538fbc-9674-462e-9a2c-b6d5f6d589b8-console-config\") pod \"console-7dbd669765-v87zk\" (UID: \"5b538fbc-9674-462e-9a2c-b6d5f6d589b8\") " pod="openshift-console/console-7dbd669765-v87zk"
Jul 30 03:39:41.378957 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:41.378730    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5b538fbc-9674-462e-9a2c-b6d5f6d589b8-oauth-serving-cert\") pod \"console-7dbd669765-v87zk\" (UID: \"5b538fbc-9674-462e-9a2c-b6d5f6d589b8\") " pod="openshift-console/console-7dbd669765-v87zk"
Jul 30 03:39:41.379470 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:41.379419    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5b538fbc-9674-462e-9a2c-b6d5f6d589b8-service-ca\") pod \"console-7dbd669765-v87zk\" (UID: \"5b538fbc-9674-462e-9a2c-b6d5f6d589b8\") " pod="openshift-console/console-7dbd669765-v87zk"
Jul 30 03:39:41.381283 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:41.381258    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5b538fbc-9674-462e-9a2c-b6d5f6d589b8-console-serving-cert\") pod \"console-7dbd669765-v87zk\" (UID: \"5b538fbc-9674-462e-9a2c-b6d5f6d589b8\") " pod="openshift-console/console-7dbd669765-v87zk"
Jul 30 03:39:41.381391 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:41.381326    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5b538fbc-9674-462e-9a2c-b6d5f6d589b8-console-oauth-config\") pod \"console-7dbd669765-v87zk\" (UID: \"5b538fbc-9674-462e-9a2c-b6d5f6d589b8\") " pod="openshift-console/console-7dbd669765-v87zk"
Jul 30 03:39:41.386835 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:41.386811    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj6d5\" (UniqueName: \"kubernetes.io/projected/5b538fbc-9674-462e-9a2c-b6d5f6d589b8-kube-api-access-pj6d5\") pod \"console-7dbd669765-v87zk\" (UID: \"5b538fbc-9674-462e-9a2c-b6d5f6d589b8\") " pod="openshift-console/console-7dbd669765-v87zk"
Jul 30 03:39:41.396757 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:41.396730    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:41.485795 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:41.485760    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dbd669765-v87zk"
Jul 30 03:39:42.754992 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:42.754964    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:42.794645 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:42.794005    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7dbd669765-v87zk"]
Jul 30 03:39:42.797885 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:39:42.797854    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b538fbc_9674_462e_9a2c_b6d5f6d589b8.slice/crio-e13c1edc85ea3f473fa8dcec7bbcffe2162333a3d876880fa69e8ed2e15d44ed WatchSource:0}: Error finding container e13c1edc85ea3f473fa8dcec7bbcffe2162333a3d876880fa69e8ed2e15d44ed: Status 404 returned error can't find the container with id e13c1edc85ea3f473fa8dcec7bbcffe2162333a3d876880fa69e8ed2e15d44ed
Jul 30 03:39:43.104321 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:43.104294    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rg4t8" event={"ID":"1f75cd36-d556-4abe-88d6-e83b63fb3799","Type":"ContainerStarted","Data":"7a9f641fde632c78ba310598cbc1a904744a04c6196a52131f5f78f63418d5d6"}
Jul 30 03:39:43.104430 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:43.104349    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rg4t8" event={"ID":"1f75cd36-d556-4abe-88d6-e83b63fb3799","Type":"ContainerStarted","Data":"be330a0e555fd2d167dc5c011198979a913074c1a2de1882d0551bece52d6862"}
Jul 30 03:39:43.104488 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:43.104441    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-rg4t8"
Jul 30 03:39:43.106522 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:43.106503    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c79d975f-9d98-44f1-8cb8-5703a3e08d9b","Type":"ContainerStarted","Data":"0d8c62849cac6f3365dc1f0f783401cd70cd532c113ae0f3e96814f7bb6e5893"}
Jul 30 03:39:43.106605 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:43.106553    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c79d975f-9d98-44f1-8cb8-5703a3e08d9b","Type":"ContainerStarted","Data":"d1d9845a4acf1d0dc1989889132acf9f0dcadf8f85af4957117d45c8ed5879c9"}
Jul 30 03:39:43.106605 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:43.106590    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c79d975f-9d98-44f1-8cb8-5703a3e08d9b","Type":"ContainerStarted","Data":"fe9ca7619ce5d762d28805b234604066a568596d139a457b592e0e620d84a650"}
Jul 30 03:39:43.107522 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:43.107507    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dbd669765-v87zk" event={"ID":"5b538fbc-9674-462e-9a2c-b6d5f6d589b8","Type":"ContainerStarted","Data":"e13c1edc85ea3f473fa8dcec7bbcffe2162333a3d876880fa69e8ed2e15d44ed"}
Jul 30 03:39:43.124385 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:43.124342    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-rg4t8" podStartSLOduration=251.002177039 podStartE2EDuration="4m13.124328063s" totalImagesPullingTime="2.122151024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:35:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:39:43.123518305 +0000 UTC m=+288.741750548" watchObservedRunningTime="2026-07-30 03:39:43.124328063 +0000 UTC m=+288.742560304"
Jul 30 03:39:44.115382 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:44.115344    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c79d975f-9d98-44f1-8cb8-5703a3e08d9b","Type":"ContainerStarted","Data":"824cfbb275edbc1dbeca6e6e57b8bc96f1b98659fe3ab7b2ebc594f20def8665"}
Jul 30 03:39:44.115824 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:44.115429    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c79d975f-9d98-44f1-8cb8-5703a3e08d9b","Type":"ContainerStarted","Data":"d0e3a0443893a3716ee9132cfaa7625247a6947741d26bd0d1df2186a8c905c1"}
Jul 30 03:39:45.122653 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:45.122599    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c79d975f-9d98-44f1-8cb8-5703a3e08d9b","Type":"ContainerStarted","Data":"c9980044557812a02fa963ce05ef38dca2930c55ea9f692a1bbe018ba06c72c7"}
Jul 30 03:39:45.153421 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:45.153356    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.050461926 podStartE2EDuration="8.153334245s" totalImagesPullingTime="5.087028538s" totalInitContainerRuntime="15.843781ms" isStatefulPod=false podCreationTimestamp="2026-07-30 03:39:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:39:45.151768497 +0000 UTC m=+290.770000741" watchObservedRunningTime="2026-07-30 03:39:45.153334245 +0000 UTC m=+290.771566487"
Jul 30 03:39:47.130524 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:47.130490    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dbd669765-v87zk" event={"ID":"5b538fbc-9674-462e-9a2c-b6d5f6d589b8","Type":"ContainerStarted","Data":"42170c1a252e7d766bd7943d1c3f46ca1fdd4f66eac1468a156e88826bb31555"}
Jul 30 03:39:47.147896 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:47.147842    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7dbd669765-v87zk" podStartSLOduration=2.674995798 podStartE2EDuration="6.147825868s" totalImagesPullingTime="3.47283007s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:39:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:39:47.147567519 +0000 UTC m=+292.765799763" watchObservedRunningTime="2026-07-30 03:39:47.147825868 +0000 UTC m=+292.766058109"
Jul 30 03:39:47.537126 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:47.537094    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:47.590282 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:47.590252    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:49.046619 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:39:49.046576    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-rqrtn" podUID="890cb2cc-a9ca-4b13-bef5-f2282e486612"
Jul 30 03:39:50.545966 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:50.545913    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:50.685103 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:50.685056    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:50.785744 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:50.785715    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:50.884844 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:50.884757    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:51.486190 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:51.486158    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7dbd669765-v87zk"
Jul 30 03:39:51.486393 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:51.486212    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7dbd669765-v87zk"
Jul 30 03:39:51.491014 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:51.490991    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7dbd669765-v87zk"
Jul 30 03:39:52.151240 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:52.151215    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7dbd669765-v87zk"
Jul 30 03:39:53.117596 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:53.117571    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-rg4t8"
Jul 30 03:39:53.253031 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:53.252998    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:53.483879 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:53.483795    2576 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f57f6567c-xxf7w" podUID="8a57c996-9a44-47d1-b408-69f315b4f8f4" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 30 03:39:53.686578 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:53.686547    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:53.786789 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:53.786764    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:54.138596 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:54.138497    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:54.197123 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:54.197083    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:54.844525 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:54.844503    2576 kubelet.go:1648] "Image garbage collection succeeded"
Jul 30 03:39:57.601649 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:57.601612    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:39:57.700116 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:39:57.700077    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:40:01.045415 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:01.045377    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqrtn"
Jul 30 03:40:01.574970 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:01.574918    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:40:01.631472 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:01.631425    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:40:02.676180 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:02.676143    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:40:02.710093 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:02.710059    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7dbd669765-v87zk"]
Jul 30 03:40:02.772989 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:02.772953    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:40:02.843054 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:02.843024    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:40:03.483910 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:03.483875    2576 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f57f6567c-xxf7w" podUID="8a57c996-9a44-47d1-b408-69f315b4f8f4" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 30 03:40:13.484335 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:13.484297    2576 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f57f6567c-xxf7w" podUID="8a57c996-9a44-47d1-b408-69f315b4f8f4" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 30 03:40:13.484788 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:13.484391    2576 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f57f6567c-xxf7w"
Jul 30 03:40:13.484855 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:13.484826    2576 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"c5b7c80253871d4d47ec5efb022d6e63e5a06a2a340e6309baa915410480b5c3"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f57f6567c-xxf7w" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Jul 30 03:40:13.484896 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:13.484874    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f57f6567c-xxf7w" podUID="8a57c996-9a44-47d1-b408-69f315b4f8f4" containerName="service-proxy" containerID="cri-o://c5b7c80253871d4d47ec5efb022d6e63e5a06a2a340e6309baa915410480b5c3" gracePeriod=30
Jul 30 03:40:14.219598 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:14.219566    2576 generic.go:356] "Generic (PLEG): container finished" podID="8a57c996-9a44-47d1-b408-69f315b4f8f4" containerID="c5b7c80253871d4d47ec5efb022d6e63e5a06a2a340e6309baa915410480b5c3" exitCode=2
Jul 30 03:40:14.219768 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:14.219622    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f57f6567c-xxf7w" event={"ID":"8a57c996-9a44-47d1-b408-69f315b4f8f4","Type":"ContainerDied","Data":"c5b7c80253871d4d47ec5efb022d6e63e5a06a2a340e6309baa915410480b5c3"}
Jul 30 03:40:14.219768 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:14.219654    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f57f6567c-xxf7w" event={"ID":"8a57c996-9a44-47d1-b408-69f315b4f8f4","Type":"ContainerStarted","Data":"1fb8252d6bba6d9e0862f7cb57865943120cfb15c01203e6031cc53c2db56517"}
Jul 30 03:40:16.565944 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:16.565895    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:40:16.665689 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:16.665653    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:40:18.702617 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:18.702579    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qkjrz_2de4f068-3e70-41d6-8c64-5b6620269083/serve-healthcheck-canary/0.log"
Jul 30 03:40:27.728489 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:27.728427    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7dbd669765-v87zk" podUID="5b538fbc-9674-462e-9a2c-b6d5f6d589b8" containerName="console" containerID="cri-o://42170c1a252e7d766bd7943d1c3f46ca1fdd4f66eac1468a156e88826bb31555" gracePeriod=15
Jul 30 03:40:27.969964 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:27.969938    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dbd669765-v87zk"
Jul 30 03:40:28.048167 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:28.048134    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pj6d5\" (UniqueName: \"kubernetes.io/projected/5b538fbc-9674-462e-9a2c-b6d5f6d589b8-kube-api-access-pj6d5\") pod \"5b538fbc-9674-462e-9a2c-b6d5f6d589b8\" (UID: \"5b538fbc-9674-462e-9a2c-b6d5f6d589b8\") "
Jul 30 03:40:28.048360 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:28.048174    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5b538fbc-9674-462e-9a2c-b6d5f6d589b8-service-ca\") pod \"5b538fbc-9674-462e-9a2c-b6d5f6d589b8\" (UID: \"5b538fbc-9674-462e-9a2c-b6d5f6d589b8\") "
Jul 30 03:40:28.048360 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:28.048214    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5b538fbc-9674-462e-9a2c-b6d5f6d589b8-oauth-serving-cert\") pod \"5b538fbc-9674-462e-9a2c-b6d5f6d589b8\" (UID: \"5b538fbc-9674-462e-9a2c-b6d5f6d589b8\") "
Jul 30 03:40:28.048360 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:28.048244    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5b538fbc-9674-462e-9a2c-b6d5f6d589b8-console-oauth-config\") pod \"5b538fbc-9674-462e-9a2c-b6d5f6d589b8\" (UID: \"5b538fbc-9674-462e-9a2c-b6d5f6d589b8\") "
Jul 30 03:40:28.048360 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:28.048273    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5b538fbc-9674-462e-9a2c-b6d5f6d589b8-console-config\") pod \"5b538fbc-9674-462e-9a2c-b6d5f6d589b8\" (UID: \"5b538fbc-9674-462e-9a2c-b6d5f6d589b8\") "
Jul 30 03:40:28.048360 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:28.048309    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5b538fbc-9674-462e-9a2c-b6d5f6d589b8-console-serving-cert\") pod \"5b538fbc-9674-462e-9a2c-b6d5f6d589b8\" (UID: \"5b538fbc-9674-462e-9a2c-b6d5f6d589b8\") "
Jul 30 03:40:28.048680 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:28.048647    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b538fbc-9674-462e-9a2c-b6d5f6d589b8-console-config" (OuterVolumeSpecName: "console-config") pod "5b538fbc-9674-462e-9a2c-b6d5f6d589b8" (UID: "5b538fbc-9674-462e-9a2c-b6d5f6d589b8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 03:40:28.048764 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:28.048693    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b538fbc-9674-462e-9a2c-b6d5f6d589b8-service-ca" (OuterVolumeSpecName: "service-ca") pod "5b538fbc-9674-462e-9a2c-b6d5f6d589b8" (UID: "5b538fbc-9674-462e-9a2c-b6d5f6d589b8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 03:40:28.048812 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:28.048756    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b538fbc-9674-462e-9a2c-b6d5f6d589b8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5b538fbc-9674-462e-9a2c-b6d5f6d589b8" (UID: "5b538fbc-9674-462e-9a2c-b6d5f6d589b8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 03:40:28.050454 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:28.050434    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b538fbc-9674-462e-9a2c-b6d5f6d589b8-kube-api-access-pj6d5" (OuterVolumeSpecName: "kube-api-access-pj6d5") pod "5b538fbc-9674-462e-9a2c-b6d5f6d589b8" (UID: "5b538fbc-9674-462e-9a2c-b6d5f6d589b8"). InnerVolumeSpecName "kube-api-access-pj6d5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:40:28.050527 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:28.050506    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b538fbc-9674-462e-9a2c-b6d5f6d589b8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5b538fbc-9674-462e-9a2c-b6d5f6d589b8" (UID: "5b538fbc-9674-462e-9a2c-b6d5f6d589b8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 03:40:28.050566 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:28.050543    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b538fbc-9674-462e-9a2c-b6d5f6d589b8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5b538fbc-9674-462e-9a2c-b6d5f6d589b8" (UID: "5b538fbc-9674-462e-9a2c-b6d5f6d589b8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 03:40:28.148903 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:28.148867    2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5b538fbc-9674-462e-9a2c-b6d5f6d589b8-console-serving-cert\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:40:28.148903 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:28.148896    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pj6d5\" (UniqueName: \"kubernetes.io/projected/5b538fbc-9674-462e-9a2c-b6d5f6d589b8-kube-api-access-pj6d5\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:40:28.148903 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:28.148909    2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5b538fbc-9674-462e-9a2c-b6d5f6d589b8-service-ca\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:40:28.149161 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:28.148944    2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5b538fbc-9674-462e-9a2c-b6d5f6d589b8-oauth-serving-cert\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:40:28.149161 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:28.148956    2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5b538fbc-9674-462e-9a2c-b6d5f6d589b8-console-oauth-config\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:40:28.149161 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:28.148968    2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5b538fbc-9674-462e-9a2c-b6d5f6d589b8-console-config\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:40:28.267484 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:28.267448    2576 generic.go:356] "Generic (PLEG): container finished" podID="5b538fbc-9674-462e-9a2c-b6d5f6d589b8" containerID="42170c1a252e7d766bd7943d1c3f46ca1fdd4f66eac1468a156e88826bb31555" exitCode=0
Jul 30 03:40:28.267671 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:28.267499    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dbd669765-v87zk"
Jul 30 03:40:28.267671 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:28.267512    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dbd669765-v87zk" event={"ID":"5b538fbc-9674-462e-9a2c-b6d5f6d589b8","Type":"ContainerDied","Data":"42170c1a252e7d766bd7943d1c3f46ca1fdd4f66eac1468a156e88826bb31555"}
Jul 30 03:40:28.267671 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:28.267552    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dbd669765-v87zk" event={"ID":"5b538fbc-9674-462e-9a2c-b6d5f6d589b8","Type":"ContainerDied","Data":"e13c1edc85ea3f473fa8dcec7bbcffe2162333a3d876880fa69e8ed2e15d44ed"}
Jul 30 03:40:28.267671 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:28.267571    2576 scope.go:117] "RemoveContainer" containerID="42170c1a252e7d766bd7943d1c3f46ca1fdd4f66eac1468a156e88826bb31555"
Jul 30 03:40:28.276713 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:28.276690    2576 scope.go:117] "RemoveContainer" containerID="42170c1a252e7d766bd7943d1c3f46ca1fdd4f66eac1468a156e88826bb31555"
Jul 30 03:40:28.277063 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:40:28.277043    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42170c1a252e7d766bd7943d1c3f46ca1fdd4f66eac1468a156e88826bb31555\": container with ID starting with 42170c1a252e7d766bd7943d1c3f46ca1fdd4f66eac1468a156e88826bb31555 not found: ID does not exist" containerID="42170c1a252e7d766bd7943d1c3f46ca1fdd4f66eac1468a156e88826bb31555"
Jul 30 03:40:28.277141 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:28.277071    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42170c1a252e7d766bd7943d1c3f46ca1fdd4f66eac1468a156e88826bb31555"} err="failed to get container status \"42170c1a252e7d766bd7943d1c3f46ca1fdd4f66eac1468a156e88826bb31555\": rpc error: code = NotFound desc = could not find container \"42170c1a252e7d766bd7943d1c3f46ca1fdd4f66eac1468a156e88826bb31555\": container with ID starting with 42170c1a252e7d766bd7943d1c3f46ca1fdd4f66eac1468a156e88826bb31555 not found: ID does not exist"
Jul 30 03:40:28.289234 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:28.289207    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7dbd669765-v87zk"]
Jul 30 03:40:28.294035 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:28.294011    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7dbd669765-v87zk"]
Jul 30 03:40:29.049881 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:29.049852    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b538fbc-9674-462e-9a2c-b6d5f6d589b8" path="/var/lib/kubelet/pods/5b538fbc-9674-462e-9a2c-b6d5f6d589b8/volumes"
Jul 30 03:40:30.860005 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:30.859915    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:40:36.491281 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:36.491249    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:40:36.593311 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:36.593265    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:40:36.649121 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:36.649088    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:40:52.549240 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:52.549166    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:40:52.606863 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:52.606837    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:40:56.825016 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:56.824984    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 03:40:56.825400 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:56.825248    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c79d975f-9d98-44f1-8cb8-5703a3e08d9b" containerName="alertmanager" containerID="cri-o://fe9ca7619ce5d762d28805b234604066a568596d139a457b592e0e620d84a650" gracePeriod=120
Jul 30 03:40:56.825400 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:56.825321    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c79d975f-9d98-44f1-8cb8-5703a3e08d9b" containerName="kube-rbac-proxy-metric" containerID="cri-o://824cfbb275edbc1dbeca6e6e57b8bc96f1b98659fe3ab7b2ebc594f20def8665" gracePeriod=120
Jul 30 03:40:56.825400 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:56.825356    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c79d975f-9d98-44f1-8cb8-5703a3e08d9b" containerName="kube-rbac-proxy" containerID="cri-o://d0e3a0443893a3716ee9132cfaa7625247a6947741d26bd0d1df2186a8c905c1" gracePeriod=120
Jul 30 03:40:56.825400 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:56.825372    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c79d975f-9d98-44f1-8cb8-5703a3e08d9b" containerName="config-reloader" containerID="cri-o://d1d9845a4acf1d0dc1989889132acf9f0dcadf8f85af4957117d45c8ed5879c9" gracePeriod=120
Jul 30 03:40:56.825597 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:56.825333    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c79d975f-9d98-44f1-8cb8-5703a3e08d9b" containerName="kube-rbac-proxy-web" containerID="cri-o://0d8c62849cac6f3365dc1f0f783401cd70cd532c113ae0f3e96814f7bb6e5893" gracePeriod=120
Jul 30 03:40:56.825597 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:56.825429    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c79d975f-9d98-44f1-8cb8-5703a3e08d9b" containerName="prom-label-proxy" containerID="cri-o://c9980044557812a02fa963ce05ef38dca2930c55ea9f692a1bbe018ba06c72c7" gracePeriod=120
Jul 30 03:40:57.374357 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:57.374327    2576 generic.go:356] "Generic (PLEG): container finished" podID="c79d975f-9d98-44f1-8cb8-5703a3e08d9b" containerID="c9980044557812a02fa963ce05ef38dca2930c55ea9f692a1bbe018ba06c72c7" exitCode=0
Jul 30 03:40:57.374357 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:57.374350    2576 generic.go:356] "Generic (PLEG): container finished" podID="c79d975f-9d98-44f1-8cb8-5703a3e08d9b" containerID="d0e3a0443893a3716ee9132cfaa7625247a6947741d26bd0d1df2186a8c905c1" exitCode=0
Jul 30 03:40:57.374357 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:57.374356    2576 generic.go:356] "Generic (PLEG): container finished" podID="c79d975f-9d98-44f1-8cb8-5703a3e08d9b" containerID="d1d9845a4acf1d0dc1989889132acf9f0dcadf8f85af4957117d45c8ed5879c9" exitCode=0
Jul 30 03:40:57.374357 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:57.374361    2576 generic.go:356] "Generic (PLEG): container finished" podID="c79d975f-9d98-44f1-8cb8-5703a3e08d9b" containerID="fe9ca7619ce5d762d28805b234604066a568596d139a457b592e0e620d84a650" exitCode=0
Jul 30 03:40:57.374614 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:57.374379    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c79d975f-9d98-44f1-8cb8-5703a3e08d9b","Type":"ContainerDied","Data":"c9980044557812a02fa963ce05ef38dca2930c55ea9f692a1bbe018ba06c72c7"}
Jul 30 03:40:57.374614 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:57.374428    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c79d975f-9d98-44f1-8cb8-5703a3e08d9b","Type":"ContainerDied","Data":"d0e3a0443893a3716ee9132cfaa7625247a6947741d26bd0d1df2186a8c905c1"}
Jul 30 03:40:57.374614 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:57.374452    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c79d975f-9d98-44f1-8cb8-5703a3e08d9b","Type":"ContainerDied","Data":"d1d9845a4acf1d0dc1989889132acf9f0dcadf8f85af4957117d45c8ed5879c9"}
Jul 30 03:40:57.374614 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:57.374475    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c79d975f-9d98-44f1-8cb8-5703a3e08d9b","Type":"ContainerDied","Data":"fe9ca7619ce5d762d28805b234604066a568596d139a457b592e0e620d84a650"}
Jul 30 03:40:58.069020 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.068997    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.181267 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.181190    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-alertmanager-main-db\") pod \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") "
Jul 30 03:40:58.181267 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.181226    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-secret-alertmanager-main-tls\") pod \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") "
Jul 30 03:40:58.181267 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.181246    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-alertmanager-trusted-ca-bundle\") pod \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") "
Jul 30 03:40:58.181550 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.181283    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-secret-alertmanager-kube-rbac-proxy-web\") pod \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") "
Jul 30 03:40:58.181550 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.181318    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdq9t\" (UniqueName: \"kubernetes.io/projected/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-kube-api-access-kdq9t\") pod \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") "
Jul 30 03:40:58.181550 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.181354    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-web-config\") pod \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") "
Jul 30 03:40:58.181550 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.181399    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") "
Jul 30 03:40:58.181550 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.181434    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-config-volume\") pod \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") "
Jul 30 03:40:58.181550 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.181475    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-metrics-client-ca\") pod \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") "
Jul 30 03:40:58.181550 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.181505    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-secret-alertmanager-kube-rbac-proxy\") pod \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") "
Jul 30 03:40:58.181550 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.181533    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-tls-assets\") pod \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") "
Jul 30 03:40:58.181995 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.181548    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "c79d975f-9d98-44f1-8cb8-5703a3e08d9b" (UID: "c79d975f-9d98-44f1-8cb8-5703a3e08d9b"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 03:40:58.181995 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.181567    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-cluster-tls-config\") pod \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") "
Jul 30 03:40:58.181995 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.181594    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-config-out\") pod \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\" (UID: \"c79d975f-9d98-44f1-8cb8-5703a3e08d9b\") "
Jul 30 03:40:58.181995 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.181651    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "c79d975f-9d98-44f1-8cb8-5703a3e08d9b" (UID: "c79d975f-9d98-44f1-8cb8-5703a3e08d9b"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 03:40:58.181995 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.181875    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "c79d975f-9d98-44f1-8cb8-5703a3e08d9b" (UID: "c79d975f-9d98-44f1-8cb8-5703a3e08d9b"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 03:40:58.181995 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.181973    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-metrics-client-ca\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:40:58.181995 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.181991    2576 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-alertmanager-main-db\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:40:58.182345 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.182006    2576 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:40:58.184295 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.184256    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "c79d975f-9d98-44f1-8cb8-5703a3e08d9b" (UID: "c79d975f-9d98-44f1-8cb8-5703a3e08d9b"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 03:40:58.184562 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.184528    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "c79d975f-9d98-44f1-8cb8-5703a3e08d9b" (UID: "c79d975f-9d98-44f1-8cb8-5703a3e08d9b"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:40:58.184562 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.184531    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-config-out" (OuterVolumeSpecName: "config-out") pod "c79d975f-9d98-44f1-8cb8-5703a3e08d9b" (UID: "c79d975f-9d98-44f1-8cb8-5703a3e08d9b"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 03:40:58.184730 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.184566    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "c79d975f-9d98-44f1-8cb8-5703a3e08d9b" (UID: "c79d975f-9d98-44f1-8cb8-5703a3e08d9b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 03:40:58.185167 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.185140    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "c79d975f-9d98-44f1-8cb8-5703a3e08d9b" (UID: "c79d975f-9d98-44f1-8cb8-5703a3e08d9b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 03:40:58.185167 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.185139    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-kube-api-access-kdq9t" (OuterVolumeSpecName: "kube-api-access-kdq9t") pod "c79d975f-9d98-44f1-8cb8-5703a3e08d9b" (UID: "c79d975f-9d98-44f1-8cb8-5703a3e08d9b"). InnerVolumeSpecName "kube-api-access-kdq9t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:40:58.185320 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.185191    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "c79d975f-9d98-44f1-8cb8-5703a3e08d9b" (UID: "c79d975f-9d98-44f1-8cb8-5703a3e08d9b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 03:40:58.185749 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.185723    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-config-volume" (OuterVolumeSpecName: "config-volume") pod "c79d975f-9d98-44f1-8cb8-5703a3e08d9b" (UID: "c79d975f-9d98-44f1-8cb8-5703a3e08d9b"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 03:40:58.188122 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.188105    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "c79d975f-9d98-44f1-8cb8-5703a3e08d9b" (UID: "c79d975f-9d98-44f1-8cb8-5703a3e08d9b"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 03:40:58.193742 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.193721    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-web-config" (OuterVolumeSpecName: "web-config") pod "c79d975f-9d98-44f1-8cb8-5703a3e08d9b" (UID: "c79d975f-9d98-44f1-8cb8-5703a3e08d9b"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 03:40:58.282947 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.282893    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdq9t\" (UniqueName: \"kubernetes.io/projected/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-kube-api-access-kdq9t\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:40:58.282947 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.282938    2576 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-web-config\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:40:58.282947 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.282949    2576 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:40:58.282947 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.282958    2576 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-config-volume\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:40:58.283216 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.282966    2576 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:40:58.283216 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.282975    2576 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-tls-assets\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:40:58.283216 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.282983    2576 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-cluster-tls-config\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:40:58.283216 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.282990    2576 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-config-out\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:40:58.283216 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.282998    2576 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-secret-alertmanager-main-tls\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:40:58.283216 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.283006    2576 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c79d975f-9d98-44f1-8cb8-5703a3e08d9b-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:40:58.380790 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.380760    2576 generic.go:356] "Generic (PLEG): container finished" podID="c79d975f-9d98-44f1-8cb8-5703a3e08d9b" containerID="824cfbb275edbc1dbeca6e6e57b8bc96f1b98659fe3ab7b2ebc594f20def8665" exitCode=0
Jul 30 03:40:58.380790 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.380784    2576 generic.go:356] "Generic (PLEG): container finished" podID="c79d975f-9d98-44f1-8cb8-5703a3e08d9b" containerID="0d8c62849cac6f3365dc1f0f783401cd70cd532c113ae0f3e96814f7bb6e5893" exitCode=0
Jul 30 03:40:58.381015 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.380831    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c79d975f-9d98-44f1-8cb8-5703a3e08d9b","Type":"ContainerDied","Data":"824cfbb275edbc1dbeca6e6e57b8bc96f1b98659fe3ab7b2ebc594f20def8665"}
Jul 30 03:40:58.381015 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.380853    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.381015 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.380932    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c79d975f-9d98-44f1-8cb8-5703a3e08d9b","Type":"ContainerDied","Data":"0d8c62849cac6f3365dc1f0f783401cd70cd532c113ae0f3e96814f7bb6e5893"}
Jul 30 03:40:58.381015 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.380963    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c79d975f-9d98-44f1-8cb8-5703a3e08d9b","Type":"ContainerDied","Data":"8773110c6b75b06c5dd079f2462c7100af07ab957bd38436fd5096107deb686d"}
Jul 30 03:40:58.381702 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.380997    2576 scope.go:117] "RemoveContainer" containerID="c9980044557812a02fa963ce05ef38dca2930c55ea9f692a1bbe018ba06c72c7"
Jul 30 03:40:58.389594 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.389578    2576 scope.go:117] "RemoveContainer" containerID="824cfbb275edbc1dbeca6e6e57b8bc96f1b98659fe3ab7b2ebc594f20def8665"
Jul 30 03:40:58.396043 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.396023    2576 scope.go:117] "RemoveContainer" containerID="d0e3a0443893a3716ee9132cfaa7625247a6947741d26bd0d1df2186a8c905c1"
Jul 30 03:40:58.402005 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.401977    2576 scope.go:117] "RemoveContainer" containerID="0d8c62849cac6f3365dc1f0f783401cd70cd532c113ae0f3e96814f7bb6e5893"
Jul 30 03:40:58.404388 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.404367    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 03:40:58.408089 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.408074    2576 scope.go:117] "RemoveContainer" containerID="d1d9845a4acf1d0dc1989889132acf9f0dcadf8f85af4957117d45c8ed5879c9"
Jul 30 03:40:58.410289 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.410264    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 03:40:58.414663 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.414644    2576 scope.go:117] "RemoveContainer" containerID="fe9ca7619ce5d762d28805b234604066a568596d139a457b592e0e620d84a650"
Jul 30 03:40:58.420865 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.420850    2576 scope.go:117] "RemoveContainer" containerID="d9a1fc1446fd616e5f8b7738ec1925dbd0f09567eb5154b9d929fc90f4d27dee"
Jul 30 03:40:58.426784 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.426770    2576 scope.go:117] "RemoveContainer" containerID="c9980044557812a02fa963ce05ef38dca2930c55ea9f692a1bbe018ba06c72c7"
Jul 30 03:40:58.427046 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:40:58.427020    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9980044557812a02fa963ce05ef38dca2930c55ea9f692a1bbe018ba06c72c7\": container with ID starting with c9980044557812a02fa963ce05ef38dca2930c55ea9f692a1bbe018ba06c72c7 not found: ID does not exist" containerID="c9980044557812a02fa963ce05ef38dca2930c55ea9f692a1bbe018ba06c72c7"
Jul 30 03:40:58.427112 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.427055    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9980044557812a02fa963ce05ef38dca2930c55ea9f692a1bbe018ba06c72c7"} err="failed to get container status \"c9980044557812a02fa963ce05ef38dca2930c55ea9f692a1bbe018ba06c72c7\": rpc error: code = NotFound desc = could not find container \"c9980044557812a02fa963ce05ef38dca2930c55ea9f692a1bbe018ba06c72c7\": container with ID starting with c9980044557812a02fa963ce05ef38dca2930c55ea9f692a1bbe018ba06c72c7 not found: ID does not exist"
Jul 30 03:40:58.427112 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.427072    2576 scope.go:117] "RemoveContainer" containerID="824cfbb275edbc1dbeca6e6e57b8bc96f1b98659fe3ab7b2ebc594f20def8665"
Jul 30 03:40:58.427301 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:40:58.427284    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"824cfbb275edbc1dbeca6e6e57b8bc96f1b98659fe3ab7b2ebc594f20def8665\": container with ID starting with 824cfbb275edbc1dbeca6e6e57b8bc96f1b98659fe3ab7b2ebc594f20def8665 not found: ID does not exist" containerID="824cfbb275edbc1dbeca6e6e57b8bc96f1b98659fe3ab7b2ebc594f20def8665"
Jul 30 03:40:58.427367 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.427312    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"824cfbb275edbc1dbeca6e6e57b8bc96f1b98659fe3ab7b2ebc594f20def8665"} err="failed to get container status \"824cfbb275edbc1dbeca6e6e57b8bc96f1b98659fe3ab7b2ebc594f20def8665\": rpc error: code = NotFound desc = could not find container \"824cfbb275edbc1dbeca6e6e57b8bc96f1b98659fe3ab7b2ebc594f20def8665\": container with ID starting with 824cfbb275edbc1dbeca6e6e57b8bc96f1b98659fe3ab7b2ebc594f20def8665 not found: ID does not exist"
Jul 30 03:40:58.427367 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.427336    2576 scope.go:117] "RemoveContainer" containerID="d0e3a0443893a3716ee9132cfaa7625247a6947741d26bd0d1df2186a8c905c1"
Jul 30 03:40:58.427560 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:40:58.427545    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0e3a0443893a3716ee9132cfaa7625247a6947741d26bd0d1df2186a8c905c1\": container with ID starting with d0e3a0443893a3716ee9132cfaa7625247a6947741d26bd0d1df2186a8c905c1 not found: ID does not exist" containerID="d0e3a0443893a3716ee9132cfaa7625247a6947741d26bd0d1df2186a8c905c1"
Jul 30 03:40:58.427597 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.427564    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0e3a0443893a3716ee9132cfaa7625247a6947741d26bd0d1df2186a8c905c1"} err="failed to get container status \"d0e3a0443893a3716ee9132cfaa7625247a6947741d26bd0d1df2186a8c905c1\": rpc error: code = NotFound desc = could not find container \"d0e3a0443893a3716ee9132cfaa7625247a6947741d26bd0d1df2186a8c905c1\": container with ID starting with d0e3a0443893a3716ee9132cfaa7625247a6947741d26bd0d1df2186a8c905c1 not found: ID does not exist"
Jul 30 03:40:58.427597 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.427577    2576 scope.go:117] "RemoveContainer" containerID="0d8c62849cac6f3365dc1f0f783401cd70cd532c113ae0f3e96814f7bb6e5893"
Jul 30 03:40:58.427777 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:40:58.427762    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d8c62849cac6f3365dc1f0f783401cd70cd532c113ae0f3e96814f7bb6e5893\": container with ID starting with 0d8c62849cac6f3365dc1f0f783401cd70cd532c113ae0f3e96814f7bb6e5893 not found: ID does not exist" containerID="0d8c62849cac6f3365dc1f0f783401cd70cd532c113ae0f3e96814f7bb6e5893"
Jul 30 03:40:58.427823 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.427780    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d8c62849cac6f3365dc1f0f783401cd70cd532c113ae0f3e96814f7bb6e5893"} err="failed to get container status \"0d8c62849cac6f3365dc1f0f783401cd70cd532c113ae0f3e96814f7bb6e5893\": rpc error: code = NotFound desc = could not find container \"0d8c62849cac6f3365dc1f0f783401cd70cd532c113ae0f3e96814f7bb6e5893\": container with ID starting with 0d8c62849cac6f3365dc1f0f783401cd70cd532c113ae0f3e96814f7bb6e5893 not found: ID does not exist"
Jul 30 03:40:58.427823 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.427791    2576 scope.go:117] "RemoveContainer" containerID="d1d9845a4acf1d0dc1989889132acf9f0dcadf8f85af4957117d45c8ed5879c9"
Jul 30 03:40:58.428032 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:40:58.428017    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1d9845a4acf1d0dc1989889132acf9f0dcadf8f85af4957117d45c8ed5879c9\": container with ID starting with d1d9845a4acf1d0dc1989889132acf9f0dcadf8f85af4957117d45c8ed5879c9 not found: ID does not exist" containerID="d1d9845a4acf1d0dc1989889132acf9f0dcadf8f85af4957117d45c8ed5879c9"
Jul 30 03:40:58.428101 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.428038    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1d9845a4acf1d0dc1989889132acf9f0dcadf8f85af4957117d45c8ed5879c9"} err="failed to get container status \"d1d9845a4acf1d0dc1989889132acf9f0dcadf8f85af4957117d45c8ed5879c9\": rpc error: code = NotFound desc = could not find container \"d1d9845a4acf1d0dc1989889132acf9f0dcadf8f85af4957117d45c8ed5879c9\": container with ID starting with d1d9845a4acf1d0dc1989889132acf9f0dcadf8f85af4957117d45c8ed5879c9 not found: ID does not exist"
Jul 30 03:40:58.428101 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.428057    2576 scope.go:117] "RemoveContainer" containerID="fe9ca7619ce5d762d28805b234604066a568596d139a457b592e0e620d84a650"
Jul 30 03:40:58.428337 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:40:58.428321    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe9ca7619ce5d762d28805b234604066a568596d139a457b592e0e620d84a650\": container with ID starting with fe9ca7619ce5d762d28805b234604066a568596d139a457b592e0e620d84a650 not found: ID does not exist" containerID="fe9ca7619ce5d762d28805b234604066a568596d139a457b592e0e620d84a650"
Jul 30 03:40:58.428402 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.428342    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe9ca7619ce5d762d28805b234604066a568596d139a457b592e0e620d84a650"} err="failed to get container status \"fe9ca7619ce5d762d28805b234604066a568596d139a457b592e0e620d84a650\": rpc error: code = NotFound desc = could not find container \"fe9ca7619ce5d762d28805b234604066a568596d139a457b592e0e620d84a650\": container with ID starting with fe9ca7619ce5d762d28805b234604066a568596d139a457b592e0e620d84a650 not found: ID does not exist"
Jul 30 03:40:58.428402 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.428357    2576 scope.go:117] "RemoveContainer" containerID="d9a1fc1446fd616e5f8b7738ec1925dbd0f09567eb5154b9d929fc90f4d27dee"
Jul 30 03:40:58.428643 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:40:58.428626    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9a1fc1446fd616e5f8b7738ec1925dbd0f09567eb5154b9d929fc90f4d27dee\": container with ID starting with d9a1fc1446fd616e5f8b7738ec1925dbd0f09567eb5154b9d929fc90f4d27dee not found: ID does not exist" containerID="d9a1fc1446fd616e5f8b7738ec1925dbd0f09567eb5154b9d929fc90f4d27dee"
Jul 30 03:40:58.428704 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.428648    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9a1fc1446fd616e5f8b7738ec1925dbd0f09567eb5154b9d929fc90f4d27dee"} err="failed to get container status \"d9a1fc1446fd616e5f8b7738ec1925dbd0f09567eb5154b9d929fc90f4d27dee\": rpc error: code = NotFound desc = could not find container \"d9a1fc1446fd616e5f8b7738ec1925dbd0f09567eb5154b9d929fc90f4d27dee\": container with ID starting with d9a1fc1446fd616e5f8b7738ec1925dbd0f09567eb5154b9d929fc90f4d27dee not found: ID does not exist"
Jul 30 03:40:58.428704 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.428662    2576 scope.go:117] "RemoveContainer" containerID="c9980044557812a02fa963ce05ef38dca2930c55ea9f692a1bbe018ba06c72c7"
Jul 30 03:40:58.428880 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.428862    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9980044557812a02fa963ce05ef38dca2930c55ea9f692a1bbe018ba06c72c7"} err="failed to get container status \"c9980044557812a02fa963ce05ef38dca2930c55ea9f692a1bbe018ba06c72c7\": rpc error: code = NotFound desc = could not find container \"c9980044557812a02fa963ce05ef38dca2930c55ea9f692a1bbe018ba06c72c7\": container with ID starting with c9980044557812a02fa963ce05ef38dca2930c55ea9f692a1bbe018ba06c72c7 not found: ID does not exist"
Jul 30 03:40:58.428967 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.428880    2576 scope.go:117] "RemoveContainer" containerID="824cfbb275edbc1dbeca6e6e57b8bc96f1b98659fe3ab7b2ebc594f20def8665"
Jul 30 03:40:58.429121 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.429106    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"824cfbb275edbc1dbeca6e6e57b8bc96f1b98659fe3ab7b2ebc594f20def8665"} err="failed to get container status \"824cfbb275edbc1dbeca6e6e57b8bc96f1b98659fe3ab7b2ebc594f20def8665\": rpc error: code = NotFound desc = could not find container \"824cfbb275edbc1dbeca6e6e57b8bc96f1b98659fe3ab7b2ebc594f20def8665\": container with ID starting with 824cfbb275edbc1dbeca6e6e57b8bc96f1b98659fe3ab7b2ebc594f20def8665 not found: ID does not exist"
Jul 30 03:40:58.429160 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.429122    2576 scope.go:117] "RemoveContainer" containerID="d0e3a0443893a3716ee9132cfaa7625247a6947741d26bd0d1df2186a8c905c1"
Jul 30 03:40:58.429372 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.429355    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0e3a0443893a3716ee9132cfaa7625247a6947741d26bd0d1df2186a8c905c1"} err="failed to get container status \"d0e3a0443893a3716ee9132cfaa7625247a6947741d26bd0d1df2186a8c905c1\": rpc error: code = NotFound desc = could not find container \"d0e3a0443893a3716ee9132cfaa7625247a6947741d26bd0d1df2186a8c905c1\": container with ID starting with d0e3a0443893a3716ee9132cfaa7625247a6947741d26bd0d1df2186a8c905c1 not found: ID does not exist"
Jul 30 03:40:58.429417 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.429374    2576 scope.go:117] "RemoveContainer" containerID="0d8c62849cac6f3365dc1f0f783401cd70cd532c113ae0f3e96814f7bb6e5893"
Jul 30 03:40:58.429602 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.429584    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d8c62849cac6f3365dc1f0f783401cd70cd532c113ae0f3e96814f7bb6e5893"} err="failed to get container status \"0d8c62849cac6f3365dc1f0f783401cd70cd532c113ae0f3e96814f7bb6e5893\": rpc error: code = NotFound desc = could not find container \"0d8c62849cac6f3365dc1f0f783401cd70cd532c113ae0f3e96814f7bb6e5893\": container with ID starting with 0d8c62849cac6f3365dc1f0f783401cd70cd532c113ae0f3e96814f7bb6e5893 not found: ID does not exist"
Jul 30 03:40:58.429673 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.429604    2576 scope.go:117] "RemoveContainer" containerID="d1d9845a4acf1d0dc1989889132acf9f0dcadf8f85af4957117d45c8ed5879c9"
Jul 30 03:40:58.429848 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.429831    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1d9845a4acf1d0dc1989889132acf9f0dcadf8f85af4957117d45c8ed5879c9"} err="failed to get container status \"d1d9845a4acf1d0dc1989889132acf9f0dcadf8f85af4957117d45c8ed5879c9\": rpc error: code = NotFound desc = could not find container \"d1d9845a4acf1d0dc1989889132acf9f0dcadf8f85af4957117d45c8ed5879c9\": container with ID starting with d1d9845a4acf1d0dc1989889132acf9f0dcadf8f85af4957117d45c8ed5879c9 not found: ID does not exist"
Jul 30 03:40:58.429915 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.429849    2576 scope.go:117] "RemoveContainer" containerID="fe9ca7619ce5d762d28805b234604066a568596d139a457b592e0e620d84a650"
Jul 30 03:40:58.430074 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.430056    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe9ca7619ce5d762d28805b234604066a568596d139a457b592e0e620d84a650"} err="failed to get container status \"fe9ca7619ce5d762d28805b234604066a568596d139a457b592e0e620d84a650\": rpc error: code = NotFound desc = could not find container \"fe9ca7619ce5d762d28805b234604066a568596d139a457b592e0e620d84a650\": container with ID starting with fe9ca7619ce5d762d28805b234604066a568596d139a457b592e0e620d84a650 not found: ID does not exist"
Jul 30 03:40:58.430146 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.430075    2576 scope.go:117] "RemoveContainer" containerID="d9a1fc1446fd616e5f8b7738ec1925dbd0f09567eb5154b9d929fc90f4d27dee"
Jul 30 03:40:58.430282 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.430266    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9a1fc1446fd616e5f8b7738ec1925dbd0f09567eb5154b9d929fc90f4d27dee"} err="failed to get container status \"d9a1fc1446fd616e5f8b7738ec1925dbd0f09567eb5154b9d929fc90f4d27dee\": rpc error: code = NotFound desc = could not find container \"d9a1fc1446fd616e5f8b7738ec1925dbd0f09567eb5154b9d929fc90f4d27dee\": container with ID starting with d9a1fc1446fd616e5f8b7738ec1925dbd0f09567eb5154b9d929fc90f4d27dee not found: ID does not exist"
Jul 30 03:40:58.436253 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.436205    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 03:40:58.436450 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.436435    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c79d975f-9d98-44f1-8cb8-5703a3e08d9b" containerName="kube-rbac-proxy-web"
Jul 30 03:40:58.436521 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.436454    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c79d975f-9d98-44f1-8cb8-5703a3e08d9b" containerName="kube-rbac-proxy-web"
Jul 30 03:40:58.436521 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.436468    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c79d975f-9d98-44f1-8cb8-5703a3e08d9b" containerName="init-config-reloader"
Jul 30 03:40:58.436521 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.436477    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c79d975f-9d98-44f1-8cb8-5703a3e08d9b" containerName="init-config-reloader"
Jul 30 03:40:58.436521 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.436488    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c79d975f-9d98-44f1-8cb8-5703a3e08d9b" containerName="kube-rbac-proxy"
Jul 30 03:40:58.436521 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.436497    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c79d975f-9d98-44f1-8cb8-5703a3e08d9b" containerName="kube-rbac-proxy"
Jul 30 03:40:58.436521 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.436507    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c79d975f-9d98-44f1-8cb8-5703a3e08d9b" containerName="prom-label-proxy"
Jul 30 03:40:58.436521 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.436516    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c79d975f-9d98-44f1-8cb8-5703a3e08d9b" containerName="prom-label-proxy"
Jul 30 03:40:58.436855 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.436541    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c79d975f-9d98-44f1-8cb8-5703a3e08d9b" containerName="alertmanager"
Jul 30 03:40:58.436855 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.436549    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c79d975f-9d98-44f1-8cb8-5703a3e08d9b" containerName="alertmanager"
Jul 30 03:40:58.436855 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.436558    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c79d975f-9d98-44f1-8cb8-5703a3e08d9b" containerName="config-reloader"
Jul 30 03:40:58.436855 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.436566    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c79d975f-9d98-44f1-8cb8-5703a3e08d9b" containerName="config-reloader"
Jul 30 03:40:58.436855 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.436575    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c79d975f-9d98-44f1-8cb8-5703a3e08d9b" containerName="kube-rbac-proxy-metric"
Jul 30 03:40:58.436855 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.436583    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c79d975f-9d98-44f1-8cb8-5703a3e08d9b" containerName="kube-rbac-proxy-metric"
Jul 30 03:40:58.436855 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.436596    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b538fbc-9674-462e-9a2c-b6d5f6d589b8" containerName="console"
Jul 30 03:40:58.436855 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.436605    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b538fbc-9674-462e-9a2c-b6d5f6d589b8" containerName="console"
Jul 30 03:40:58.436855 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.436662    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c79d975f-9d98-44f1-8cb8-5703a3e08d9b" containerName="alertmanager"
Jul 30 03:40:58.436855 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.436674    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c79d975f-9d98-44f1-8cb8-5703a3e08d9b" containerName="kube-rbac-proxy-metric"
Jul 30 03:40:58.436855 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.436684    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c79d975f-9d98-44f1-8cb8-5703a3e08d9b" containerName="config-reloader"
Jul 30 03:40:58.436855 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.436693    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c79d975f-9d98-44f1-8cb8-5703a3e08d9b" containerName="kube-rbac-proxy-web"
Jul 30 03:40:58.436855 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.436701    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c79d975f-9d98-44f1-8cb8-5703a3e08d9b" containerName="prom-label-proxy"
Jul 30 03:40:58.436855 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.436708    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b538fbc-9674-462e-9a2c-b6d5f6d589b8" containerName="console"
Jul 30 03:40:58.436855 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.436718    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c79d975f-9d98-44f1-8cb8-5703a3e08d9b" containerName="kube-rbac-proxy"
Jul 30 03:40:58.441869 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.441853    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.444272 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.444258    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-qqkzl\""
Jul 30 03:40:58.444342 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.444257    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 30 03:40:58.444544 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.444523    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 30 03:40:58.444544 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.444538    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 30 03:40:58.444828 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.444813    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 30 03:40:58.444904 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.444844    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 30 03:40:58.444904 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.444867    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 30 03:40:58.444904 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.444876    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 30 03:40:58.445214 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.445198    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 30 03:40:58.449630 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.449613    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 30 03:40:58.456236 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.456216    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 03:40:58.584547 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.584513    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/da7311f9-4fbf-44cf-acda-e1cf0159f191-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.584547 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.584543    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/da7311f9-4fbf-44cf-acda-e1cf0159f191-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.584781 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.584572    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mptxj\" (UniqueName: \"kubernetes.io/projected/da7311f9-4fbf-44cf-acda-e1cf0159f191-kube-api-access-mptxj\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.584781 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.584621    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/da7311f9-4fbf-44cf-acda-e1cf0159f191-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.584781 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.584641    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/da7311f9-4fbf-44cf-acda-e1cf0159f191-config-out\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.584781 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.584659    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/da7311f9-4fbf-44cf-acda-e1cf0159f191-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.584781 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.584679    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/da7311f9-4fbf-44cf-acda-e1cf0159f191-tls-assets\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.584781 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.584711    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/da7311f9-4fbf-44cf-acda-e1cf0159f191-config-volume\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.584781 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.584726    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/da7311f9-4fbf-44cf-acda-e1cf0159f191-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.584781 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.584745    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da7311f9-4fbf-44cf-acda-e1cf0159f191-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.585084 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.584798    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/da7311f9-4fbf-44cf-acda-e1cf0159f191-web-config\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.585084 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.584851    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da7311f9-4fbf-44cf-acda-e1cf0159f191-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.585084 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.584876    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/da7311f9-4fbf-44cf-acda-e1cf0159f191-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.685903 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.685870    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/da7311f9-4fbf-44cf-acda-e1cf0159f191-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.685903 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.685915    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/da7311f9-4fbf-44cf-acda-e1cf0159f191-config-out\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.686118 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.686039    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/da7311f9-4fbf-44cf-acda-e1cf0159f191-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.686118 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.686073    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/da7311f9-4fbf-44cf-acda-e1cf0159f191-tls-assets\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.686118 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.686113    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/da7311f9-4fbf-44cf-acda-e1cf0159f191-config-volume\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.686289 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.686139    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/da7311f9-4fbf-44cf-acda-e1cf0159f191-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.686289 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.686169    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da7311f9-4fbf-44cf-acda-e1cf0159f191-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.686289 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.686198    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/da7311f9-4fbf-44cf-acda-e1cf0159f191-web-config\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.686289 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.686234    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da7311f9-4fbf-44cf-acda-e1cf0159f191-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.686289 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.686268    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/da7311f9-4fbf-44cf-acda-e1cf0159f191-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.686560 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.686305    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/da7311f9-4fbf-44cf-acda-e1cf0159f191-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.686560 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.686330    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/da7311f9-4fbf-44cf-acda-e1cf0159f191-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.686560 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.686388    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mptxj\" (UniqueName: \"kubernetes.io/projected/da7311f9-4fbf-44cf-acda-e1cf0159f191-kube-api-access-mptxj\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.687154 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.687136    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/da7311f9-4fbf-44cf-acda-e1cf0159f191-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.687647 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.687616    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da7311f9-4fbf-44cf-acda-e1cf0159f191-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.687770 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.687745    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da7311f9-4fbf-44cf-acda-e1cf0159f191-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.688888 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.688863    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/da7311f9-4fbf-44cf-acda-e1cf0159f191-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.689049 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.689022    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/da7311f9-4fbf-44cf-acda-e1cf0159f191-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.689049 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.689041    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/da7311f9-4fbf-44cf-acda-e1cf0159f191-web-config\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.689199 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.689070    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/da7311f9-4fbf-44cf-acda-e1cf0159f191-tls-assets\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.689199 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.689075    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/da7311f9-4fbf-44cf-acda-e1cf0159f191-config-out\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.689917 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.689887    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/da7311f9-4fbf-44cf-acda-e1cf0159f191-config-volume\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.690041 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.689962    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/da7311f9-4fbf-44cf-acda-e1cf0159f191-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.690200 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.690181    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/da7311f9-4fbf-44cf-acda-e1cf0159f191-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.690507 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.690493    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/da7311f9-4fbf-44cf-acda-e1cf0159f191-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.694909 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.694894    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mptxj\" (UniqueName: \"kubernetes.io/projected/da7311f9-4fbf-44cf-acda-e1cf0159f191-kube-api-access-mptxj\") pod \"alertmanager-main-0\" (UID: \"da7311f9-4fbf-44cf-acda-e1cf0159f191\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.750683 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.750655    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 30 03:40:58.879038 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:58.878954    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 03:40:58.881555 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:40:58.881527    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda7311f9_4fbf_44cf_acda_e1cf0159f191.slice/crio-de69abe53798a16c5455027c0a7a243567ecb2f44a09e9a11ec0388b8b84ebdb WatchSource:0}: Error finding container de69abe53798a16c5455027c0a7a243567ecb2f44a09e9a11ec0388b8b84ebdb: Status 404 returned error can't find the container with id de69abe53798a16c5455027c0a7a243567ecb2f44a09e9a11ec0388b8b84ebdb
Jul 30 03:40:59.049494 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:59.049468    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c79d975f-9d98-44f1-8cb8-5703a3e08d9b" path="/var/lib/kubelet/pods/c79d975f-9d98-44f1-8cb8-5703a3e08d9b/volumes"
Jul 30 03:40:59.386764 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:59.386734    2576 generic.go:356] "Generic (PLEG): container finished" podID="da7311f9-4fbf-44cf-acda-e1cf0159f191" containerID="168f9b6ef58897ae9245b2aa61390a94242c5219c6b8d948e59b5b244d669f1c" exitCode=0
Jul 30 03:40:59.387162 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:59.386792    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da7311f9-4fbf-44cf-acda-e1cf0159f191","Type":"ContainerDied","Data":"168f9b6ef58897ae9245b2aa61390a94242c5219c6b8d948e59b5b244d669f1c"}
Jul 30 03:40:59.387162 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:40:59.386841    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da7311f9-4fbf-44cf-acda-e1cf0159f191","Type":"ContainerStarted","Data":"de69abe53798a16c5455027c0a7a243567ecb2f44a09e9a11ec0388b8b84ebdb"}
Jul 30 03:41:00.393230 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:00.393194    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da7311f9-4fbf-44cf-acda-e1cf0159f191","Type":"ContainerStarted","Data":"f1dcb8f7bc43ab1948af9e654b0ab41a42bc47728522527de07ad0533e03d876"}
Jul 30 03:41:00.393605 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:00.393258    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da7311f9-4fbf-44cf-acda-e1cf0159f191","Type":"ContainerStarted","Data":"0912a65240bec91bd9b409283ca91f1d1ef12be66cacdc0f91e3db11c714b356"}
Jul 30 03:41:00.393605 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:00.393284    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da7311f9-4fbf-44cf-acda-e1cf0159f191","Type":"ContainerStarted","Data":"17c031d432894e75af2f7f9ddb068d610af02f5ed3de933abba1cfab07855311"}
Jul 30 03:41:00.393605 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:00.393312    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da7311f9-4fbf-44cf-acda-e1cf0159f191","Type":"ContainerStarted","Data":"b289a7035249ef899581c13a8acfc8ff098fa02a124524d34274724575698197"}
Jul 30 03:41:00.393605 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:00.393337    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da7311f9-4fbf-44cf-acda-e1cf0159f191","Type":"ContainerStarted","Data":"44fbf645ccf1392dd500d6a7a29dd72afde7e6c2635b76f1b37ffe4c24d4f609"}
Jul 30 03:41:00.393605 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:00.393358    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da7311f9-4fbf-44cf-acda-e1cf0159f191","Type":"ContainerStarted","Data":"43aaff56e979e4feea3e69f1709ecff9fcaf57b597d538e7df3b39816c7791ff"}
Jul 30 03:41:00.422387 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:00.422336    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.406621241 podStartE2EDuration="2.422320543s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.699302ms" isStatefulPod=false podCreationTimestamp="2026-07-30 03:40:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:41:00.419914108 +0000 UTC m=+366.038146349" watchObservedRunningTime="2026-07-30 03:41:00.422320543 +0000 UTC m=+366.040552783"
Jul 30 03:41:03.013618 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:03.013581    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:41:03.116791 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:03.116761    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:41:04.712272 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:04.712236    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5fdbf99bdb-hw8kc"]
Jul 30 03:41:04.715366 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:04.715348    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fdbf99bdb-hw8kc"
Jul 30 03:41:04.718293 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:04.718269    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 30 03:41:04.718399 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:04.718332    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 30 03:41:04.718399 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:04.718341    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 30 03:41:04.718508 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:04.718435    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 30 03:41:04.719534 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:04.719503    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 30 03:41:04.719534 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:04.719516    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 30 03:41:04.719668 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:04.719600    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 30 03:41:04.719771 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:04.719749    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-qwnxn\""
Jul 30 03:41:04.723719 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:04.723702    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 30 03:41:04.726771 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:04.726753    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5fdbf99bdb-hw8kc"]
Jul 30 03:41:04.773261 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:04.773236    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:41:04.835105 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:04.835079    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a95c6faa-d3b4-4311-83de-3e05bd962148-console-serving-cert\") pod \"console-5fdbf99bdb-hw8kc\" (UID: \"a95c6faa-d3b4-4311-83de-3e05bd962148\") " pod="openshift-console/console-5fdbf99bdb-hw8kc"
Jul 30 03:41:04.835191 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:04.835107    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a95c6faa-d3b4-4311-83de-3e05bd962148-console-oauth-config\") pod \"console-5fdbf99bdb-hw8kc\" (UID: \"a95c6faa-d3b4-4311-83de-3e05bd962148\") " pod="openshift-console/console-5fdbf99bdb-hw8kc"
Jul 30 03:41:04.835191 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:04.835136    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsl5r\" (UniqueName: \"kubernetes.io/projected/a95c6faa-d3b4-4311-83de-3e05bd962148-kube-api-access-nsl5r\") pod \"console-5fdbf99bdb-hw8kc\" (UID: \"a95c6faa-d3b4-4311-83de-3e05bd962148\") " pod="openshift-console/console-5fdbf99bdb-hw8kc"
Jul 30 03:41:04.835277 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:04.835204    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a95c6faa-d3b4-4311-83de-3e05bd962148-console-config\") pod \"console-5fdbf99bdb-hw8kc\" (UID: \"a95c6faa-d3b4-4311-83de-3e05bd962148\") " pod="openshift-console/console-5fdbf99bdb-hw8kc"
Jul 30 03:41:04.835277 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:04.835231    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a95c6faa-d3b4-4311-83de-3e05bd962148-oauth-serving-cert\") pod \"console-5fdbf99bdb-hw8kc\" (UID: \"a95c6faa-d3b4-4311-83de-3e05bd962148\") " pod="openshift-console/console-5fdbf99bdb-hw8kc"
Jul 30 03:41:04.835339 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:04.835291    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a95c6faa-d3b4-4311-83de-3e05bd962148-service-ca\") pod \"console-5fdbf99bdb-hw8kc\" (UID: \"a95c6faa-d3b4-4311-83de-3e05bd962148\") " pod="openshift-console/console-5fdbf99bdb-hw8kc"
Jul 30 03:41:04.835339 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:04.835320    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a95c6faa-d3b4-4311-83de-3e05bd962148-trusted-ca-bundle\") pod \"console-5fdbf99bdb-hw8kc\" (UID: \"a95c6faa-d3b4-4311-83de-3e05bd962148\") " pod="openshift-console/console-5fdbf99bdb-hw8kc"
Jul 30 03:41:04.875809 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:04.875788    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:41:04.935977 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:04.935943    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a95c6faa-d3b4-4311-83de-3e05bd962148-trusted-ca-bundle\") pod \"console-5fdbf99bdb-hw8kc\" (UID: \"a95c6faa-d3b4-4311-83de-3e05bd962148\") " pod="openshift-console/console-5fdbf99bdb-hw8kc"
Jul 30 03:41:04.936073 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:04.936003    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a95c6faa-d3b4-4311-83de-3e05bd962148-console-serving-cert\") pod \"console-5fdbf99bdb-hw8kc\" (UID: \"a95c6faa-d3b4-4311-83de-3e05bd962148\") " pod="openshift-console/console-5fdbf99bdb-hw8kc"
Jul 30 03:41:04.936073 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:04.936023    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a95c6faa-d3b4-4311-83de-3e05bd962148-console-oauth-config\") pod \"console-5fdbf99bdb-hw8kc\" (UID: \"a95c6faa-d3b4-4311-83de-3e05bd962148\") " pod="openshift-console/console-5fdbf99bdb-hw8kc"
Jul 30 03:41:04.936073 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:04.936045    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsl5r\" (UniqueName: \"kubernetes.io/projected/a95c6faa-d3b4-4311-83de-3e05bd962148-kube-api-access-nsl5r\") pod \"console-5fdbf99bdb-hw8kc\" (UID: \"a95c6faa-d3b4-4311-83de-3e05bd962148\") " pod="openshift-console/console-5fdbf99bdb-hw8kc"
Jul 30 03:41:04.936216 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:04.936094    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a95c6faa-d3b4-4311-83de-3e05bd962148-console-config\") pod \"console-5fdbf99bdb-hw8kc\" (UID: \"a95c6faa-d3b4-4311-83de-3e05bd962148\") " pod="openshift-console/console-5fdbf99bdb-hw8kc"
Jul 30 03:41:04.936216 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:04.936127    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a95c6faa-d3b4-4311-83de-3e05bd962148-oauth-serving-cert\") pod \"console-5fdbf99bdb-hw8kc\" (UID: \"a95c6faa-d3b4-4311-83de-3e05bd962148\") " pod="openshift-console/console-5fdbf99bdb-hw8kc"
Jul 30 03:41:04.936216 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:04.936166    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a95c6faa-d3b4-4311-83de-3e05bd962148-service-ca\") pod \"console-5fdbf99bdb-hw8kc\" (UID: \"a95c6faa-d3b4-4311-83de-3e05bd962148\") " pod="openshift-console/console-5fdbf99bdb-hw8kc"
Jul 30 03:41:04.936827 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:04.936804    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a95c6faa-d3b4-4311-83de-3e05bd962148-trusted-ca-bundle\") pod \"console-5fdbf99bdb-hw8kc\" (UID: \"a95c6faa-d3b4-4311-83de-3e05bd962148\") " pod="openshift-console/console-5fdbf99bdb-hw8kc"
Jul 30 03:41:04.936955 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:04.936814    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a95c6faa-d3b4-4311-83de-3e05bd962148-service-ca\") pod \"console-5fdbf99bdb-hw8kc\" (UID: \"a95c6faa-d3b4-4311-83de-3e05bd962148\") " pod="openshift-console/console-5fdbf99bdb-hw8kc"
Jul 30 03:41:04.936955 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:04.936826    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a95c6faa-d3b4-4311-83de-3e05bd962148-console-config\") pod \"console-5fdbf99bdb-hw8kc\" (UID: \"a95c6faa-d3b4-4311-83de-3e05bd962148\") " pod="openshift-console/console-5fdbf99bdb-hw8kc"
Jul 30 03:41:04.936955 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:04.936854    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a95c6faa-d3b4-4311-83de-3e05bd962148-oauth-serving-cert\") pod \"console-5fdbf99bdb-hw8kc\" (UID: \"a95c6faa-d3b4-4311-83de-3e05bd962148\") " pod="openshift-console/console-5fdbf99bdb-hw8kc"
Jul 30 03:41:04.938402 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:04.938369    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a95c6faa-d3b4-4311-83de-3e05bd962148-console-oauth-config\") pod \"console-5fdbf99bdb-hw8kc\" (UID: \"a95c6faa-d3b4-4311-83de-3e05bd962148\") " pod="openshift-console/console-5fdbf99bdb-hw8kc"
Jul 30 03:41:04.938553 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:04.938536    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a95c6faa-d3b4-4311-83de-3e05bd962148-console-serving-cert\") pod \"console-5fdbf99bdb-hw8kc\" (UID: \"a95c6faa-d3b4-4311-83de-3e05bd962148\") " pod="openshift-console/console-5fdbf99bdb-hw8kc"
Jul 30 03:41:04.946268 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:04.946242    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsl5r\" (UniqueName: \"kubernetes.io/projected/a95c6faa-d3b4-4311-83de-3e05bd962148-kube-api-access-nsl5r\") pod \"console-5fdbf99bdb-hw8kc\" (UID: \"a95c6faa-d3b4-4311-83de-3e05bd962148\") " pod="openshift-console/console-5fdbf99bdb-hw8kc"
Jul 30 03:41:04.975404 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:04.975309    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:41:05.024590 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:05.024556    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fdbf99bdb-hw8kc"
Jul 30 03:41:05.148416 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:05.148396    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5fdbf99bdb-hw8kc"]
Jul 30 03:41:05.150444 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:41:05.150425    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda95c6faa_d3b4_4311_83de_3e05bd962148.slice/crio-93ade8d4a0eb1f8f259d7251544d8cbf5ecb96be7df375c99405b54792d33e30 WatchSource:0}: Error finding container 93ade8d4a0eb1f8f259d7251544d8cbf5ecb96be7df375c99405b54792d33e30: Status 404 returned error can't find the container with id 93ade8d4a0eb1f8f259d7251544d8cbf5ecb96be7df375c99405b54792d33e30
Jul 30 03:41:05.412014 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:05.411976    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fdbf99bdb-hw8kc" event={"ID":"a95c6faa-d3b4-4311-83de-3e05bd962148","Type":"ContainerStarted","Data":"2eefe7bb57c8448ed99f3b05418c6afd3f7e791209bf61cfa2dea5925267a6b3"}
Jul 30 03:41:05.412187 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:05.412041    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fdbf99bdb-hw8kc" event={"ID":"a95c6faa-d3b4-4311-83de-3e05bd962148","Type":"ContainerStarted","Data":"93ade8d4a0eb1f8f259d7251544d8cbf5ecb96be7df375c99405b54792d33e30"}
Jul 30 03:41:05.429729 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:05.429687    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5fdbf99bdb-hw8kc" podStartSLOduration=1.429672028 podStartE2EDuration="1.429672028s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:41:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:41:05.429165727 +0000 UTC m=+371.047397965" watchObservedRunningTime="2026-07-30 03:41:05.429672028 +0000 UTC m=+371.047904269"
Jul 30 03:41:05.517164 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:05.517135    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:41:05.572470 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:05.572445    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:41:08.127182 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:08.127151    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:41:08.182520 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:08.182494    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:41:08.767981 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:08.767944    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/890cb2cc-a9ca-4b13-bef5-f2282e486612-metrics-certs\") pod \"network-metrics-daemon-rqrtn\" (UID: \"890cb2cc-a9ca-4b13-bef5-f2282e486612\") " pod="openshift-multus/network-metrics-daemon-rqrtn"
Jul 30 03:41:08.770266 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:08.770244    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/890cb2cc-a9ca-4b13-bef5-f2282e486612-metrics-certs\") pod \"network-metrics-daemon-rqrtn\" (UID: \"890cb2cc-a9ca-4b13-bef5-f2282e486612\") " pod="openshift-multus/network-metrics-daemon-rqrtn"
Jul 30 03:41:08.849240 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:08.849212    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bvtgp\""
Jul 30 03:41:08.857305 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:08.857284    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqrtn"
Jul 30 03:41:08.973868 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:08.973839    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rqrtn"]
Jul 30 03:41:08.977014 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:41:08.976986    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod890cb2cc_a9ca_4b13_bef5_f2282e486612.slice/crio-bee2b6379b567bea2f15e65e212e60ccd66c9544488ac7ef08f4c5e7f26308a6 WatchSource:0}: Error finding container bee2b6379b567bea2f15e65e212e60ccd66c9544488ac7ef08f4c5e7f26308a6: Status 404 returned error can't find the container with id bee2b6379b567bea2f15e65e212e60ccd66c9544488ac7ef08f4c5e7f26308a6
Jul 30 03:41:08.978975 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:08.978960    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 03:41:09.428089 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:09.428054    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rqrtn" event={"ID":"890cb2cc-a9ca-4b13-bef5-f2282e486612","Type":"ContainerStarted","Data":"bee2b6379b567bea2f15e65e212e60ccd66c9544488ac7ef08f4c5e7f26308a6"}
Jul 30 03:41:10.434958 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:10.434867    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rqrtn" event={"ID":"890cb2cc-a9ca-4b13-bef5-f2282e486612","Type":"ContainerStarted","Data":"d5b1af168d057d38da73fefc53d8cf867f5b1e67fa327de316e44da4f953d8e3"}
Jul 30 03:41:10.434958 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:10.434915    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rqrtn" event={"ID":"890cb2cc-a9ca-4b13-bef5-f2282e486612","Type":"ContainerStarted","Data":"b0cac5419fe14553bded28551fc9844395254ecf912e40a8b9f797205a6e224b"}
Jul 30 03:41:10.452422 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:10.452366    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rqrtn" podStartSLOduration=374.387610894 podStartE2EDuration="6m15.452346273s" totalImagesPullingTime="1.064735379s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:34:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:41:10.450326051 +0000 UTC m=+376.068558291" watchObservedRunningTime="2026-07-30 03:41:10.452346273 +0000 UTC m=+376.070578514"
Jul 30 03:41:10.536883 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:10.536851    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:41:10.598947 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:10.598909    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:41:15.025335 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:15.025300    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5fdbf99bdb-hw8kc"
Jul 30 03:41:15.025727 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:15.025383    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5fdbf99bdb-hw8kc"
Jul 30 03:41:15.030097 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:15.030073    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5fdbf99bdb-hw8kc"
Jul 30 03:41:15.455171 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:15.455104    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5fdbf99bdb-hw8kc"
Jul 30 03:41:15.630253 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:15.630224    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:41:15.730296 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:15.730222    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:41:15.783187 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:15.783160    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:41:16.563142 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:16.563105    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:41:16.663145 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:16.663117    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:41:48.728888 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:41:48.728856    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:42:00.569015 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:00.568982    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:42:00.675158 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:00.675125    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:42:01.245027 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:01.244994    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f5wb9"]
Jul 30 03:42:01.249465 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:01.249444    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f5wb9"
Jul 30 03:42:01.251907 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:01.251885    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 30 03:42:01.253005 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:01.252988    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 30 03:42:01.253005 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:01.252996    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-bsxn6\""
Jul 30 03:42:01.256415 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:01.256024    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f5wb9"]
Jul 30 03:42:01.331293 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:01.331255    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcb8n\" (UniqueName: \"kubernetes.io/projected/825f87db-8764-4dbe-9a8d-e54276a28caa-kube-api-access-bcb8n\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f5wb9\" (UID: \"825f87db-8764-4dbe-9a8d-e54276a28caa\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f5wb9"
Jul 30 03:42:01.331462 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:01.331333    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/825f87db-8764-4dbe-9a8d-e54276a28caa-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f5wb9\" (UID: \"825f87db-8764-4dbe-9a8d-e54276a28caa\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f5wb9"
Jul 30 03:42:01.331462 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:01.331379    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/825f87db-8764-4dbe-9a8d-e54276a28caa-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f5wb9\" (UID: \"825f87db-8764-4dbe-9a8d-e54276a28caa\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f5wb9"
Jul 30 03:42:01.432570 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:01.432530    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/825f87db-8764-4dbe-9a8d-e54276a28caa-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f5wb9\" (UID: \"825f87db-8764-4dbe-9a8d-e54276a28caa\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f5wb9"
Jul 30 03:42:01.432680 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:01.432590    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/825f87db-8764-4dbe-9a8d-e54276a28caa-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f5wb9\" (UID: \"825f87db-8764-4dbe-9a8d-e54276a28caa\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f5wb9"
Jul 30 03:42:01.432680 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:01.432616    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcb8n\" (UniqueName: \"kubernetes.io/projected/825f87db-8764-4dbe-9a8d-e54276a28caa-kube-api-access-bcb8n\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f5wb9\" (UID: \"825f87db-8764-4dbe-9a8d-e54276a28caa\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f5wb9"
Jul 30 03:42:01.432988 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:01.432968    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/825f87db-8764-4dbe-9a8d-e54276a28caa-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f5wb9\" (UID: \"825f87db-8764-4dbe-9a8d-e54276a28caa\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f5wb9"
Jul 30 03:42:01.433034 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:01.432978    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/825f87db-8764-4dbe-9a8d-e54276a28caa-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f5wb9\" (UID: \"825f87db-8764-4dbe-9a8d-e54276a28caa\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f5wb9"
Jul 30 03:42:01.441166 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:01.441135    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcb8n\" (UniqueName: \"kubernetes.io/projected/825f87db-8764-4dbe-9a8d-e54276a28caa-kube-api-access-bcb8n\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f5wb9\" (UID: \"825f87db-8764-4dbe-9a8d-e54276a28caa\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f5wb9"
Jul 30 03:42:01.559128 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:01.559100    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f5wb9"
Jul 30 03:42:01.679343 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:01.679317    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f5wb9"]
Jul 30 03:42:01.681874 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:42:01.681846    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod825f87db_8764_4dbe_9a8d_e54276a28caa.slice/crio-53124b9cbfe0d115b9b351b6b982435bf756630453f1e17fba79e4444b99e59b WatchSource:0}: Error finding container 53124b9cbfe0d115b9b351b6b982435bf756630453f1e17fba79e4444b99e59b: Status 404 returned error can't find the container with id 53124b9cbfe0d115b9b351b6b982435bf756630453f1e17fba79e4444b99e59b
Jul 30 03:42:02.615094 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:02.615061    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f5wb9" event={"ID":"825f87db-8764-4dbe-9a8d-e54276a28caa","Type":"ContainerStarted","Data":"53124b9cbfe0d115b9b351b6b982435bf756630453f1e17fba79e4444b99e59b"}
Jul 30 03:42:09.642205 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:09.642120    2576 generic.go:356] "Generic (PLEG): container finished" podID="825f87db-8764-4dbe-9a8d-e54276a28caa" containerID="c3f4b6efc30e85a72fd48454a7119af07b9db7143006be5f660cc097f4aa3dc3" exitCode=0
Jul 30 03:42:09.642205 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:09.642190    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f5wb9" event={"ID":"825f87db-8764-4dbe-9a8d-e54276a28caa","Type":"ContainerDied","Data":"c3f4b6efc30e85a72fd48454a7119af07b9db7143006be5f660cc097f4aa3dc3"}
Jul 30 03:42:12.656451 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:12.656414    2576 generic.go:356] "Generic (PLEG): container finished" podID="825f87db-8764-4dbe-9a8d-e54276a28caa" containerID="9210012195adce15098184a4c06f80ae55d9d18c646d149e463408f942840844" exitCode=0
Jul 30 03:42:12.656820 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:12.656494    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f5wb9" event={"ID":"825f87db-8764-4dbe-9a8d-e54276a28caa","Type":"ContainerDied","Data":"9210012195adce15098184a4c06f80ae55d9d18c646d149e463408f942840844"}
Jul 30 03:42:20.689558 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:20.689521    2576 generic.go:356] "Generic (PLEG): container finished" podID="825f87db-8764-4dbe-9a8d-e54276a28caa" containerID="9073be7ecebf524540fc3f8ea27c6cf0afb8014381fad61994c608e379084f91" exitCode=0
Jul 30 03:42:20.689937 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:20.689569    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f5wb9" event={"ID":"825f87db-8764-4dbe-9a8d-e54276a28caa","Type":"ContainerDied","Data":"9073be7ecebf524540fc3f8ea27c6cf0afb8014381fad61994c608e379084f91"}
Jul 30 03:42:21.848759 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:21.848726    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f5wb9"
Jul 30 03:42:22.003226 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:22.003144    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/825f87db-8764-4dbe-9a8d-e54276a28caa-util\") pod \"825f87db-8764-4dbe-9a8d-e54276a28caa\" (UID: \"825f87db-8764-4dbe-9a8d-e54276a28caa\") "
Jul 30 03:42:22.003226 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:22.003184    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bcb8n\" (UniqueName: \"kubernetes.io/projected/825f87db-8764-4dbe-9a8d-e54276a28caa-kube-api-access-bcb8n\") pod \"825f87db-8764-4dbe-9a8d-e54276a28caa\" (UID: \"825f87db-8764-4dbe-9a8d-e54276a28caa\") "
Jul 30 03:42:22.003226 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:22.003230    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/825f87db-8764-4dbe-9a8d-e54276a28caa-bundle\") pod \"825f87db-8764-4dbe-9a8d-e54276a28caa\" (UID: \"825f87db-8764-4dbe-9a8d-e54276a28caa\") "
Jul 30 03:42:22.003947 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:22.003903    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/825f87db-8764-4dbe-9a8d-e54276a28caa-bundle" (OuterVolumeSpecName: "bundle") pod "825f87db-8764-4dbe-9a8d-e54276a28caa" (UID: "825f87db-8764-4dbe-9a8d-e54276a28caa"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 03:42:22.005439 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:22.005420    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/825f87db-8764-4dbe-9a8d-e54276a28caa-kube-api-access-bcb8n" (OuterVolumeSpecName: "kube-api-access-bcb8n") pod "825f87db-8764-4dbe-9a8d-e54276a28caa" (UID: "825f87db-8764-4dbe-9a8d-e54276a28caa"). InnerVolumeSpecName "kube-api-access-bcb8n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:42:22.007824 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:22.007803    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/825f87db-8764-4dbe-9a8d-e54276a28caa-util" (OuterVolumeSpecName: "util") pod "825f87db-8764-4dbe-9a8d-e54276a28caa" (UID: "825f87db-8764-4dbe-9a8d-e54276a28caa"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 03:42:22.104349 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:22.104311    2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/825f87db-8764-4dbe-9a8d-e54276a28caa-util\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:42:22.104349 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:22.104343    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bcb8n\" (UniqueName: \"kubernetes.io/projected/825f87db-8764-4dbe-9a8d-e54276a28caa-kube-api-access-bcb8n\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:42:22.104349 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:22.104354    2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/825f87db-8764-4dbe-9a8d-e54276a28caa-bundle\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:42:22.699087 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:22.699055    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f5wb9" event={"ID":"825f87db-8764-4dbe-9a8d-e54276a28caa","Type":"ContainerDied","Data":"53124b9cbfe0d115b9b351b6b982435bf756630453f1e17fba79e4444b99e59b"}
Jul 30 03:42:22.699264 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:22.699096    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f5wb9"
Jul 30 03:42:22.699264 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:22.699101    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="53124b9cbfe0d115b9b351b6b982435bf756630453f1e17fba79e4444b99e59b"
Jul 30 03:42:24.243077 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:24.243044    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:42:26.021364 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:26.021329    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:42:27.862709 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:27.862662    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:42:27.963000 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:27.962966    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:42:30.386586 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:30.386548    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:42:30.489161 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:30.489131    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:42:30.632506 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:30.632477    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:42:30.737000 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:30.736904    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:42:31.184917 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:31.184874    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmthrk"]
Jul 30 03:42:31.185222 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:31.185205    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="825f87db-8764-4dbe-9a8d-e54276a28caa" containerName="util"
Jul 30 03:42:31.185313 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:31.185225    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="825f87db-8764-4dbe-9a8d-e54276a28caa" containerName="util"
Jul 30 03:42:31.185313 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:31.185241    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="825f87db-8764-4dbe-9a8d-e54276a28caa" containerName="extract"
Jul 30 03:42:31.185313 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:31.185248    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="825f87db-8764-4dbe-9a8d-e54276a28caa" containerName="extract"
Jul 30 03:42:31.185313 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:31.185258    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="825f87db-8764-4dbe-9a8d-e54276a28caa" containerName="pull"
Jul 30 03:42:31.185313 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:31.185265    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="825f87db-8764-4dbe-9a8d-e54276a28caa" containerName="pull"
Jul 30 03:42:31.185584 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:31.185339    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="825f87db-8764-4dbe-9a8d-e54276a28caa" containerName="extract"
Jul 30 03:42:31.187505 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:31.187485    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmthrk"
Jul 30 03:42:31.189839 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:31.189822    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-bsxn6\""
Jul 30 03:42:31.190235 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:31.190222    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 30 03:42:31.191064 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:31.191045    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 30 03:42:31.196116 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:31.196098    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmthrk"]
Jul 30 03:42:31.270098 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:31.270056    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cc49d276-ee8b-4aac-91cc-f9376bad14ff-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmthrk\" (UID: \"cc49d276-ee8b-4aac-91cc-f9376bad14ff\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmthrk"
Jul 30 03:42:31.270098 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:31.270104    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cc49d276-ee8b-4aac-91cc-f9376bad14ff-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmthrk\" (UID: \"cc49d276-ee8b-4aac-91cc-f9376bad14ff\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmthrk"
Jul 30 03:42:31.270312 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:31.270139    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jwcx\" (UniqueName: \"kubernetes.io/projected/cc49d276-ee8b-4aac-91cc-f9376bad14ff-kube-api-access-7jwcx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmthrk\" (UID: \"cc49d276-ee8b-4aac-91cc-f9376bad14ff\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmthrk"
Jul 30 03:42:31.371039 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:31.371004    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cc49d276-ee8b-4aac-91cc-f9376bad14ff-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmthrk\" (UID: \"cc49d276-ee8b-4aac-91cc-f9376bad14ff\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmthrk"
Jul 30 03:42:31.371127 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:31.371046    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cc49d276-ee8b-4aac-91cc-f9376bad14ff-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmthrk\" (UID: \"cc49d276-ee8b-4aac-91cc-f9376bad14ff\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmthrk"
Jul 30 03:42:31.371127 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:31.371073    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jwcx\" (UniqueName: \"kubernetes.io/projected/cc49d276-ee8b-4aac-91cc-f9376bad14ff-kube-api-access-7jwcx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmthrk\" (UID: \"cc49d276-ee8b-4aac-91cc-f9376bad14ff\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmthrk"
Jul 30 03:42:31.371712 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:31.371681    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cc49d276-ee8b-4aac-91cc-f9376bad14ff-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmthrk\" (UID: \"cc49d276-ee8b-4aac-91cc-f9376bad14ff\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmthrk"
Jul 30 03:42:31.371902 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:31.371729    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cc49d276-ee8b-4aac-91cc-f9376bad14ff-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmthrk\" (UID: \"cc49d276-ee8b-4aac-91cc-f9376bad14ff\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmthrk"
Jul 30 03:42:31.381484 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:31.381448    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jwcx\" (UniqueName: \"kubernetes.io/projected/cc49d276-ee8b-4aac-91cc-f9376bad14ff-kube-api-access-7jwcx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmthrk\" (UID: \"cc49d276-ee8b-4aac-91cc-f9376bad14ff\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmthrk"
Jul 30 03:42:31.497674 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:31.497574    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmthrk"
Jul 30 03:42:31.618709 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:31.618686    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmthrk"]
Jul 30 03:42:31.620905 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:42:31.620880    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc49d276_ee8b_4aac_91cc_f9376bad14ff.slice/crio-08cafb34a6ae7283a2f6dc0032aa22e2a2eaee00dd59aafa99908499299f24c6 WatchSource:0}: Error finding container 08cafb34a6ae7283a2f6dc0032aa22e2a2eaee00dd59aafa99908499299f24c6: Status 404 returned error can't find the container with id 08cafb34a6ae7283a2f6dc0032aa22e2a2eaee00dd59aafa99908499299f24c6
Jul 30 03:42:31.731355 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:31.731331    2576 generic.go:356] "Generic (PLEG): container finished" podID="cc49d276-ee8b-4aac-91cc-f9376bad14ff" containerID="17a0d83c1f5ca8e3368f661d0b1f83d2c4bcca063c4e3d214a4eb9cd2f7ac76a" exitCode=0
Jul 30 03:42:31.731478 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:31.731423    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmthrk" event={"ID":"cc49d276-ee8b-4aac-91cc-f9376bad14ff","Type":"ContainerDied","Data":"17a0d83c1f5ca8e3368f661d0b1f83d2c4bcca063c4e3d214a4eb9cd2f7ac76a"}
Jul 30 03:42:31.731536 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:31.731483    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmthrk" event={"ID":"cc49d276-ee8b-4aac-91cc-f9376bad14ff","Type":"ContainerStarted","Data":"08cafb34a6ae7283a2f6dc0032aa22e2a2eaee00dd59aafa99908499299f24c6"}
Jul 30 03:42:32.384206 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:32.384176    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:42:32.776442 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:32.776409    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:42:34.652332 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:34.652298    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:42:34.743996 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:34.743971    2576 generic.go:356] "Generic (PLEG): container finished" podID="cc49d276-ee8b-4aac-91cc-f9376bad14ff" containerID="7c970e13c1d193e99a147641ff49c276d66cc369c3fa80e93318a81726b7d869" exitCode=0
Jul 30 03:42:34.744141 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:34.744031    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmthrk" event={"ID":"cc49d276-ee8b-4aac-91cc-f9376bad14ff","Type":"ContainerDied","Data":"7c970e13c1d193e99a147641ff49c276d66cc369c3fa80e93318a81726b7d869"}
Jul 30 03:42:34.753256 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:34.753235    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:42:35.749492 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:35.749449    2576 generic.go:356] "Generic (PLEG): container finished" podID="cc49d276-ee8b-4aac-91cc-f9376bad14ff" containerID="76c4d1386b08077fadbb2f0d8dfb60b526256779b33ac1856928137cbac358e9" exitCode=0
Jul 30 03:42:35.749944 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:35.749548    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmthrk" event={"ID":"cc49d276-ee8b-4aac-91cc-f9376bad14ff","Type":"ContainerDied","Data":"76c4d1386b08077fadbb2f0d8dfb60b526256779b33ac1856928137cbac358e9"}
Jul 30 03:42:36.872186 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:36.872163    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmthrk"
Jul 30 03:42:37.015155 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:37.015079    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cc49d276-ee8b-4aac-91cc-f9376bad14ff-bundle\") pod \"cc49d276-ee8b-4aac-91cc-f9376bad14ff\" (UID: \"cc49d276-ee8b-4aac-91cc-f9376bad14ff\") "
Jul 30 03:42:37.015155 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:37.015116    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jwcx\" (UniqueName: \"kubernetes.io/projected/cc49d276-ee8b-4aac-91cc-f9376bad14ff-kube-api-access-7jwcx\") pod \"cc49d276-ee8b-4aac-91cc-f9376bad14ff\" (UID: \"cc49d276-ee8b-4aac-91cc-f9376bad14ff\") "
Jul 30 03:42:37.015350 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:37.015204    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cc49d276-ee8b-4aac-91cc-f9376bad14ff-util\") pod \"cc49d276-ee8b-4aac-91cc-f9376bad14ff\" (UID: \"cc49d276-ee8b-4aac-91cc-f9376bad14ff\") "
Jul 30 03:42:37.015560 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:37.015536    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc49d276-ee8b-4aac-91cc-f9376bad14ff-bundle" (OuterVolumeSpecName: "bundle") pod "cc49d276-ee8b-4aac-91cc-f9376bad14ff" (UID: "cc49d276-ee8b-4aac-91cc-f9376bad14ff"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 03:42:37.017207 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:37.017177    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc49d276-ee8b-4aac-91cc-f9376bad14ff-kube-api-access-7jwcx" (OuterVolumeSpecName: "kube-api-access-7jwcx") pod "cc49d276-ee8b-4aac-91cc-f9376bad14ff" (UID: "cc49d276-ee8b-4aac-91cc-f9376bad14ff"). InnerVolumeSpecName "kube-api-access-7jwcx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:42:37.020598 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:37.020482    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc49d276-ee8b-4aac-91cc-f9376bad14ff-util" (OuterVolumeSpecName: "util") pod "cc49d276-ee8b-4aac-91cc-f9376bad14ff" (UID: "cc49d276-ee8b-4aac-91cc-f9376bad14ff"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 03:42:37.116295 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:37.116268    2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cc49d276-ee8b-4aac-91cc-f9376bad14ff-util\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:42:37.116295 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:37.116292    2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cc49d276-ee8b-4aac-91cc-f9376bad14ff-bundle\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:42:37.116295 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:37.116302    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jwcx\" (UniqueName: \"kubernetes.io/projected/cc49d276-ee8b-4aac-91cc-f9376bad14ff-kube-api-access-7jwcx\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:42:37.759160 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:37.759129    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmthrk"
Jul 30 03:42:37.759160 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:37.759146    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmthrk" event={"ID":"cc49d276-ee8b-4aac-91cc-f9376bad14ff","Type":"ContainerDied","Data":"08cafb34a6ae7283a2f6dc0032aa22e2a2eaee00dd59aafa99908499299f24c6"}
Jul 30 03:42:37.759554 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:37.759195    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="08cafb34a6ae7283a2f6dc0032aa22e2a2eaee00dd59aafa99908499299f24c6"
Jul 30 03:42:40.668372 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:40.668339    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:42:40.764024 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:40.764000    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:42:41.181094 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:41.181054    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:42:41.838233 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:41.838194    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-4s2v7"]
Jul 30 03:42:41.838696 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:41.838522    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc49d276-ee8b-4aac-91cc-f9376bad14ff" containerName="pull"
Jul 30 03:42:41.838696 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:41.838538    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc49d276-ee8b-4aac-91cc-f9376bad14ff" containerName="pull"
Jul 30 03:42:41.838696 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:41.838567    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc49d276-ee8b-4aac-91cc-f9376bad14ff" containerName="extract"
Jul 30 03:42:41.838696 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:41.838575    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc49d276-ee8b-4aac-91cc-f9376bad14ff" containerName="extract"
Jul 30 03:42:41.838696 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:41.838590    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc49d276-ee8b-4aac-91cc-f9376bad14ff" containerName="util"
Jul 30 03:42:41.838696 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:41.838597    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc49d276-ee8b-4aac-91cc-f9376bad14ff" containerName="util"
Jul 30 03:42:41.838696 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:41.838657    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc49d276-ee8b-4aac-91cc-f9376bad14ff" containerName="extract"
Jul 30 03:42:41.840878 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:41.840858    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-4s2v7"
Jul 30 03:42:41.843217 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:41.843193    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 30 03:42:41.844325 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:41.844311    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-msqnr\""
Jul 30 03:42:41.844386 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:41.844313    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 30 03:42:41.850963 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:41.850939    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-4s2v7"]
Jul 30 03:42:41.948854 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:41.948815    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2rqn\" (UniqueName: \"kubernetes.io/projected/ddb0240f-d9cc-4d91-82be-967859eb2556-kube-api-access-m2rqn\") pod \"cert-manager-55b4d954d5-4s2v7\" (UID: \"ddb0240f-d9cc-4d91-82be-967859eb2556\") " pod="cert-manager/cert-manager-55b4d954d5-4s2v7"
Jul 30 03:42:41.948854 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:41.948864    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ddb0240f-d9cc-4d91-82be-967859eb2556-bound-sa-token\") pod \"cert-manager-55b4d954d5-4s2v7\" (UID: \"ddb0240f-d9cc-4d91-82be-967859eb2556\") " pod="cert-manager/cert-manager-55b4d954d5-4s2v7"
Jul 30 03:42:42.050174 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:42.050143    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ddb0240f-d9cc-4d91-82be-967859eb2556-bound-sa-token\") pod \"cert-manager-55b4d954d5-4s2v7\" (UID: \"ddb0240f-d9cc-4d91-82be-967859eb2556\") " pod="cert-manager/cert-manager-55b4d954d5-4s2v7"
Jul 30 03:42:42.050282 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:42.050205    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2rqn\" (UniqueName: \"kubernetes.io/projected/ddb0240f-d9cc-4d91-82be-967859eb2556-kube-api-access-m2rqn\") pod \"cert-manager-55b4d954d5-4s2v7\" (UID: \"ddb0240f-d9cc-4d91-82be-967859eb2556\") " pod="cert-manager/cert-manager-55b4d954d5-4s2v7"
Jul 30 03:42:42.058500 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:42.058469    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ddb0240f-d9cc-4d91-82be-967859eb2556-bound-sa-token\") pod \"cert-manager-55b4d954d5-4s2v7\" (UID: \"ddb0240f-d9cc-4d91-82be-967859eb2556\") " pod="cert-manager/cert-manager-55b4d954d5-4s2v7"
Jul 30 03:42:42.058750 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:42.058727    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2rqn\" (UniqueName: \"kubernetes.io/projected/ddb0240f-d9cc-4d91-82be-967859eb2556-kube-api-access-m2rqn\") pod \"cert-manager-55b4d954d5-4s2v7\" (UID: \"ddb0240f-d9cc-4d91-82be-967859eb2556\") " pod="cert-manager/cert-manager-55b4d954d5-4s2v7"
Jul 30 03:42:42.149505 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:42.149413    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-4s2v7"
Jul 30 03:42:42.268694 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:42.268671    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-4s2v7"]
Jul 30 03:42:42.271419 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:42:42.271394    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podddb0240f_d9cc_4d91_82be_967859eb2556.slice/crio-caa90cff74e9b84f9627fc51d4201145c627c817b053bea9374f2522fbe775fe WatchSource:0}: Error finding container caa90cff74e9b84f9627fc51d4201145c627c817b053bea9374f2522fbe775fe: Status 404 returned error can't find the container with id caa90cff74e9b84f9627fc51d4201145c627c817b053bea9374f2522fbe775fe
Jul 30 03:42:42.777880 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:42.777843    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-4s2v7" event={"ID":"ddb0240f-d9cc-4d91-82be-967859eb2556","Type":"ContainerStarted","Data":"caa90cff74e9b84f9627fc51d4201145c627c817b053bea9374f2522fbe775fe"}
Jul 30 03:42:44.787034 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:44.787006    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-4s2v7" event={"ID":"ddb0240f-d9cc-4d91-82be-967859eb2556","Type":"ContainerStarted","Data":"a90636a7414ccfba41b0e35cc944f5ced1e8fb9c2998f677b25ea07d294bcbe3"}
Jul 30 03:42:44.803204 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:44.803165    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-4s2v7" podStartSLOduration=1.363447887 podStartE2EDuration="3.803150315s" totalImagesPullingTime="2.439702428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:42:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:42:44.801911646 +0000 UTC m=+470.420143888" watchObservedRunningTime="2026-07-30 03:42:44.803150315 +0000 UTC m=+470.421382556"
Jul 30 03:42:47.064941 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:47.064893    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:42:47.167958 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:47.167913    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:42:48.752764 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:48.752727    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzp5w"]
Jul 30 03:42:48.755356 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:48.755333    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzp5w"
Jul 30 03:42:48.757797 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:48.757774    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 30 03:42:48.757899 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:48.757773    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 30 03:42:48.758825 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:48.758785    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-bsxn6\""
Jul 30 03:42:48.765686 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:48.765569    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzp5w"]
Jul 30 03:42:48.902374 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:48.902339    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzp5w\" (UID: \"7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzp5w"
Jul 30 03:42:48.902374 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:48.902373    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzp5w\" (UID: \"7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzp5w"
Jul 30 03:42:48.902588 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:48.902407    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9txvc\" (UniqueName: \"kubernetes.io/projected/7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b-kube-api-access-9txvc\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzp5w\" (UID: \"7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzp5w"
Jul 30 03:42:49.003646 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:49.003570    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9txvc\" (UniqueName: \"kubernetes.io/projected/7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b-kube-api-access-9txvc\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzp5w\" (UID: \"7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzp5w"
Jul 30 03:42:49.003646 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:49.003629    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzp5w\" (UID: \"7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzp5w"
Jul 30 03:42:49.003646 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:49.003645    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzp5w\" (UID: \"7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzp5w"
Jul 30 03:42:49.003989 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:49.003973    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzp5w\" (UID: \"7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzp5w"
Jul 30 03:42:49.004065 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:49.004047    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzp5w\" (UID: \"7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzp5w"
Jul 30 03:42:49.013244 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:49.013221    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9txvc\" (UniqueName: \"kubernetes.io/projected/7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b-kube-api-access-9txvc\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzp5w\" (UID: \"7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzp5w"
Jul 30 03:42:49.063786 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:49.063765    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzp5w"
Jul 30 03:42:49.189509 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:49.189487    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzp5w"]
Jul 30 03:42:49.191536 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:42:49.191508    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bb453d1_2c3f_4fd9_911e_1f36e9cb4e3b.slice/crio-bd1178a625049cb73476298694a8667305fee96fca993dc16198495c00a4121a WatchSource:0}: Error finding container bd1178a625049cb73476298694a8667305fee96fca993dc16198495c00a4121a: Status 404 returned error can't find the container with id bd1178a625049cb73476298694a8667305fee96fca993dc16198495c00a4121a
Jul 30 03:42:49.257591 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:49.257532    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:42:49.807163 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:49.807126    2576 generic.go:356] "Generic (PLEG): container finished" podID="7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b" containerID="b59f13cd1bdc14dedfd9ecfcd1585fd3734164840cc77213823a8331fc8b134e" exitCode=0
Jul 30 03:42:49.807535 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:49.807211    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzp5w" event={"ID":"7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b","Type":"ContainerDied","Data":"b59f13cd1bdc14dedfd9ecfcd1585fd3734164840cc77213823a8331fc8b134e"}
Jul 30 03:42:49.807535 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:49.807260    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzp5w" event={"ID":"7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b","Type":"ContainerStarted","Data":"bd1178a625049cb73476298694a8667305fee96fca993dc16198495c00a4121a"}
Jul 30 03:42:51.817329 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:51.817292    2576 generic.go:356] "Generic (PLEG): container finished" podID="7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b" containerID="89435afca1f41901b61c8914d3e8b849bfa67985baba380913f315acd96f2ac2" exitCode=0
Jul 30 03:42:51.817719 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:51.817376    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzp5w" event={"ID":"7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b","Type":"ContainerDied","Data":"89435afca1f41901b61c8914d3e8b849bfa67985baba380913f315acd96f2ac2"}
Jul 30 03:42:52.822877 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:52.822846    2576 generic.go:356] "Generic (PLEG): container finished" podID="7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b" containerID="d6db19d25fe357ab70cb764023f59522301b7015dd26232ffe37b67d225f8a22" exitCode=0
Jul 30 03:42:52.823353 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:52.822910    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzp5w" event={"ID":"7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b","Type":"ContainerDied","Data":"d6db19d25fe357ab70cb764023f59522301b7015dd26232ffe37b67d225f8a22"}
Jul 30 03:42:53.943039 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:53.943015    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzp5w"
Jul 30 03:42:54.044473 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:54.044429    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b-util\") pod \"7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b\" (UID: \"7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b\") "
Jul 30 03:42:54.044473 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:54.044474    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9txvc\" (UniqueName: \"kubernetes.io/projected/7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b-kube-api-access-9txvc\") pod \"7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b\" (UID: \"7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b\") "
Jul 30 03:42:54.044675 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:54.044494    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b-bundle\") pod \"7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b\" (UID: \"7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b\") "
Jul 30 03:42:54.045212 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:54.045180    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b-bundle" (OuterVolumeSpecName: "bundle") pod "7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b" (UID: "7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 03:42:54.046552 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:54.046528    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b-kube-api-access-9txvc" (OuterVolumeSpecName: "kube-api-access-9txvc") pod "7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b" (UID: "7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b"). InnerVolumeSpecName "kube-api-access-9txvc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:42:54.049717 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:54.049695    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b-util" (OuterVolumeSpecName: "util") pod "7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b" (UID: "7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 03:42:54.145367 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:54.145258    2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b-util\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:42:54.145367 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:54.145303    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9txvc\" (UniqueName: \"kubernetes.io/projected/7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b-kube-api-access-9txvc\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:42:54.145367 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:54.145316    2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b-bundle\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:42:54.832283 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:54.832257    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzp5w"
Jul 30 03:42:54.832283 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:54.832265    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzp5w" event={"ID":"7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b","Type":"ContainerDied","Data":"bd1178a625049cb73476298694a8667305fee96fca993dc16198495c00a4121a"}
Jul 30 03:42:54.832481 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:54.832301    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bd1178a625049cb73476298694a8667305fee96fca993dc16198495c00a4121a"
Jul 30 03:42:59.575946 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:59.575894    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:42:59.675793 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:42:59.675735    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:43:00.688213 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:00.688176    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:43:00.775510 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:00.775458    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:43:01.277220 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:01.277186    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:43:01.440910 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:01.440878    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:43:01.557672 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:01.557566    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:43:01.607083 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:01.607055    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5946c6c8fb-2h4sd"]
Jul 30 03:43:01.607294 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:01.607282    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b" containerName="pull"
Jul 30 03:43:01.607335 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:01.607295    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b" containerName="pull"
Jul 30 03:43:01.607335 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:01.607308    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b" containerName="util"
Jul 30 03:43:01.607335 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:01.607313    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b" containerName="util"
Jul 30 03:43:01.607335 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:01.607326    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b" containerName="extract"
Jul 30 03:43:01.607335 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:01.607331    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b" containerName="extract"
Jul 30 03:43:01.607482 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:01.607369    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bb453d1-2c3f-4fd9-911e-1f36e9cb4e3b" containerName="extract"
Jul 30 03:43:01.609881 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:01.609867    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5946c6c8fb-2h4sd"
Jul 30 03:43:01.612269 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:01.612249    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 30 03:43:01.613844 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:01.613824    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 30 03:43:01.613956 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:01.613850    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 30 03:43:01.614008 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:01.613976    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-rfxwq\""
Jul 30 03:43:01.614863 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:01.614844    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 30 03:43:01.627290 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:01.627271    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5946c6c8fb-2h4sd"]
Jul 30 03:43:01.695546 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:01.695513    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9bc8c99b-7801-4b28-a193-95d40194a98b-tmp\") pod \"opendatahub-operator-controller-manager-5946c6c8fb-2h4sd\" (UID: \"9bc8c99b-7801-4b28-a193-95d40194a98b\") " pod="opendatahub/opendatahub-operator-controller-manager-5946c6c8fb-2h4sd"
Jul 30 03:43:01.695546 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:01.695547    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9bc8c99b-7801-4b28-a193-95d40194a98b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5946c6c8fb-2h4sd\" (UID: \"9bc8c99b-7801-4b28-a193-95d40194a98b\") " pod="opendatahub/opendatahub-operator-controller-manager-5946c6c8fb-2h4sd"
Jul 30 03:43:01.695977 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:01.695567    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9bc8c99b-7801-4b28-a193-95d40194a98b-webhook-cert\") pod \"opendatahub-operator-controller-manager-5946c6c8fb-2h4sd\" (UID: \"9bc8c99b-7801-4b28-a193-95d40194a98b\") " pod="opendatahub/opendatahub-operator-controller-manager-5946c6c8fb-2h4sd"
Jul 30 03:43:01.695977 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:01.695600    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47ldk\" (UniqueName: \"kubernetes.io/projected/9bc8c99b-7801-4b28-a193-95d40194a98b-kube-api-access-47ldk\") pod \"opendatahub-operator-controller-manager-5946c6c8fb-2h4sd\" (UID: \"9bc8c99b-7801-4b28-a193-95d40194a98b\") " pod="opendatahub/opendatahub-operator-controller-manager-5946c6c8fb-2h4sd"
Jul 30 03:43:01.695977 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:01.695630    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9bc8c99b-7801-4b28-a193-95d40194a98b-manifests\") pod \"opendatahub-operator-controller-manager-5946c6c8fb-2h4sd\" (UID: \"9bc8c99b-7801-4b28-a193-95d40194a98b\") " pod="opendatahub/opendatahub-operator-controller-manager-5946c6c8fb-2h4sd"
Jul 30 03:43:01.750632 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:01.750600    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:43:01.795990 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:01.795960    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9bc8c99b-7801-4b28-a193-95d40194a98b-tmp\") pod \"opendatahub-operator-controller-manager-5946c6c8fb-2h4sd\" (UID: \"9bc8c99b-7801-4b28-a193-95d40194a98b\") " pod="opendatahub/opendatahub-operator-controller-manager-5946c6c8fb-2h4sd"
Jul 30 03:43:01.796105 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:01.796003    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9bc8c99b-7801-4b28-a193-95d40194a98b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5946c6c8fb-2h4sd\" (UID: \"9bc8c99b-7801-4b28-a193-95d40194a98b\") " pod="opendatahub/opendatahub-operator-controller-manager-5946c6c8fb-2h4sd"
Jul 30 03:43:01.796105 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:01.796024    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9bc8c99b-7801-4b28-a193-95d40194a98b-webhook-cert\") pod \"opendatahub-operator-controller-manager-5946c6c8fb-2h4sd\" (UID: \"9bc8c99b-7801-4b28-a193-95d40194a98b\") " pod="opendatahub/opendatahub-operator-controller-manager-5946c6c8fb-2h4sd"
Jul 30 03:43:01.796105 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:01.796040    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47ldk\" (UniqueName: \"kubernetes.io/projected/9bc8c99b-7801-4b28-a193-95d40194a98b-kube-api-access-47ldk\") pod \"opendatahub-operator-controller-manager-5946c6c8fb-2h4sd\" (UID: \"9bc8c99b-7801-4b28-a193-95d40194a98b\") " pod="opendatahub/opendatahub-operator-controller-manager-5946c6c8fb-2h4sd"
Jul 30 03:43:01.796105 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:01.796065    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9bc8c99b-7801-4b28-a193-95d40194a98b-manifests\") pod \"opendatahub-operator-controller-manager-5946c6c8fb-2h4sd\" (UID: \"9bc8c99b-7801-4b28-a193-95d40194a98b\") " pod="opendatahub/opendatahub-operator-controller-manager-5946c6c8fb-2h4sd"
Jul 30 03:43:01.796403 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:01.796385    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9bc8c99b-7801-4b28-a193-95d40194a98b-manifests\") pod \"opendatahub-operator-controller-manager-5946c6c8fb-2h4sd\" (UID: \"9bc8c99b-7801-4b28-a193-95d40194a98b\") " pod="opendatahub/opendatahub-operator-controller-manager-5946c6c8fb-2h4sd"
Jul 30 03:43:01.798274 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:01.798254    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9bc8c99b-7801-4b28-a193-95d40194a98b-tmp\") pod \"opendatahub-operator-controller-manager-5946c6c8fb-2h4sd\" (UID: \"9bc8c99b-7801-4b28-a193-95d40194a98b\") " pod="opendatahub/opendatahub-operator-controller-manager-5946c6c8fb-2h4sd"
Jul 30 03:43:01.798544 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:01.798528    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9bc8c99b-7801-4b28-a193-95d40194a98b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5946c6c8fb-2h4sd\" (UID: \"9bc8c99b-7801-4b28-a193-95d40194a98b\") " pod="opendatahub/opendatahub-operator-controller-manager-5946c6c8fb-2h4sd"
Jul 30 03:43:01.798591 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:01.798573    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9bc8c99b-7801-4b28-a193-95d40194a98b-webhook-cert\") pod \"opendatahub-operator-controller-manager-5946c6c8fb-2h4sd\" (UID: \"9bc8c99b-7801-4b28-a193-95d40194a98b\") " pod="opendatahub/opendatahub-operator-controller-manager-5946c6c8fb-2h4sd"
Jul 30 03:43:01.805034 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:01.805012    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47ldk\" (UniqueName: \"kubernetes.io/projected/9bc8c99b-7801-4b28-a193-95d40194a98b-kube-api-access-47ldk\") pod \"opendatahub-operator-controller-manager-5946c6c8fb-2h4sd\" (UID: \"9bc8c99b-7801-4b28-a193-95d40194a98b\") " pod="opendatahub/opendatahub-operator-controller-manager-5946c6c8fb-2h4sd"
Jul 30 03:43:01.852227 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:01.852136    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:43:01.920031 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:01.920000    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5946c6c8fb-2h4sd"
Jul 30 03:43:01.952242 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:01.952209    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:43:02.050173 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:02.050141    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5946c6c8fb-2h4sd"]
Jul 30 03:43:02.052838 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:02.052817    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:43:02.055172 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:43:02.055150    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bc8c99b_7801_4b28_a193_95d40194a98b.slice/crio-ece847cd0f37e3d2e38849dadb1670039fe9f3ea6cbd329a4e7a3f61ecb763a0 WatchSource:0}: Error finding container ece847cd0f37e3d2e38849dadb1670039fe9f3ea6cbd329a4e7a3f61ecb763a0: Status 404 returned error can't find the container with id ece847cd0f37e3d2e38849dadb1670039fe9f3ea6cbd329a4e7a3f61ecb763a0
Jul 30 03:43:02.152412 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:02.152318    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:43:02.865993 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:02.865958    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5946c6c8fb-2h4sd" event={"ID":"9bc8c99b-7801-4b28-a193-95d40194a98b","Type":"ContainerStarted","Data":"ece847cd0f37e3d2e38849dadb1670039fe9f3ea6cbd329a4e7a3f61ecb763a0"}
Jul 30 03:43:05.881094 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:05.881059    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5946c6c8fb-2h4sd" event={"ID":"9bc8c99b-7801-4b28-a193-95d40194a98b","Type":"ContainerStarted","Data":"e47312bc053faee3bd44315bf6d01464c69c641af27b7714f81bf1d525cc9662"}
Jul 30 03:43:06.886698 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:06.886665    2576 generic.go:356] "Generic (PLEG): container finished" podID="9bc8c99b-7801-4b28-a193-95d40194a98b" containerID="e47312bc053faee3bd44315bf6d01464c69c641af27b7714f81bf1d525cc9662" exitCode=0
Jul 30 03:43:06.887105 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:06.886751    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5946c6c8fb-2h4sd" event={"ID":"9bc8c99b-7801-4b28-a193-95d40194a98b","Type":"ContainerDied","Data":"e47312bc053faee3bd44315bf6d01464c69c641af27b7714f81bf1d525cc9662"}
Jul 30 03:43:07.892953 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:07.892907    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5946c6c8fb-2h4sd" event={"ID":"9bc8c99b-7801-4b28-a193-95d40194a98b","Type":"ContainerStarted","Data":"cb114e5b1d5bb9fa439a74a07ff0cc0aa4ccc2f96b156c49cc026de53b570a58"}
Jul 30 03:43:07.893335 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:07.893055    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5946c6c8fb-2h4sd"
Jul 30 03:43:07.915764 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:07.915720    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5946c6c8fb-2h4sd" podStartSLOduration=2.447881461 podStartE2EDuration="6.915706188s" totalImagesPullingTime="3.382286621s" totalInitContainerRuntime="1.085538106s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:43:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:43:07.914090893 +0000 UTC m=+493.532323142" watchObservedRunningTime="2026-07-30 03:43:07.915706188 +0000 UTC m=+493.533938428"
Jul 30 03:43:09.460369 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:09.460333    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:43:09.575594 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:09.575564    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:43:18.899225 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:18.899196    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5946c6c8fb-2h4sd"
Jul 30 03:43:21.011629 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:21.011564    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:43:21.130509 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:21.130474    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:43:21.738289 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:21.738259    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:43:21.837753 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:21.837724    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:43:22.007207 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:22.007126    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:43:22.108338 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:22.108303    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:43:22.345982 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:22.345950    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:43:22.652556 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:22.652478    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:43:26.864520 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:26.864493    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:43:26.937207 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:26.937174    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-fb6bd797-dh9vd"]
Jul 30 03:43:26.940932 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:26.940901    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-fb6bd797-dh9vd"
Jul 30 03:43:26.943599 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:26.943576    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 30 03:43:26.943710 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:26.943584    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-z496c\""
Jul 30 03:43:26.944751 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:26.944733    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 30 03:43:26.944839 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:26.944736    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 30 03:43:26.944839 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:26.944776    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 30 03:43:26.950803 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:26.950785    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-fb6bd797-dh9vd"]
Jul 30 03:43:26.982266 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:26.982239    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc92711f-ae9e-46fa-b2e5-49c9f392b846-tls-certs\") pod \"kube-auth-proxy-fb6bd797-dh9vd\" (UID: \"fc92711f-ae9e-46fa-b2e5-49c9f392b846\") " pod="openshift-ingress/kube-auth-proxy-fb6bd797-dh9vd"
Jul 30 03:43:26.982423 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:26.982274    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc92711f-ae9e-46fa-b2e5-49c9f392b846-tmp\") pod \"kube-auth-proxy-fb6bd797-dh9vd\" (UID: \"fc92711f-ae9e-46fa-b2e5-49c9f392b846\") " pod="openshift-ingress/kube-auth-proxy-fb6bd797-dh9vd"
Jul 30 03:43:26.982423 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:26.982313    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbk26\" (UniqueName: \"kubernetes.io/projected/fc92711f-ae9e-46fa-b2e5-49c9f392b846-kube-api-access-bbk26\") pod \"kube-auth-proxy-fb6bd797-dh9vd\" (UID: \"fc92711f-ae9e-46fa-b2e5-49c9f392b846\") " pod="openshift-ingress/kube-auth-proxy-fb6bd797-dh9vd"
Jul 30 03:43:27.083014 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:27.082973    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbk26\" (UniqueName: \"kubernetes.io/projected/fc92711f-ae9e-46fa-b2e5-49c9f392b846-kube-api-access-bbk26\") pod \"kube-auth-proxy-fb6bd797-dh9vd\" (UID: \"fc92711f-ae9e-46fa-b2e5-49c9f392b846\") " pod="openshift-ingress/kube-auth-proxy-fb6bd797-dh9vd"
Jul 30 03:43:27.083184 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:27.083071    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc92711f-ae9e-46fa-b2e5-49c9f392b846-tls-certs\") pod \"kube-auth-proxy-fb6bd797-dh9vd\" (UID: \"fc92711f-ae9e-46fa-b2e5-49c9f392b846\") " pod="openshift-ingress/kube-auth-proxy-fb6bd797-dh9vd"
Jul 30 03:43:27.083184 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:27.083099    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc92711f-ae9e-46fa-b2e5-49c9f392b846-tmp\") pod \"kube-auth-proxy-fb6bd797-dh9vd\" (UID: \"fc92711f-ae9e-46fa-b2e5-49c9f392b846\") " pod="openshift-ingress/kube-auth-proxy-fb6bd797-dh9vd"
Jul 30 03:43:27.083251 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:43:27.083223    2576 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 30 03:43:27.083318 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:43:27.083307    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc92711f-ae9e-46fa-b2e5-49c9f392b846-tls-certs podName:fc92711f-ae9e-46fa-b2e5-49c9f392b846 nodeName:}" failed. No retries permitted until 2026-07-30 03:43:27.583290536 +0000 UTC m=+513.201522767 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/fc92711f-ae9e-46fa-b2e5-49c9f392b846-tls-certs") pod "kube-auth-proxy-fb6bd797-dh9vd" (UID: "fc92711f-ae9e-46fa-b2e5-49c9f392b846") : secret "kube-auth-proxy-tls" not found
Jul 30 03:43:27.085374 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:27.085348    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc92711f-ae9e-46fa-b2e5-49c9f392b846-tmp\") pod \"kube-auth-proxy-fb6bd797-dh9vd\" (UID: \"fc92711f-ae9e-46fa-b2e5-49c9f392b846\") " pod="openshift-ingress/kube-auth-proxy-fb6bd797-dh9vd"
Jul 30 03:43:27.092297 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:27.092279    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbk26\" (UniqueName: \"kubernetes.io/projected/fc92711f-ae9e-46fa-b2e5-49c9f392b846-kube-api-access-bbk26\") pod \"kube-auth-proxy-fb6bd797-dh9vd\" (UID: \"fc92711f-ae9e-46fa-b2e5-49c9f392b846\") " pod="openshift-ingress/kube-auth-proxy-fb6bd797-dh9vd"
Jul 30 03:43:27.278259 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:27.278228    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:43:27.588227 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:27.588124    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc92711f-ae9e-46fa-b2e5-49c9f392b846-tls-certs\") pod \"kube-auth-proxy-fb6bd797-dh9vd\" (UID: \"fc92711f-ae9e-46fa-b2e5-49c9f392b846\") " pod="openshift-ingress/kube-auth-proxy-fb6bd797-dh9vd"
Jul 30 03:43:27.590539 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:27.590511    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc92711f-ae9e-46fa-b2e5-49c9f392b846-tls-certs\") pod \"kube-auth-proxy-fb6bd797-dh9vd\" (UID: \"fc92711f-ae9e-46fa-b2e5-49c9f392b846\") " pod="openshift-ingress/kube-auth-proxy-fb6bd797-dh9vd"
Jul 30 03:43:27.851344 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:27.851239    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-fb6bd797-dh9vd"
Jul 30 03:43:28.001708 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:28.001586    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-fb6bd797-dh9vd"]
Jul 30 03:43:28.004357 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:43:28.004334    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc92711f_ae9e_46fa_b2e5_49c9f392b846.slice/crio-b946faaa4a197ff34bec9b45748814750d9fa36735143e451511c098cab81013 WatchSource:0}: Error finding container b946faaa4a197ff34bec9b45748814750d9fa36735143e451511c098cab81013: Status 404 returned error can't find the container with id b946faaa4a197ff34bec9b45748814750d9fa36735143e451511c098cab81013
Jul 30 03:43:28.088660 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:28.088627    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:43:28.094149 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:28.094120    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5"]
Jul 30 03:43:28.098262 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:28.098242    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5"
Jul 30 03:43:28.101013 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:28.100993    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 30 03:43:28.101096 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:28.101081    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-kw97h\""
Jul 30 03:43:28.101864 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:28.101820    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 30 03:43:28.101864 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:28.101829    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 30 03:43:28.102019 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:28.101826    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 30 03:43:28.119062 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:28.119039    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5"]
Jul 30 03:43:28.192433 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:28.192403    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fea286df-f944-43ff-a298-33503b311a37-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-wgks5\" (UID: \"fea286df-f944-43ff-a298-33503b311a37\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5"
Jul 30 03:43:28.192787 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:28.192449    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fea286df-f944-43ff-a298-33503b311a37-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-wgks5\" (UID: \"fea286df-f944-43ff-a298-33503b311a37\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5"
Jul 30 03:43:28.192787 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:28.192468    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fea286df-f944-43ff-a298-33503b311a37-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-wgks5\" (UID: \"fea286df-f944-43ff-a298-33503b311a37\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5"
Jul 30 03:43:28.192787 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:28.192549    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fea286df-f944-43ff-a298-33503b311a37-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-wgks5\" (UID: \"fea286df-f944-43ff-a298-33503b311a37\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5"
Jul 30 03:43:28.192787 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:28.192590    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bqlp\" (UniqueName: \"kubernetes.io/projected/fea286df-f944-43ff-a298-33503b311a37-kube-api-access-7bqlp\") pod \"istiod-openshift-gateway-7b7995d5cd-wgks5\" (UID: \"fea286df-f944-43ff-a298-33503b311a37\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5"
Jul 30 03:43:28.192787 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:28.192632    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fea286df-f944-43ff-a298-33503b311a37-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-wgks5\" (UID: \"fea286df-f944-43ff-a298-33503b311a37\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5"
Jul 30 03:43:28.192787 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:28.192651    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fea286df-f944-43ff-a298-33503b311a37-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-wgks5\" (UID: \"fea286df-f944-43ff-a298-33503b311a37\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5"
Jul 30 03:43:28.204162 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:28.204140    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:43:28.293311 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:28.293279    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fea286df-f944-43ff-a298-33503b311a37-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-wgks5\" (UID: \"fea286df-f944-43ff-a298-33503b311a37\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5"
Jul 30 03:43:28.293311 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:28.293312    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fea286df-f944-43ff-a298-33503b311a37-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-wgks5\" (UID: \"fea286df-f944-43ff-a298-33503b311a37\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5"
Jul 30 03:43:28.293548 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:28.293333    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fea286df-f944-43ff-a298-33503b311a37-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-wgks5\" (UID: \"fea286df-f944-43ff-a298-33503b311a37\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5"
Jul 30 03:43:28.293548 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:28.293365    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fea286df-f944-43ff-a298-33503b311a37-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-wgks5\" (UID: \"fea286df-f944-43ff-a298-33503b311a37\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5"
Jul 30 03:43:28.293548 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:28.293389    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fea286df-f944-43ff-a298-33503b311a37-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-wgks5\" (UID: \"fea286df-f944-43ff-a298-33503b311a37\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5"
Jul 30 03:43:28.293548 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:28.293437    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fea286df-f944-43ff-a298-33503b311a37-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-wgks5\" (UID: \"fea286df-f944-43ff-a298-33503b311a37\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5"
Jul 30 03:43:28.293548 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:28.293482    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bqlp\" (UniqueName: \"kubernetes.io/projected/fea286df-f944-43ff-a298-33503b311a37-kube-api-access-7bqlp\") pod \"istiod-openshift-gateway-7b7995d5cd-wgks5\" (UID: \"fea286df-f944-43ff-a298-33503b311a37\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5"
Jul 30 03:43:28.294100 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:28.294072    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fea286df-f944-43ff-a298-33503b311a37-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-wgks5\" (UID: \"fea286df-f944-43ff-a298-33503b311a37\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5"
Jul 30 03:43:28.295724 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:28.295693    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fea286df-f944-43ff-a298-33503b311a37-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-wgks5\" (UID: \"fea286df-f944-43ff-a298-33503b311a37\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5"
Jul 30 03:43:28.295724 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:28.295708    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fea286df-f944-43ff-a298-33503b311a37-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-wgks5\" (UID: \"fea286df-f944-43ff-a298-33503b311a37\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5"
Jul 30 03:43:28.295837 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:28.295743    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fea286df-f944-43ff-a298-33503b311a37-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-wgks5\" (UID: \"fea286df-f944-43ff-a298-33503b311a37\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5"
Jul 30 03:43:28.295837 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:28.295808    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fea286df-f944-43ff-a298-33503b311a37-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-wgks5\" (UID: \"fea286df-f944-43ff-a298-33503b311a37\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5"
Jul 30 03:43:28.301315 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:28.301285    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fea286df-f944-43ff-a298-33503b311a37-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-wgks5\" (UID: \"fea286df-f944-43ff-a298-33503b311a37\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5"
Jul 30 03:43:28.301573 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:28.301556    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bqlp\" (UniqueName: \"kubernetes.io/projected/fea286df-f944-43ff-a298-33503b311a37-kube-api-access-7bqlp\") pod \"istiod-openshift-gateway-7b7995d5cd-wgks5\" (UID: \"fea286df-f944-43ff-a298-33503b311a37\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5"
Jul 30 03:43:28.408022 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:28.407940    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5"
Jul 30 03:43:28.556009 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:28.555978    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5"]
Jul 30 03:43:28.557804 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:43:28.557772    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfea286df_f944_43ff_a298_33503b311a37.slice/crio-78e46d07cf5cd2e1a8c34f61448f9cc1a99cb0fe79e884ea944be2d434739d40 WatchSource:0}: Error finding container 78e46d07cf5cd2e1a8c34f61448f9cc1a99cb0fe79e884ea944be2d434739d40: Status 404 returned error can't find the container with id 78e46d07cf5cd2e1a8c34f61448f9cc1a99cb0fe79e884ea944be2d434739d40
Jul 30 03:43:28.999486 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:28.999454    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-fb6bd797-dh9vd" event={"ID":"fc92711f-ae9e-46fa-b2e5-49c9f392b846","Type":"ContainerStarted","Data":"b946faaa4a197ff34bec9b45748814750d9fa36735143e451511c098cab81013"}
Jul 30 03:43:29.000738 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:29.000692    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5" event={"ID":"fea286df-f944-43ff-a298-33503b311a37","Type":"ContainerStarted","Data":"78e46d07cf5cd2e1a8c34f61448f9cc1a99cb0fe79e884ea944be2d434739d40"}
Jul 30 03:43:29.407225 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:29.407156    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:43:31.324550 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:31.324517    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 03:43:31.324853 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:31.324582    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 03:43:31.740241 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:31.740163    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:43:31.801018 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:31.800984    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-d42fv"]
Jul 30 03:43:31.804726 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:31.804705    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-d42fv"
Jul 30 03:43:31.807291 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:31.807266    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 30 03:43:31.807427 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:31.807345    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-bclgq\""
Jul 30 03:43:31.823638 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:31.823614    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-d42fv"]
Jul 30 03:43:31.840778 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:31.840750    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:43:31.924676 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:31.924643    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/39e8666b-7a30-4644-8979-dc793338ba98-cert\") pod \"kserve-controller-manager-d6f4b65b8-d42fv\" (UID: \"39e8666b-7a30-4644-8979-dc793338ba98\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-d42fv"
Jul 30 03:43:31.924870 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:31.924695    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4wt9\" (UniqueName: \"kubernetes.io/projected/39e8666b-7a30-4644-8979-dc793338ba98-kube-api-access-j4wt9\") pod \"kserve-controller-manager-d6f4b65b8-d42fv\" (UID: \"39e8666b-7a30-4644-8979-dc793338ba98\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-d42fv"
Jul 30 03:43:31.948112 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:31.948079    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:43:32.017426 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:32.017337    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-fb6bd797-dh9vd" event={"ID":"fc92711f-ae9e-46fa-b2e5-49c9f392b846","Type":"ContainerStarted","Data":"afead3deda0ff46c7bf7457515faa2d364cce96c127df6794d7771323b2cf1fe"}
Jul 30 03:43:32.018870 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:32.018845    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5" event={"ID":"fea286df-f944-43ff-a298-33503b311a37","Type":"ContainerStarted","Data":"59a7d8f7c273219f637ce82b3c79a9f4406754bd693957ae5bec16df581b9c8b"}
Jul 30 03:43:32.019059 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:32.019048    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5"
Jul 30 03:43:32.020443 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:32.020418    2576 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-wgks5 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 30 03:43:32.020540 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:32.020473    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5" podUID="fea286df-f944-43ff-a298-33503b311a37" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 03:43:32.026020 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:32.025999    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/39e8666b-7a30-4644-8979-dc793338ba98-cert\") pod \"kserve-controller-manager-d6f4b65b8-d42fv\" (UID: \"39e8666b-7a30-4644-8979-dc793338ba98\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-d42fv"
Jul 30 03:43:32.026132 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:32.026052    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4wt9\" (UniqueName: \"kubernetes.io/projected/39e8666b-7a30-4644-8979-dc793338ba98-kube-api-access-j4wt9\") pod \"kserve-controller-manager-d6f4b65b8-d42fv\" (UID: \"39e8666b-7a30-4644-8979-dc793338ba98\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-d42fv"
Jul 30 03:43:32.026195 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:43:32.026144    2576 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 30 03:43:32.026251 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:43:32.026207    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39e8666b-7a30-4644-8979-dc793338ba98-cert podName:39e8666b-7a30-4644-8979-dc793338ba98 nodeName:}" failed. No retries permitted until 2026-07-30 03:43:32.526187973 +0000 UTC m=+518.144420195 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/39e8666b-7a30-4644-8979-dc793338ba98-cert") pod "kserve-controller-manager-d6f4b65b8-d42fv" (UID: "39e8666b-7a30-4644-8979-dc793338ba98") : secret "kserve-webhook-server-cert" not found
Jul 30 03:43:32.036547 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:32.036521    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4wt9\" (UniqueName: \"kubernetes.io/projected/39e8666b-7a30-4644-8979-dc793338ba98-kube-api-access-j4wt9\") pod \"kserve-controller-manager-d6f4b65b8-d42fv\" (UID: \"39e8666b-7a30-4644-8979-dc793338ba98\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-d42fv"
Jul 30 03:43:32.058895 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:32.058857    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-fb6bd797-dh9vd" podStartSLOduration=2.742303762 podStartE2EDuration="6.058844281s" totalImagesPullingTime="3.316540519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:43:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:43:32.056678859 +0000 UTC m=+517.674911099" watchObservedRunningTime="2026-07-30 03:43:32.058844281 +0000 UTC m=+517.677076521"
Jul 30 03:43:32.096980 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:32.096713    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5" podStartSLOduration=1.332075847 podStartE2EDuration="4.09669117s" totalImagesPullingTime="2.764615323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:43:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:43:32.095778588 +0000 UTC m=+517.714010830" watchObservedRunningTime="2026-07-30 03:43:32.09669117 +0000 UTC m=+517.714923415"
Jul 30 03:43:32.282812 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:32.282779    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:43:32.383223 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:32.383192    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:43:32.529764 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:32.529723    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/39e8666b-7a30-4644-8979-dc793338ba98-cert\") pod \"kserve-controller-manager-d6f4b65b8-d42fv\" (UID: \"39e8666b-7a30-4644-8979-dc793338ba98\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-d42fv"
Jul 30 03:43:32.532082 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:32.532062    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/39e8666b-7a30-4644-8979-dc793338ba98-cert\") pod \"kserve-controller-manager-d6f4b65b8-d42fv\" (UID: \"39e8666b-7a30-4644-8979-dc793338ba98\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-d42fv"
Jul 30 03:43:32.714713 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:32.714613    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-d42fv"
Jul 30 03:43:32.838710 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:32.838680    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-d42fv"]
Jul 30 03:43:32.841740 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:43:32.841713    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39e8666b_7a30_4644_8979_dc793338ba98.slice/crio-e697c4110e63c55be50582e037c0bafb65fa4558ac282aac0324e122852f3eaf WatchSource:0}: Error finding container e697c4110e63c55be50582e037c0bafb65fa4558ac282aac0324e122852f3eaf: Status 404 returned error can't find the container with id e697c4110e63c55be50582e037c0bafb65fa4558ac282aac0324e122852f3eaf
Jul 30 03:43:33.024030 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.023998    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-d42fv" event={"ID":"39e8666b-7a30-4644-8979-dc793338ba98","Type":"ContainerStarted","Data":"e697c4110e63c55be50582e037c0bafb65fa4558ac282aac0324e122852f3eaf"}
Jul 30 03:43:33.024756 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.024735    2576 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-wgks5 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 30 03:43:33.024870 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.024780    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5" podUID="fea286df-f944-43ff-a298-33503b311a37" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 03:43:33.085443 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.085416    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:43:33.152168 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.152138    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl"]
Jul 30 03:43:33.156447 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.156430    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl"
Jul 30 03:43:33.161127 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.161106    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-crwp2\""
Jul 30 03:43:33.186299 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.186275    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl"]
Jul 30 03:43:33.232335 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.232302    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:43:33.236361 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.236336    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4bwj\" (UniqueName: \"kubernetes.io/projected/244fd030-1879-462b-8307-7c9de84f4666-kube-api-access-q4bwj\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl\" (UID: \"244fd030-1879-462b-8307-7c9de84f4666\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl"
Jul 30 03:43:33.236518 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.236370    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/244fd030-1879-462b-8307-7c9de84f4666-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl\" (UID: \"244fd030-1879-462b-8307-7c9de84f4666\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl"
Jul 30 03:43:33.236518 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.236398    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/244fd030-1879-462b-8307-7c9de84f4666-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl\" (UID: \"244fd030-1879-462b-8307-7c9de84f4666\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl"
Jul 30 03:43:33.236518 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.236459    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/244fd030-1879-462b-8307-7c9de84f4666-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl\" (UID: \"244fd030-1879-462b-8307-7c9de84f4666\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl"
Jul 30 03:43:33.236518 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.236507    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/244fd030-1879-462b-8307-7c9de84f4666-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl\" (UID: \"244fd030-1879-462b-8307-7c9de84f4666\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl"
Jul 30 03:43:33.236656 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.236537    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/244fd030-1879-462b-8307-7c9de84f4666-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl\" (UID: \"244fd030-1879-462b-8307-7c9de84f4666\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl"
Jul 30 03:43:33.236656 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.236580    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/244fd030-1879-462b-8307-7c9de84f4666-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl\" (UID: \"244fd030-1879-462b-8307-7c9de84f4666\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl"
Jul 30 03:43:33.236656 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.236607    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/244fd030-1879-462b-8307-7c9de84f4666-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl\" (UID: \"244fd030-1879-462b-8307-7c9de84f4666\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl"
Jul 30 03:43:33.236656 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.236641    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/244fd030-1879-462b-8307-7c9de84f4666-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl\" (UID: \"244fd030-1879-462b-8307-7c9de84f4666\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl"
Jul 30 03:43:33.337279 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.337201    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/244fd030-1879-462b-8307-7c9de84f4666-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl\" (UID: \"244fd030-1879-462b-8307-7c9de84f4666\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl"
Jul 30 03:43:33.337279 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.337237    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/244fd030-1879-462b-8307-7c9de84f4666-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl\" (UID: \"244fd030-1879-462b-8307-7c9de84f4666\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl"
Jul 30 03:43:33.337279 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.337267    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/244fd030-1879-462b-8307-7c9de84f4666-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl\" (UID: \"244fd030-1879-462b-8307-7c9de84f4666\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl"
Jul 30 03:43:33.337563 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.337290    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/244fd030-1879-462b-8307-7c9de84f4666-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl\" (UID: \"244fd030-1879-462b-8307-7c9de84f4666\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl"
Jul 30 03:43:33.337563 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.337417    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4bwj\" (UniqueName: \"kubernetes.io/projected/244fd030-1879-462b-8307-7c9de84f4666-kube-api-access-q4bwj\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl\" (UID: \"244fd030-1879-462b-8307-7c9de84f4666\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl"
Jul 30 03:43:33.337563 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.337476    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/244fd030-1879-462b-8307-7c9de84f4666-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl\" (UID: \"244fd030-1879-462b-8307-7c9de84f4666\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl"
Jul 30 03:43:33.337563 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.337525    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/244fd030-1879-462b-8307-7c9de84f4666-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl\" (UID: \"244fd030-1879-462b-8307-7c9de84f4666\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl"
Jul 30 03:43:33.337762 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.337562    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/244fd030-1879-462b-8307-7c9de84f4666-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl\" (UID: \"244fd030-1879-462b-8307-7c9de84f4666\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl"
Jul 30 03:43:33.337762 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.337629    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/244fd030-1879-462b-8307-7c9de84f4666-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl\" (UID: \"244fd030-1879-462b-8307-7c9de84f4666\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl"
Jul 30 03:43:33.337762 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.337645    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/244fd030-1879-462b-8307-7c9de84f4666-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl\" (UID: \"244fd030-1879-462b-8307-7c9de84f4666\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl"
Jul 30 03:43:33.337762 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.337693    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/244fd030-1879-462b-8307-7c9de84f4666-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl\" (UID: \"244fd030-1879-462b-8307-7c9de84f4666\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl"
Jul 30 03:43:33.338016 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.337806    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/244fd030-1879-462b-8307-7c9de84f4666-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl\" (UID: \"244fd030-1879-462b-8307-7c9de84f4666\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl"
Jul 30 03:43:33.338069 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.338011    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/244fd030-1879-462b-8307-7c9de84f4666-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl\" (UID: \"244fd030-1879-462b-8307-7c9de84f4666\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl"
Jul 30 03:43:33.338188 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.338169    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/244fd030-1879-462b-8307-7c9de84f4666-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl\" (UID: \"244fd030-1879-462b-8307-7c9de84f4666\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl"
Jul 30 03:43:33.339504 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.339487    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/244fd030-1879-462b-8307-7c9de84f4666-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl\" (UID: \"244fd030-1879-462b-8307-7c9de84f4666\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl"
Jul 30 03:43:33.339858 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.339842    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/244fd030-1879-462b-8307-7c9de84f4666-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl\" (UID: \"244fd030-1879-462b-8307-7c9de84f4666\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl"
Jul 30 03:43:33.346535 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.346512    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/244fd030-1879-462b-8307-7c9de84f4666-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl\" (UID: \"244fd030-1879-462b-8307-7c9de84f4666\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl"
Jul 30 03:43:33.346743 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.346722    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4bwj\" (UniqueName: \"kubernetes.io/projected/244fd030-1879-462b-8307-7c9de84f4666-kube-api-access-q4bwj\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl\" (UID: \"244fd030-1879-462b-8307-7c9de84f4666\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl"
Jul 30 03:43:33.361152 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.361130    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:43:33.466366 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.466331    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl"
Jul 30 03:43:33.574285 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.574180    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:43:33.616329 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:33.616296    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl"]
Jul 30 03:43:33.617954 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:43:33.617880    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod244fd030_1879_462b_8307_7c9de84f4666.slice/crio-40bc608838fb8ff2b0ea8b5ae225538fe02fd2a3eb8aae0ff474165dce74231f WatchSource:0}: Error finding container 40bc608838fb8ff2b0ea8b5ae225538fe02fd2a3eb8aae0ff474165dce74231f: Status 404 returned error can't find the container with id 40bc608838fb8ff2b0ea8b5ae225538fe02fd2a3eb8aae0ff474165dce74231f
Jul 30 03:43:34.030914 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:34.030885    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl" event={"ID":"244fd030-1879-462b-8307-7c9de84f4666","Type":"ContainerStarted","Data":"40bc608838fb8ff2b0ea8b5ae225538fe02fd2a3eb8aae0ff474165dce74231f"}
Jul 30 03:43:34.597125 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:34.597090    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5"]
Jul 30 03:43:34.597611 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:34.597313    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5" podUID="fea286df-f944-43ff-a298-33503b311a37" containerName="discovery" containerID="cri-o://59a7d8f7c273219f637ce82b3c79a9f4406754bd693957ae5bec16df581b9c8b" gracePeriod=30
Jul 30 03:43:34.598196 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:34.598173    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5"
Jul 30 03:43:35.560157 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:35.560120    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:43:36.043379 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:36.043343    2576 generic.go:356] "Generic (PLEG): container finished" podID="fea286df-f944-43ff-a298-33503b311a37" containerID="59a7d8f7c273219f637ce82b3c79a9f4406754bd693957ae5bec16df581b9c8b" exitCode=0
Jul 30 03:43:36.043812 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:36.043393    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5" event={"ID":"fea286df-f944-43ff-a298-33503b311a37","Type":"ContainerDied","Data":"59a7d8f7c273219f637ce82b3c79a9f4406754bd693957ae5bec16df581b9c8b"}
Jul 30 03:43:36.291800 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:36.291772    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5"
Jul 30 03:43:36.368539 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:36.368499    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fea286df-f944-43ff-a298-33503b311a37-cacerts\") pod \"fea286df-f944-43ff-a298-33503b311a37\" (UID: \"fea286df-f944-43ff-a298-33503b311a37\") "
Jul 30 03:43:36.368539 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:36.368548    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fea286df-f944-43ff-a298-33503b311a37-istio-kubeconfig\") pod \"fea286df-f944-43ff-a298-33503b311a37\" (UID: \"fea286df-f944-43ff-a298-33503b311a37\") "
Jul 30 03:43:36.368770 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:36.368599    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fea286df-f944-43ff-a298-33503b311a37-istio-csr-ca-configmap\") pod \"fea286df-f944-43ff-a298-33503b311a37\" (UID: \"fea286df-f944-43ff-a298-33503b311a37\") "
Jul 30 03:43:36.368770 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:36.368627    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7bqlp\" (UniqueName: \"kubernetes.io/projected/fea286df-f944-43ff-a298-33503b311a37-kube-api-access-7bqlp\") pod \"fea286df-f944-43ff-a298-33503b311a37\" (UID: \"fea286df-f944-43ff-a298-33503b311a37\") "
Jul 30 03:43:36.368770 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:36.368645    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fea286df-f944-43ff-a298-33503b311a37-local-certs\") pod \"fea286df-f944-43ff-a298-33503b311a37\" (UID: \"fea286df-f944-43ff-a298-33503b311a37\") "
Jul 30 03:43:36.368770 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:36.368679    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fea286df-f944-43ff-a298-33503b311a37-istio-csr-dns-cert\") pod \"fea286df-f944-43ff-a298-33503b311a37\" (UID: \"fea286df-f944-43ff-a298-33503b311a37\") "
Jul 30 03:43:36.368770 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:36.368725    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fea286df-f944-43ff-a298-33503b311a37-istio-token\") pod \"fea286df-f944-43ff-a298-33503b311a37\" (UID: \"fea286df-f944-43ff-a298-33503b311a37\") "
Jul 30 03:43:36.369056 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:36.368991    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fea286df-f944-43ff-a298-33503b311a37-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "fea286df-f944-43ff-a298-33503b311a37" (UID: "fea286df-f944-43ff-a298-33503b311a37"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 03:43:36.371283 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:36.371256    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fea286df-f944-43ff-a298-33503b311a37-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "fea286df-f944-43ff-a298-33503b311a37" (UID: "fea286df-f944-43ff-a298-33503b311a37"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 03:43:36.371733 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:36.371706    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fea286df-f944-43ff-a298-33503b311a37-istio-token" (OuterVolumeSpecName: "istio-token") pod "fea286df-f944-43ff-a298-33503b311a37" (UID: "fea286df-f944-43ff-a298-33503b311a37"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:43:36.371842 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:36.371730    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fea286df-f944-43ff-a298-33503b311a37-local-certs" (OuterVolumeSpecName: "local-certs") pod "fea286df-f944-43ff-a298-33503b311a37" (UID: "fea286df-f944-43ff-a298-33503b311a37"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 03:43:36.371842 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:36.371781    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fea286df-f944-43ff-a298-33503b311a37-cacerts" (OuterVolumeSpecName: "cacerts") pod "fea286df-f944-43ff-a298-33503b311a37" (UID: "fea286df-f944-43ff-a298-33503b311a37"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 03:43:36.371842 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:36.371800    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fea286df-f944-43ff-a298-33503b311a37-kube-api-access-7bqlp" (OuterVolumeSpecName: "kube-api-access-7bqlp") pod "fea286df-f944-43ff-a298-33503b311a37" (UID: "fea286df-f944-43ff-a298-33503b311a37"). InnerVolumeSpecName "kube-api-access-7bqlp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:43:36.371842 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:36.371805    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fea286df-f944-43ff-a298-33503b311a37-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "fea286df-f944-43ff-a298-33503b311a37" (UID: "fea286df-f944-43ff-a298-33503b311a37"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 03:43:36.469525 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:36.469494    2576 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fea286df-f944-43ff-a298-33503b311a37-cacerts\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:43:36.469525 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:36.469523    2576 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fea286df-f944-43ff-a298-33503b311a37-istio-kubeconfig\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:43:36.469716 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:36.469537    2576 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fea286df-f944-43ff-a298-33503b311a37-istio-csr-ca-configmap\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:43:36.469716 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:36.469549    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7bqlp\" (UniqueName: \"kubernetes.io/projected/fea286df-f944-43ff-a298-33503b311a37-kube-api-access-7bqlp\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:43:36.469716 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:36.469561    2576 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fea286df-f944-43ff-a298-33503b311a37-local-certs\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:43:36.469716 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:36.469572    2576 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fea286df-f944-43ff-a298-33503b311a37-istio-csr-dns-cert\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:43:36.469716 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:36.469582    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fea286df-f944-43ff-a298-33503b311a37-istio-token\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:43:36.725047 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:36.725005    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 03:43:36.725157 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:36.725088    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 03:43:36.725157 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:36.725116    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 03:43:37.050267 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:37.050239    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5"
Jul 30 03:43:37.050719 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:37.050336    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5" event={"ID":"fea286df-f944-43ff-a298-33503b311a37","Type":"ContainerDied","Data":"78e46d07cf5cd2e1a8c34f61448f9cc1a99cb0fe79e884ea944be2d434739d40"}
Jul 30 03:43:37.050719 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:37.050383    2576 scope.go:117] "RemoveContainer" containerID="59a7d8f7c273219f637ce82b3c79a9f4406754bd693957ae5bec16df581b9c8b"
Jul 30 03:43:37.052120 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:37.052088    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl" event={"ID":"244fd030-1879-462b-8307-7c9de84f4666","Type":"ContainerStarted","Data":"29b91a54d39de197b9245aa14a5525278db362ca9f63807e2cab4127955d4ab9"}
Jul 30 03:43:37.053909 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:37.053884    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-d42fv" event={"ID":"39e8666b-7a30-4644-8979-dc793338ba98","Type":"ContainerStarted","Data":"91177d4713b63cf18e0265c19df40a2c93d2be7549012aaa9860ff8a8fc655da"}
Jul 30 03:43:37.054048 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:37.053987    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-d42fv"
Jul 30 03:43:37.096983 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:37.096909    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-d42fv" podStartSLOduration=3.838676065 podStartE2EDuration="6.096888504s" totalImagesPullingTime="2.258212439s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:43:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:43:37.095970245 +0000 UTC m=+522.714202480" watchObservedRunningTime="2026-07-30 03:43:37.096888504 +0000 UTC m=+522.715120732"
Jul 30 03:43:37.130602 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:37.130545    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl" podStartSLOduration=1.025848036 podStartE2EDuration="4.130524045s" totalImagesPullingTime="3.104676009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:43:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:43:37.129119098 +0000 UTC m=+522.747351338" watchObservedRunningTime="2026-07-30 03:43:37.130524045 +0000 UTC m=+522.748756288"
Jul 30 03:43:37.164381 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:37.164353    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5"]
Jul 30 03:43:37.173494 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:37.173467    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wgks5"]
Jul 30 03:43:37.467471 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:37.467373    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl"
Jul 30 03:43:37.472006 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:37.471981    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl"
Jul 30 03:43:38.059631 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:38.059608    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl"
Jul 30 03:43:38.060463 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:38.060446    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl"
Jul 30 03:43:39.050061 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:43:39.050027    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fea286df-f944-43ff-a298-33503b311a37" path="/var/lib/kubelet/pods/fea286df-f944-43ff-a298-33503b311a37/volumes"
Jul 30 03:44:06.589182 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:06.589151    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:44:06.691675 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:06.691643    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:44:08.064613 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:08.064581    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-d42fv"
Jul 30 03:44:22.288507 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:22.288473    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:44:22.385569 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:22.385542    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:44:22.406104 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:22.406076    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-d6shr"]
Jul 30 03:44:22.406498 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:22.406483    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fea286df-f944-43ff-a298-33503b311a37" containerName="discovery"
Jul 30 03:44:22.406558 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:22.406502    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="fea286df-f944-43ff-a298-33503b311a37" containerName="discovery"
Jul 30 03:44:22.406558 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:22.406553    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="fea286df-f944-43ff-a298-33503b311a37" containerName="discovery"
Jul 30 03:44:22.408254 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:22.408239    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-d6shr"
Jul 30 03:44:22.411248 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:22.411229    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-ghxvb\""
Jul 30 03:44:22.411361 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:22.411252    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 30 03:44:22.412261 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:22.412237    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 30 03:44:22.419971 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:22.419948    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-d6shr"]
Jul 30 03:44:22.428764 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:22.428743    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95dmq\" (UniqueName: \"kubernetes.io/projected/21d3835b-1fc6-4068-b93f-4b2686fec4c5-kube-api-access-95dmq\") pod \"kuadrant-operator-catalog-d6shr\" (UID: \"21d3835b-1fc6-4068-b93f-4b2686fec4c5\") " pod="kuadrant-system/kuadrant-operator-catalog-d6shr"
Jul 30 03:44:22.486039 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:22.486018    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:44:22.529883 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:22.529857    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95dmq\" (UniqueName: \"kubernetes.io/projected/21d3835b-1fc6-4068-b93f-4b2686fec4c5-kube-api-access-95dmq\") pod \"kuadrant-operator-catalog-d6shr\" (UID: \"21d3835b-1fc6-4068-b93f-4b2686fec4c5\") " pod="kuadrant-system/kuadrant-operator-catalog-d6shr"
Jul 30 03:44:22.538046 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:22.538021    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95dmq\" (UniqueName: \"kubernetes.io/projected/21d3835b-1fc6-4068-b93f-4b2686fec4c5-kube-api-access-95dmq\") pod \"kuadrant-operator-catalog-d6shr\" (UID: \"21d3835b-1fc6-4068-b93f-4b2686fec4c5\") " pod="kuadrant-system/kuadrant-operator-catalog-d6shr"
Jul 30 03:44:22.718786 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:22.718695    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-d6shr"
Jul 30 03:44:22.771086 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:22.771061    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-d6shr"]
Jul 30 03:44:22.842433 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:22.842409    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-d6shr"]
Jul 30 03:44:22.844227 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:44:22.844204    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21d3835b_1fc6_4068_b93f_4b2686fec4c5.slice/crio-e58646dc76bcb213d4e63b37c4db9b0b2f957797d3908799dd376ef03abec996 WatchSource:0}: Error finding container e58646dc76bcb213d4e63b37c4db9b0b2f957797d3908799dd376ef03abec996: Status 404 returned error can't find the container with id e58646dc76bcb213d4e63b37c4db9b0b2f957797d3908799dd376ef03abec996
Jul 30 03:44:22.980735 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:22.980660    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-txf8l"]
Jul 30 03:44:22.983317 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:22.983303    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-txf8l"
Jul 30 03:44:22.991823 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:22.991799    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-txf8l"]
Jul 30 03:44:23.033868 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:23.033843    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cq6gj\" (UniqueName: \"kubernetes.io/projected/9d1726a3-a5af-49f0-b216-79753b6dc67b-kube-api-access-cq6gj\") pod \"kuadrant-operator-catalog-txf8l\" (UID: \"9d1726a3-a5af-49f0-b216-79753b6dc67b\") " pod="kuadrant-system/kuadrant-operator-catalog-txf8l"
Jul 30 03:44:23.134879 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:23.134850    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cq6gj\" (UniqueName: \"kubernetes.io/projected/9d1726a3-a5af-49f0-b216-79753b6dc67b-kube-api-access-cq6gj\") pod \"kuadrant-operator-catalog-txf8l\" (UID: \"9d1726a3-a5af-49f0-b216-79753b6dc67b\") " pod="kuadrant-system/kuadrant-operator-catalog-txf8l"
Jul 30 03:44:23.144418 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:23.144387    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cq6gj\" (UniqueName: \"kubernetes.io/projected/9d1726a3-a5af-49f0-b216-79753b6dc67b-kube-api-access-cq6gj\") pod \"kuadrant-operator-catalog-txf8l\" (UID: \"9d1726a3-a5af-49f0-b216-79753b6dc67b\") " pod="kuadrant-system/kuadrant-operator-catalog-txf8l"
Jul 30 03:44:23.206690 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:23.206640    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:44:23.248380 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:23.248304    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-d6shr" event={"ID":"21d3835b-1fc6-4068-b93f-4b2686fec4c5","Type":"ContainerStarted","Data":"e58646dc76bcb213d4e63b37c4db9b0b2f957797d3908799dd376ef03abec996"}
Jul 30 03:44:23.292577 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:23.292550    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-txf8l"
Jul 30 03:44:23.447519 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:23.447444    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-txf8l"]
Jul 30 03:44:23.464155 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:44:23.464130    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d1726a3_a5af_49f0_b216_79753b6dc67b.slice/crio-a5c404c372b3760880dd55c516d78d23b787d9b6cbcdd60883ec89f381ac030e WatchSource:0}: Error finding container a5c404c372b3760880dd55c516d78d23b787d9b6cbcdd60883ec89f381ac030e: Status 404 returned error can't find the container with id a5c404c372b3760880dd55c516d78d23b787d9b6cbcdd60883ec89f381ac030e
Jul 30 03:44:23.472402 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:23.472382    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:44:24.255827 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:24.255793    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-txf8l" event={"ID":"9d1726a3-a5af-49f0-b216-79753b6dc67b","Type":"ContainerStarted","Data":"a5c404c372b3760880dd55c516d78d23b787d9b6cbcdd60883ec89f381ac030e"}
Jul 30 03:44:25.261124 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:25.261021    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-d6shr" event={"ID":"21d3835b-1fc6-4068-b93f-4b2686fec4c5","Type":"ContainerStarted","Data":"4920584996f6164b69208c211a9f54587ed784f3b6a3495dcee7a520654017df"}
Jul 30 03:44:25.261124 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:25.261106    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-d6shr" podUID="21d3835b-1fc6-4068-b93f-4b2686fec4c5" containerName="registry-server" containerID="cri-o://4920584996f6164b69208c211a9f54587ed784f3b6a3495dcee7a520654017df" gracePeriod=2
Jul 30 03:44:25.262576 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:25.262548    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-txf8l" event={"ID":"9d1726a3-a5af-49f0-b216-79753b6dc67b","Type":"ContainerStarted","Data":"52bc871d0cfd8849fbad6423c2fb8b463c8c8ed6f92dd1f9c04fb5d4f5d881de"}
Jul 30 03:44:25.278321 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:25.278282    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-d6shr" podStartSLOduration=1.317186247 podStartE2EDuration="3.278268167s" totalImagesPullingTime="1.96108192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:44:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:44:25.277048073 +0000 UTC m=+570.895280314" watchObservedRunningTime="2026-07-30 03:44:25.278268167 +0000 UTC m=+570.896500398"
Jul 30 03:44:25.292770 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:25.292729    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-txf8l" podStartSLOduration=1.948804697 podStartE2EDuration="3.292715104s" totalImagesPullingTime="1.343910407s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:44:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:44:25.290946016 +0000 UTC m=+570.909178247" watchObservedRunningTime="2026-07-30 03:44:25.292715104 +0000 UTC m=+570.910947344"
Jul 30 03:44:25.497468 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:25.497439    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-d6shr"
Jul 30 03:44:25.558355 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:25.558324    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-95dmq\" (UniqueName: \"kubernetes.io/projected/21d3835b-1fc6-4068-b93f-4b2686fec4c5-kube-api-access-95dmq\") pod \"21d3835b-1fc6-4068-b93f-4b2686fec4c5\" (UID: \"21d3835b-1fc6-4068-b93f-4b2686fec4c5\") "
Jul 30 03:44:25.564010 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:25.561294    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21d3835b-1fc6-4068-b93f-4b2686fec4c5-kube-api-access-95dmq" (OuterVolumeSpecName: "kube-api-access-95dmq") pod "21d3835b-1fc6-4068-b93f-4b2686fec4c5" (UID: "21d3835b-1fc6-4068-b93f-4b2686fec4c5"). InnerVolumeSpecName "kube-api-access-95dmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:44:25.660737 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:25.660700    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-95dmq\" (UniqueName: \"kubernetes.io/projected/21d3835b-1fc6-4068-b93f-4b2686fec4c5-kube-api-access-95dmq\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:44:26.268200 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:26.268115    2576 generic.go:356] "Generic (PLEG): container finished" podID="21d3835b-1fc6-4068-b93f-4b2686fec4c5" containerID="4920584996f6164b69208c211a9f54587ed784f3b6a3495dcee7a520654017df" exitCode=0
Jul 30 03:44:26.268200 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:26.268167    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-d6shr"
Jul 30 03:44:26.268598 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:26.268205    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-d6shr" event={"ID":"21d3835b-1fc6-4068-b93f-4b2686fec4c5","Type":"ContainerDied","Data":"4920584996f6164b69208c211a9f54587ed784f3b6a3495dcee7a520654017df"}
Jul 30 03:44:26.268598 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:26.268270    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-d6shr" event={"ID":"21d3835b-1fc6-4068-b93f-4b2686fec4c5","Type":"ContainerDied","Data":"e58646dc76bcb213d4e63b37c4db9b0b2f957797d3908799dd376ef03abec996"}
Jul 30 03:44:26.268598 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:26.268298    2576 scope.go:117] "RemoveContainer" containerID="4920584996f6164b69208c211a9f54587ed784f3b6a3495dcee7a520654017df"
Jul 30 03:44:26.276943 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:26.276915    2576 scope.go:117] "RemoveContainer" containerID="4920584996f6164b69208c211a9f54587ed784f3b6a3495dcee7a520654017df"
Jul 30 03:44:26.277209 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:44:26.277192    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4920584996f6164b69208c211a9f54587ed784f3b6a3495dcee7a520654017df\": container with ID starting with 4920584996f6164b69208c211a9f54587ed784f3b6a3495dcee7a520654017df not found: ID does not exist" containerID="4920584996f6164b69208c211a9f54587ed784f3b6a3495dcee7a520654017df"
Jul 30 03:44:26.277288 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:26.277215    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4920584996f6164b69208c211a9f54587ed784f3b6a3495dcee7a520654017df"} err="failed to get container status \"4920584996f6164b69208c211a9f54587ed784f3b6a3495dcee7a520654017df\": rpc error: code = NotFound desc = could not find container \"4920584996f6164b69208c211a9f54587ed784f3b6a3495dcee7a520654017df\": container with ID starting with 4920584996f6164b69208c211a9f54587ed784f3b6a3495dcee7a520654017df not found: ID does not exist"
Jul 30 03:44:26.288648 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:26.288614    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-d6shr"]
Jul 30 03:44:26.290791 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:26.290772    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-d6shr"]
Jul 30 03:44:27.049878 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:27.049844    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21d3835b-1fc6-4068-b93f-4b2686fec4c5" path="/var/lib/kubelet/pods/21d3835b-1fc6-4068-b93f-4b2686fec4c5/volumes"
Jul 30 03:44:33.292952 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:33.292906    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-txf8l"
Jul 30 03:44:33.293344 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:33.292975    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-txf8l"
Jul 30 03:44:33.314660 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:33.314638    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-txf8l"
Jul 30 03:44:33.334750 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:33.334728    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-txf8l"
Jul 30 03:44:34.927020 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:34.926989    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r4s7n"]
Jul 30 03:44:34.927395 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:34.927276    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21d3835b-1fc6-4068-b93f-4b2686fec4c5" containerName="registry-server"
Jul 30 03:44:34.927395 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:34.927287    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="21d3835b-1fc6-4068-b93f-4b2686fec4c5" containerName="registry-server"
Jul 30 03:44:34.927395 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:34.927336    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="21d3835b-1fc6-4068-b93f-4b2686fec4c5" containerName="registry-server"
Jul 30 03:44:34.930853 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:34.930836    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r4s7n"
Jul 30 03:44:34.933444 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:34.933418    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kqcjr\""
Jul 30 03:44:34.938818 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:34.938794    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r4s7n"]
Jul 30 03:44:35.035883 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:35.035856    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hbmm\" (UniqueName: \"kubernetes.io/projected/9ada5a4e-7a8c-413b-a838-2d98660edf99-kube-api-access-9hbmm\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r4s7n\" (UID: \"9ada5a4e-7a8c-413b-a838-2d98660edf99\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r4s7n"
Jul 30 03:44:35.036065 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:35.035906    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9ada5a4e-7a8c-413b-a838-2d98660edf99-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r4s7n\" (UID: \"9ada5a4e-7a8c-413b-a838-2d98660edf99\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r4s7n"
Jul 30 03:44:35.036065 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:35.035966    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9ada5a4e-7a8c-413b-a838-2d98660edf99-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r4s7n\" (UID: \"9ada5a4e-7a8c-413b-a838-2d98660edf99\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r4s7n"
Jul 30 03:44:35.137067 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:35.137034    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hbmm\" (UniqueName: \"kubernetes.io/projected/9ada5a4e-7a8c-413b-a838-2d98660edf99-kube-api-access-9hbmm\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r4s7n\" (UID: \"9ada5a4e-7a8c-413b-a838-2d98660edf99\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r4s7n"
Jul 30 03:44:35.137238 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:35.137091    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9ada5a4e-7a8c-413b-a838-2d98660edf99-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r4s7n\" (UID: \"9ada5a4e-7a8c-413b-a838-2d98660edf99\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r4s7n"
Jul 30 03:44:35.137238 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:35.137196    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9ada5a4e-7a8c-413b-a838-2d98660edf99-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r4s7n\" (UID: \"9ada5a4e-7a8c-413b-a838-2d98660edf99\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r4s7n"
Jul 30 03:44:35.137392 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:35.137378    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9ada5a4e-7a8c-413b-a838-2d98660edf99-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r4s7n\" (UID: \"9ada5a4e-7a8c-413b-a838-2d98660edf99\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r4s7n"
Jul 30 03:44:35.137573 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:35.137554    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9ada5a4e-7a8c-413b-a838-2d98660edf99-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r4s7n\" (UID: \"9ada5a4e-7a8c-413b-a838-2d98660edf99\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r4s7n"
Jul 30 03:44:35.145988 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:35.145963    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hbmm\" (UniqueName: \"kubernetes.io/projected/9ada5a4e-7a8c-413b-a838-2d98660edf99-kube-api-access-9hbmm\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r4s7n\" (UID: \"9ada5a4e-7a8c-413b-a838-2d98660edf99\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r4s7n"
Jul 30 03:44:35.240785 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:35.240750    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r4s7n"
Jul 30 03:44:35.329391 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:35.329366    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cst2q"]
Jul 30 03:44:35.334120 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:35.334099    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cst2q"
Jul 30 03:44:35.341600 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:35.341573    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cst2q"]
Jul 30 03:44:35.365188 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:35.365073    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r4s7n"]
Jul 30 03:44:35.367444 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:44:35.367420    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ada5a4e_7a8c_413b_a838_2d98660edf99.slice/crio-4cbcce96c23654201d43239fd4f9b4442adc236d594e9f1de42127ad9017a842 WatchSource:0}: Error finding container 4cbcce96c23654201d43239fd4f9b4442adc236d594e9f1de42127ad9017a842: Status 404 returned error can't find the container with id 4cbcce96c23654201d43239fd4f9b4442adc236d594e9f1de42127ad9017a842
Jul 30 03:44:35.440231 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:35.440197    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0ceb213f-4e8b-4567-865d-5f370ebc38ea-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cst2q\" (UID: \"0ceb213f-4e8b-4567-865d-5f370ebc38ea\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cst2q"
Jul 30 03:44:35.440407 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:35.440243    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgfvb\" (UniqueName: \"kubernetes.io/projected/0ceb213f-4e8b-4567-865d-5f370ebc38ea-kube-api-access-wgfvb\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cst2q\" (UID: \"0ceb213f-4e8b-4567-865d-5f370ebc38ea\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cst2q"
Jul 30 03:44:35.440407 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:35.440301    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0ceb213f-4e8b-4567-865d-5f370ebc38ea-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cst2q\" (UID: \"0ceb213f-4e8b-4567-865d-5f370ebc38ea\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cst2q"
Jul 30 03:44:35.541755 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:35.541664    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0ceb213f-4e8b-4567-865d-5f370ebc38ea-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cst2q\" (UID: \"0ceb213f-4e8b-4567-865d-5f370ebc38ea\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cst2q"
Jul 30 03:44:35.541755 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:35.541738    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0ceb213f-4e8b-4567-865d-5f370ebc38ea-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cst2q\" (UID: \"0ceb213f-4e8b-4567-865d-5f370ebc38ea\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cst2q"
Jul 30 03:44:35.541999 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:35.541759    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgfvb\" (UniqueName: \"kubernetes.io/projected/0ceb213f-4e8b-4567-865d-5f370ebc38ea-kube-api-access-wgfvb\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cst2q\" (UID: \"0ceb213f-4e8b-4567-865d-5f370ebc38ea\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cst2q"
Jul 30 03:44:35.542076 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:35.542055    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0ceb213f-4e8b-4567-865d-5f370ebc38ea-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cst2q\" (UID: \"0ceb213f-4e8b-4567-865d-5f370ebc38ea\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cst2q"
Jul 30 03:44:35.542153 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:35.542134    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0ceb213f-4e8b-4567-865d-5f370ebc38ea-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cst2q\" (UID: \"0ceb213f-4e8b-4567-865d-5f370ebc38ea\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cst2q"
Jul 30 03:44:35.550472 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:35.550443    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgfvb\" (UniqueName: \"kubernetes.io/projected/0ceb213f-4e8b-4567-865d-5f370ebc38ea-kube-api-access-wgfvb\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cst2q\" (UID: \"0ceb213f-4e8b-4567-865d-5f370ebc38ea\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cst2q"
Jul 30 03:44:35.645593 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:35.645556    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cst2q"
Jul 30 03:44:35.730240 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:35.730211    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrdnb"]
Jul 30 03:44:35.734632 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:35.734613    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrdnb"
Jul 30 03:44:35.740719 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:35.740698    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrdnb"]
Jul 30 03:44:35.774718 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:35.774694    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cst2q"]
Jul 30 03:44:35.776217 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:44:35.776193    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ceb213f_4e8b_4567_865d_5f370ebc38ea.slice/crio-c2bb66bf1a8d6b1ff642cc9f1ba616de612ea925ee962102b389cb4329e3c140 WatchSource:0}: Error finding container c2bb66bf1a8d6b1ff642cc9f1ba616de612ea925ee962102b389cb4329e3c140: Status 404 returned error can't find the container with id c2bb66bf1a8d6b1ff642cc9f1ba616de612ea925ee962102b389cb4329e3c140
Jul 30 03:44:35.844027 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:35.843995    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f32f58e2-cb21-499f-b72c-4ce90eed5bba-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrdnb\" (UID: \"f32f58e2-cb21-499f-b72c-4ce90eed5bba\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrdnb"
Jul 30 03:44:35.844198 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:35.844068    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gb4r\" (UniqueName: \"kubernetes.io/projected/f32f58e2-cb21-499f-b72c-4ce90eed5bba-kube-api-access-9gb4r\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrdnb\" (UID: \"f32f58e2-cb21-499f-b72c-4ce90eed5bba\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrdnb"
Jul 30 03:44:35.844198 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:35.844103    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f32f58e2-cb21-499f-b72c-4ce90eed5bba-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrdnb\" (UID: \"f32f58e2-cb21-499f-b72c-4ce90eed5bba\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrdnb"
Jul 30 03:44:35.945325 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:35.945292    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gb4r\" (UniqueName: \"kubernetes.io/projected/f32f58e2-cb21-499f-b72c-4ce90eed5bba-kube-api-access-9gb4r\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrdnb\" (UID: \"f32f58e2-cb21-499f-b72c-4ce90eed5bba\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrdnb"
Jul 30 03:44:35.945661 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:35.945333    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f32f58e2-cb21-499f-b72c-4ce90eed5bba-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrdnb\" (UID: \"f32f58e2-cb21-499f-b72c-4ce90eed5bba\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrdnb"
Jul 30 03:44:35.945661 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:35.945377    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f32f58e2-cb21-499f-b72c-4ce90eed5bba-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrdnb\" (UID: \"f32f58e2-cb21-499f-b72c-4ce90eed5bba\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrdnb"
Jul 30 03:44:35.945749 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:35.945731    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f32f58e2-cb21-499f-b72c-4ce90eed5bba-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrdnb\" (UID: \"f32f58e2-cb21-499f-b72c-4ce90eed5bba\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrdnb"
Jul 30 03:44:35.945795 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:35.945780    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f32f58e2-cb21-499f-b72c-4ce90eed5bba-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrdnb\" (UID: \"f32f58e2-cb21-499f-b72c-4ce90eed5bba\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrdnb"
Jul 30 03:44:35.954848 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:35.954817    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gb4r\" (UniqueName: \"kubernetes.io/projected/f32f58e2-cb21-499f-b72c-4ce90eed5bba-kube-api-access-9gb4r\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrdnb\" (UID: \"f32f58e2-cb21-499f-b72c-4ce90eed5bba\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrdnb"
Jul 30 03:44:36.045386 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:36.045340    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrdnb"
Jul 30 03:44:36.175806 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:36.175781    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrdnb"]
Jul 30 03:44:36.177672 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:44:36.177647    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf32f58e2_cb21_499f_b72c_4ce90eed5bba.slice/crio-4686c337ad41197ea5400e90ec8cae5d4b9335040a91bbfdeb537bfdbd91a969 WatchSource:0}: Error finding container 4686c337ad41197ea5400e90ec8cae5d4b9335040a91bbfdeb537bfdbd91a969: Status 404 returned error can't find the container with id 4686c337ad41197ea5400e90ec8cae5d4b9335040a91bbfdeb537bfdbd91a969
Jul 30 03:44:36.315548 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:36.315481    2576 generic.go:356] "Generic (PLEG): container finished" podID="0ceb213f-4e8b-4567-865d-5f370ebc38ea" containerID="330c803f72d1cb53dda0f3c3eb143a3c35c33a5502fcf334f08ec4c112cadab6" exitCode=0
Jul 30 03:44:36.315681 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:36.315571    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cst2q" event={"ID":"0ceb213f-4e8b-4567-865d-5f370ebc38ea","Type":"ContainerDied","Data":"330c803f72d1cb53dda0f3c3eb143a3c35c33a5502fcf334f08ec4c112cadab6"}
Jul 30 03:44:36.315681 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:36.315618    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cst2q" event={"ID":"0ceb213f-4e8b-4567-865d-5f370ebc38ea","Type":"ContainerStarted","Data":"c2bb66bf1a8d6b1ff642cc9f1ba616de612ea925ee962102b389cb4329e3c140"}
Jul 30 03:44:36.317214 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:36.317191    2576 generic.go:356] "Generic (PLEG): container finished" podID="9ada5a4e-7a8c-413b-a838-2d98660edf99" containerID="f3030e36b82150eb7500ed826520ab47a284702173fc7e1b0d087343913d0294" exitCode=0
Jul 30 03:44:36.317321 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:36.317284    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r4s7n" event={"ID":"9ada5a4e-7a8c-413b-a838-2d98660edf99","Type":"ContainerDied","Data":"f3030e36b82150eb7500ed826520ab47a284702173fc7e1b0d087343913d0294"}
Jul 30 03:44:36.317380 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:36.317334    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r4s7n" event={"ID":"9ada5a4e-7a8c-413b-a838-2d98660edf99","Type":"ContainerStarted","Data":"4cbcce96c23654201d43239fd4f9b4442adc236d594e9f1de42127ad9017a842"}
Jul 30 03:44:36.318972 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:36.318951    2576 generic.go:356] "Generic (PLEG): container finished" podID="f32f58e2-cb21-499f-b72c-4ce90eed5bba" containerID="f0980e459bf944e71d66ee9b46425f4de5de1af2c00a6329b791d503a7a307bf" exitCode=0
Jul 30 03:44:36.319073 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:36.319020    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrdnb" event={"ID":"f32f58e2-cb21-499f-b72c-4ce90eed5bba","Type":"ContainerDied","Data":"f0980e459bf944e71d66ee9b46425f4de5de1af2c00a6329b791d503a7a307bf"}
Jul 30 03:44:36.319073 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:36.319055    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrdnb" event={"ID":"f32f58e2-cb21-499f-b72c-4ce90eed5bba","Type":"ContainerStarted","Data":"4686c337ad41197ea5400e90ec8cae5d4b9335040a91bbfdeb537bfdbd91a969"}
Jul 30 03:44:36.340046 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:36.340023    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqfb"]
Jul 30 03:44:36.343308 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:36.343294    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqfb"
Jul 30 03:44:36.352401 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:36.352383    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqfb"]
Jul 30 03:44:36.449985 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:36.449945    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0ceed716-436d-4616-a637-524eed35ff96-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqfb\" (UID: \"0ceed716-436d-4616-a637-524eed35ff96\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqfb"
Jul 30 03:44:36.449985 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:36.449988    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dg82s\" (UniqueName: \"kubernetes.io/projected/0ceed716-436d-4616-a637-524eed35ff96-kube-api-access-dg82s\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqfb\" (UID: \"0ceed716-436d-4616-a637-524eed35ff96\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqfb"
Jul 30 03:44:36.450234 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:36.450039    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0ceed716-436d-4616-a637-524eed35ff96-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqfb\" (UID: \"0ceed716-436d-4616-a637-524eed35ff96\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqfb"
Jul 30 03:44:36.550985 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:36.550947    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0ceed716-436d-4616-a637-524eed35ff96-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqfb\" (UID: \"0ceed716-436d-4616-a637-524eed35ff96\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqfb"
Jul 30 03:44:36.550985 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:36.550988    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dg82s\" (UniqueName: \"kubernetes.io/projected/0ceed716-436d-4616-a637-524eed35ff96-kube-api-access-dg82s\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqfb\" (UID: \"0ceed716-436d-4616-a637-524eed35ff96\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqfb"
Jul 30 03:44:36.551188 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:36.551023    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0ceed716-436d-4616-a637-524eed35ff96-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqfb\" (UID: \"0ceed716-436d-4616-a637-524eed35ff96\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqfb"
Jul 30 03:44:36.551317 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:36.551298    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0ceed716-436d-4616-a637-524eed35ff96-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqfb\" (UID: \"0ceed716-436d-4616-a637-524eed35ff96\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqfb"
Jul 30 03:44:36.551404 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:36.551384    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0ceed716-436d-4616-a637-524eed35ff96-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqfb\" (UID: \"0ceed716-436d-4616-a637-524eed35ff96\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqfb"
Jul 30 03:44:36.560158 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:36.560133    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dg82s\" (UniqueName: \"kubernetes.io/projected/0ceed716-436d-4616-a637-524eed35ff96-kube-api-access-dg82s\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqfb\" (UID: \"0ceed716-436d-4616-a637-524eed35ff96\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqfb"
Jul 30 03:44:36.651916 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:36.651835    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqfb"
Jul 30 03:44:36.779079 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:36.779052    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqfb"]
Jul 30 03:44:36.780849 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:44:36.780826    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ceed716_436d_4616_a637_524eed35ff96.slice/crio-d43f7dfa11bda7dfb106187f34ecbce6ba3936154363cfdd33dd410cc0cd2bc8 WatchSource:0}: Error finding container d43f7dfa11bda7dfb106187f34ecbce6ba3936154363cfdd33dd410cc0cd2bc8: Status 404 returned error can't find the container with id d43f7dfa11bda7dfb106187f34ecbce6ba3936154363cfdd33dd410cc0cd2bc8
Jul 30 03:44:37.325565 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:37.325530    2576 generic.go:356] "Generic (PLEG): container finished" podID="f32f58e2-cb21-499f-b72c-4ce90eed5bba" containerID="c6332e6506db24d7265b15e23fb8a8c88c1892fc9924aca54f051101e72fb67b" exitCode=0
Jul 30 03:44:37.325943 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:37.325609    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrdnb" event={"ID":"f32f58e2-cb21-499f-b72c-4ce90eed5bba","Type":"ContainerDied","Data":"c6332e6506db24d7265b15e23fb8a8c88c1892fc9924aca54f051101e72fb67b"}
Jul 30 03:44:37.327395 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:37.327368    2576 generic.go:356] "Generic (PLEG): container finished" podID="0ceed716-436d-4616-a637-524eed35ff96" containerID="02c0631e0e1e4c3d68f35bc5e082e2f6a81d4a35d1603d1a150398cd91161151" exitCode=0
Jul 30 03:44:37.327494 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:37.327426    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqfb" event={"ID":"0ceed716-436d-4616-a637-524eed35ff96","Type":"ContainerDied","Data":"02c0631e0e1e4c3d68f35bc5e082e2f6a81d4a35d1603d1a150398cd91161151"}
Jul 30 03:44:37.327494 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:37.327464    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqfb" event={"ID":"0ceed716-436d-4616-a637-524eed35ff96","Type":"ContainerStarted","Data":"d43f7dfa11bda7dfb106187f34ecbce6ba3936154363cfdd33dd410cc0cd2bc8"}
Jul 30 03:44:38.335362 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:38.335330    2576 generic.go:356] "Generic (PLEG): container finished" podID="f32f58e2-cb21-499f-b72c-4ce90eed5bba" containerID="687425839bef60106fb074fa8dfc65fafd2bd58ca3202ef489e181154937d324" exitCode=0
Jul 30 03:44:38.335789 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:38.335424    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrdnb" event={"ID":"f32f58e2-cb21-499f-b72c-4ce90eed5bba","Type":"ContainerDied","Data":"687425839bef60106fb074fa8dfc65fafd2bd58ca3202ef489e181154937d324"}
Jul 30 03:44:38.336996 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:38.336968    2576 generic.go:356] "Generic (PLEG): container finished" podID="0ceed716-436d-4616-a637-524eed35ff96" containerID="2447ce72be9a4b1eb854f8908044d1f3d6677bc7c68a66a3cf1603b42427d7f6" exitCode=0
Jul 30 03:44:38.337122 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:38.337046    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqfb" event={"ID":"0ceed716-436d-4616-a637-524eed35ff96","Type":"ContainerDied","Data":"2447ce72be9a4b1eb854f8908044d1f3d6677bc7c68a66a3cf1603b42427d7f6"}
Jul 30 03:44:38.338801 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:38.338782    2576 generic.go:356] "Generic (PLEG): container finished" podID="0ceb213f-4e8b-4567-865d-5f370ebc38ea" containerID="3e531c07045c4c326b27edea1516d554887b74c68f1d9fb5e64b412a7a54672a" exitCode=0
Jul 30 03:44:38.338881 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:38.338858    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cst2q" event={"ID":"0ceb213f-4e8b-4567-865d-5f370ebc38ea","Type":"ContainerDied","Data":"3e531c07045c4c326b27edea1516d554887b74c68f1d9fb5e64b412a7a54672a"}
Jul 30 03:44:38.340803 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:38.340780    2576 generic.go:356] "Generic (PLEG): container finished" podID="9ada5a4e-7a8c-413b-a838-2d98660edf99" containerID="806369a49348af35bd4664bf5f598be8bad63cef9320f59f6b612ffaf34404cb" exitCode=0
Jul 30 03:44:38.340893 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:38.340815    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r4s7n" event={"ID":"9ada5a4e-7a8c-413b-a838-2d98660edf99","Type":"ContainerDied","Data":"806369a49348af35bd4664bf5f598be8bad63cef9320f59f6b612ffaf34404cb"}
Jul 30 03:44:39.347579 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:39.347544    2576 generic.go:356] "Generic (PLEG): container finished" podID="0ceed716-436d-4616-a637-524eed35ff96" containerID="71fc46293a6d8c2e1cb0c2451205f2993f139c2aee993358a043020583eccd0f" exitCode=0
Jul 30 03:44:39.347997 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:39.347631    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqfb" event={"ID":"0ceed716-436d-4616-a637-524eed35ff96","Type":"ContainerDied","Data":"71fc46293a6d8c2e1cb0c2451205f2993f139c2aee993358a043020583eccd0f"}
Jul 30 03:44:39.349572 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:39.349550    2576 generic.go:356] "Generic (PLEG): container finished" podID="0ceb213f-4e8b-4567-865d-5f370ebc38ea" containerID="e6c9f0ed2f399eaff8d0df95ee4f9be3c9f98dd4ff909665e684833092fedf41" exitCode=0
Jul 30 03:44:39.349688 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:39.349633    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cst2q" event={"ID":"0ceb213f-4e8b-4567-865d-5f370ebc38ea","Type":"ContainerDied","Data":"e6c9f0ed2f399eaff8d0df95ee4f9be3c9f98dd4ff909665e684833092fedf41"}
Jul 30 03:44:39.351600 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:39.351581    2576 generic.go:356] "Generic (PLEG): container finished" podID="9ada5a4e-7a8c-413b-a838-2d98660edf99" containerID="926730c5e45aabedd52a5cfbbf4d7879089ebeed0d25e8f0f912bfd0200ee6c7" exitCode=0
Jul 30 03:44:39.351700 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:39.351641    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r4s7n" event={"ID":"9ada5a4e-7a8c-413b-a838-2d98660edf99","Type":"ContainerDied","Data":"926730c5e45aabedd52a5cfbbf4d7879089ebeed0d25e8f0f912bfd0200ee6c7"}
Jul 30 03:44:39.479666 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:39.479638    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrdnb"
Jul 30 03:44:39.578719 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:39.578678    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f32f58e2-cb21-499f-b72c-4ce90eed5bba-util\") pod \"f32f58e2-cb21-499f-b72c-4ce90eed5bba\" (UID: \"f32f58e2-cb21-499f-b72c-4ce90eed5bba\") "
Jul 30 03:44:39.578886 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:39.578764    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f32f58e2-cb21-499f-b72c-4ce90eed5bba-bundle\") pod \"f32f58e2-cb21-499f-b72c-4ce90eed5bba\" (UID: \"f32f58e2-cb21-499f-b72c-4ce90eed5bba\") "
Jul 30 03:44:39.578886 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:39.578855    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9gb4r\" (UniqueName: \"kubernetes.io/projected/f32f58e2-cb21-499f-b72c-4ce90eed5bba-kube-api-access-9gb4r\") pod \"f32f58e2-cb21-499f-b72c-4ce90eed5bba\" (UID: \"f32f58e2-cb21-499f-b72c-4ce90eed5bba\") "
Jul 30 03:44:39.579255 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:39.579224    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f32f58e2-cb21-499f-b72c-4ce90eed5bba-bundle" (OuterVolumeSpecName: "bundle") pod "f32f58e2-cb21-499f-b72c-4ce90eed5bba" (UID: "f32f58e2-cb21-499f-b72c-4ce90eed5bba"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 03:44:39.580888 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:39.580861    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f32f58e2-cb21-499f-b72c-4ce90eed5bba-kube-api-access-9gb4r" (OuterVolumeSpecName: "kube-api-access-9gb4r") pod "f32f58e2-cb21-499f-b72c-4ce90eed5bba" (UID: "f32f58e2-cb21-499f-b72c-4ce90eed5bba"). InnerVolumeSpecName "kube-api-access-9gb4r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:44:39.583826 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:39.583803    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f32f58e2-cb21-499f-b72c-4ce90eed5bba-util" (OuterVolumeSpecName: "util") pod "f32f58e2-cb21-499f-b72c-4ce90eed5bba" (UID: "f32f58e2-cb21-499f-b72c-4ce90eed5bba"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 03:44:39.679989 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:39.679885    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9gb4r\" (UniqueName: \"kubernetes.io/projected/f32f58e2-cb21-499f-b72c-4ce90eed5bba-kube-api-access-9gb4r\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:44:39.679989 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:39.679918    2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f32f58e2-cb21-499f-b72c-4ce90eed5bba-util\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:44:39.679989 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:39.679953    2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f32f58e2-cb21-499f-b72c-4ce90eed5bba-bundle\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:44:40.358491 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:40.358453    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrdnb" event={"ID":"f32f58e2-cb21-499f-b72c-4ce90eed5bba","Type":"ContainerDied","Data":"4686c337ad41197ea5400e90ec8cae5d4b9335040a91bbfdeb537bfdbd91a969"}
Jul 30 03:44:40.358895 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:40.358508    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrdnb"
Jul 30 03:44:40.358895 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:40.358511    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4686c337ad41197ea5400e90ec8cae5d4b9335040a91bbfdeb537bfdbd91a969"
Jul 30 03:44:40.490940 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:40.490908    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cst2q"
Jul 30 03:44:40.532572 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:40.532545    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqfb"
Jul 30 03:44:40.535590 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:40.535563    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r4s7n"
Jul 30 03:44:40.588149 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:40.588122    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0ceb213f-4e8b-4567-865d-5f370ebc38ea-bundle\") pod \"0ceb213f-4e8b-4567-865d-5f370ebc38ea\" (UID: \"0ceb213f-4e8b-4567-865d-5f370ebc38ea\") "
Jul 30 03:44:40.588309 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:40.588167    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9hbmm\" (UniqueName: \"kubernetes.io/projected/9ada5a4e-7a8c-413b-a838-2d98660edf99-kube-api-access-9hbmm\") pod \"9ada5a4e-7a8c-413b-a838-2d98660edf99\" (UID: \"9ada5a4e-7a8c-413b-a838-2d98660edf99\") "
Jul 30 03:44:40.588309 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:40.588184    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dg82s\" (UniqueName: \"kubernetes.io/projected/0ceed716-436d-4616-a637-524eed35ff96-kube-api-access-dg82s\") pod \"0ceed716-436d-4616-a637-524eed35ff96\" (UID: \"0ceed716-436d-4616-a637-524eed35ff96\") "
Jul 30 03:44:40.588309 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:40.588209    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0ceb213f-4e8b-4567-865d-5f370ebc38ea-util\") pod \"0ceb213f-4e8b-4567-865d-5f370ebc38ea\" (UID: \"0ceb213f-4e8b-4567-865d-5f370ebc38ea\") "
Jul 30 03:44:40.588309 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:40.588249    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wgfvb\" (UniqueName: \"kubernetes.io/projected/0ceb213f-4e8b-4567-865d-5f370ebc38ea-kube-api-access-wgfvb\") pod \"0ceb213f-4e8b-4567-865d-5f370ebc38ea\" (UID: \"0ceb213f-4e8b-4567-865d-5f370ebc38ea\") "
Jul 30 03:44:40.588309 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:40.588276    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0ceed716-436d-4616-a637-524eed35ff96-bundle\") pod \"0ceed716-436d-4616-a637-524eed35ff96\" (UID: \"0ceed716-436d-4616-a637-524eed35ff96\") "
Jul 30 03:44:40.588309 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:40.588299    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9ada5a4e-7a8c-413b-a838-2d98660edf99-bundle\") pod \"9ada5a4e-7a8c-413b-a838-2d98660edf99\" (UID: \"9ada5a4e-7a8c-413b-a838-2d98660edf99\") "
Jul 30 03:44:40.588629 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:40.588324    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9ada5a4e-7a8c-413b-a838-2d98660edf99-util\") pod \"9ada5a4e-7a8c-413b-a838-2d98660edf99\" (UID: \"9ada5a4e-7a8c-413b-a838-2d98660edf99\") "
Jul 30 03:44:40.588629 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:40.588384    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0ceed716-436d-4616-a637-524eed35ff96-util\") pod \"0ceed716-436d-4616-a637-524eed35ff96\" (UID: \"0ceed716-436d-4616-a637-524eed35ff96\") "
Jul 30 03:44:40.588878 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:40.588851    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ceed716-436d-4616-a637-524eed35ff96-bundle" (OuterVolumeSpecName: "bundle") pod "0ceed716-436d-4616-a637-524eed35ff96" (UID: "0ceed716-436d-4616-a637-524eed35ff96"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 03:44:40.588993 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:40.588890    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ceb213f-4e8b-4567-865d-5f370ebc38ea-bundle" (OuterVolumeSpecName: "bundle") pod "0ceb213f-4e8b-4567-865d-5f370ebc38ea" (UID: "0ceb213f-4e8b-4567-865d-5f370ebc38ea"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 03:44:40.589123 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:40.589097    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ada5a4e-7a8c-413b-a838-2d98660edf99-bundle" (OuterVolumeSpecName: "bundle") pod "9ada5a4e-7a8c-413b-a838-2d98660edf99" (UID: "9ada5a4e-7a8c-413b-a838-2d98660edf99"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 03:44:40.590468 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:40.590444    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ada5a4e-7a8c-413b-a838-2d98660edf99-kube-api-access-9hbmm" (OuterVolumeSpecName: "kube-api-access-9hbmm") pod "9ada5a4e-7a8c-413b-a838-2d98660edf99" (UID: "9ada5a4e-7a8c-413b-a838-2d98660edf99"). InnerVolumeSpecName "kube-api-access-9hbmm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:44:40.590889 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:40.590872    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ceed716-436d-4616-a637-524eed35ff96-kube-api-access-dg82s" (OuterVolumeSpecName: "kube-api-access-dg82s") pod "0ceed716-436d-4616-a637-524eed35ff96" (UID: "0ceed716-436d-4616-a637-524eed35ff96"). InnerVolumeSpecName "kube-api-access-dg82s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:44:40.591113 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:40.591023    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ceb213f-4e8b-4567-865d-5f370ebc38ea-kube-api-access-wgfvb" (OuterVolumeSpecName: "kube-api-access-wgfvb") pod "0ceb213f-4e8b-4567-865d-5f370ebc38ea" (UID: "0ceb213f-4e8b-4567-865d-5f370ebc38ea"). InnerVolumeSpecName "kube-api-access-wgfvb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:44:40.597009 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:40.596986    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ceed716-436d-4616-a637-524eed35ff96-util" (OuterVolumeSpecName: "util") pod "0ceed716-436d-4616-a637-524eed35ff96" (UID: "0ceed716-436d-4616-a637-524eed35ff96"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 03:44:40.597391 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:40.597371    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ceb213f-4e8b-4567-865d-5f370ebc38ea-util" (OuterVolumeSpecName: "util") pod "0ceb213f-4e8b-4567-865d-5f370ebc38ea" (UID: "0ceb213f-4e8b-4567-865d-5f370ebc38ea"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 03:44:40.689184 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:40.689107    2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0ceed716-436d-4616-a637-524eed35ff96-util\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:44:40.689184 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:40.689134    2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0ceb213f-4e8b-4567-865d-5f370ebc38ea-bundle\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:44:40.689184 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:40.689144    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hbmm\" (UniqueName: \"kubernetes.io/projected/9ada5a4e-7a8c-413b-a838-2d98660edf99-kube-api-access-9hbmm\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:44:40.689184 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:40.689153    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dg82s\" (UniqueName: \"kubernetes.io/projected/0ceed716-436d-4616-a637-524eed35ff96-kube-api-access-dg82s\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:44:40.689184 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:40.689161    2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0ceb213f-4e8b-4567-865d-5f370ebc38ea-util\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:44:40.689184 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:40.689171    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wgfvb\" (UniqueName: \"kubernetes.io/projected/0ceb213f-4e8b-4567-865d-5f370ebc38ea-kube-api-access-wgfvb\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:44:40.689184 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:40.689178    2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0ceed716-436d-4616-a637-524eed35ff96-bundle\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:44:40.689184 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:40.689186    2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9ada5a4e-7a8c-413b-a838-2d98660edf99-bundle\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:44:41.014790 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:41.014711    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ada5a4e-7a8c-413b-a838-2d98660edf99-util" (OuterVolumeSpecName: "util") pod "9ada5a4e-7a8c-413b-a838-2d98660edf99" (UID: "9ada5a4e-7a8c-413b-a838-2d98660edf99"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 03:44:41.091935 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:41.091894    2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9ada5a4e-7a8c-413b-a838-2d98660edf99-util\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:44:41.365155 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:41.365131    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqfb"
Jul 30 03:44:41.365155 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:41.365147    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqfb" event={"ID":"0ceed716-436d-4616-a637-524eed35ff96","Type":"ContainerDied","Data":"d43f7dfa11bda7dfb106187f34ecbce6ba3936154363cfdd33dd410cc0cd2bc8"}
Jul 30 03:44:41.365642 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:41.365195    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d43f7dfa11bda7dfb106187f34ecbce6ba3936154363cfdd33dd410cc0cd2bc8"
Jul 30 03:44:41.367019 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:41.366997    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cst2q"
Jul 30 03:44:41.367196 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:41.367026    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cst2q" event={"ID":"0ceb213f-4e8b-4567-865d-5f370ebc38ea","Type":"ContainerDied","Data":"c2bb66bf1a8d6b1ff642cc9f1ba616de612ea925ee962102b389cb4329e3c140"}
Jul 30 03:44:41.367196 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:41.367066    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c2bb66bf1a8d6b1ff642cc9f1ba616de612ea925ee962102b389cb4329e3c140"
Jul 30 03:44:41.369419 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:41.369399    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r4s7n" event={"ID":"9ada5a4e-7a8c-413b-a838-2d98660edf99","Type":"ContainerDied","Data":"4cbcce96c23654201d43239fd4f9b4442adc236d594e9f1de42127ad9017a842"}
Jul 30 03:44:41.369525 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:41.369430    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4cbcce96c23654201d43239fd4f9b4442adc236d594e9f1de42127ad9017a842"
Jul 30 03:44:41.369679 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:41.369665    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r4s7n"
Jul 30 03:44:46.063147 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:46.063114    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:44:47.871271 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:47.871238    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:44:51.780633 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:51.780602    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:44:52.866116 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:52.866086    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5cl74"]
Jul 30 03:44:52.866493 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:52.866405    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ceb213f-4e8b-4567-865d-5f370ebc38ea" containerName="pull"
Jul 30 03:44:52.866493 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:52.866416    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ceb213f-4e8b-4567-865d-5f370ebc38ea" containerName="pull"
Jul 30 03:44:52.866493 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:52.866429    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ceed716-436d-4616-a637-524eed35ff96" containerName="util"
Jul 30 03:44:52.866493 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:52.866434    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ceed716-436d-4616-a637-524eed35ff96" containerName="util"
Jul 30 03:44:52.866493 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:52.866441    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f32f58e2-cb21-499f-b72c-4ce90eed5bba" containerName="extract"
Jul 30 03:44:52.866493 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:52.866447    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f32f58e2-cb21-499f-b72c-4ce90eed5bba" containerName="extract"
Jul 30 03:44:52.866493 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:52.866461    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ceb213f-4e8b-4567-865d-5f370ebc38ea" containerName="util"
Jul 30 03:44:52.866493 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:52.866468    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ceb213f-4e8b-4567-865d-5f370ebc38ea" containerName="util"
Jul 30 03:44:52.866493 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:52.866473    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f32f58e2-cb21-499f-b72c-4ce90eed5bba" containerName="pull"
Jul 30 03:44:52.866493 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:52.866479    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f32f58e2-cb21-499f-b72c-4ce90eed5bba" containerName="pull"
Jul 30 03:44:52.866493 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:52.866485    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ada5a4e-7a8c-413b-a838-2d98660edf99" containerName="extract"
Jul 30 03:44:52.866493 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:52.866490    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ada5a4e-7a8c-413b-a838-2d98660edf99" containerName="extract"
Jul 30 03:44:52.866493 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:52.866498    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ada5a4e-7a8c-413b-a838-2d98660edf99" containerName="pull"
Jul 30 03:44:52.866965 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:52.866503    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ada5a4e-7a8c-413b-a838-2d98660edf99" containerName="pull"
Jul 30 03:44:52.866965 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:52.866510    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ceed716-436d-4616-a637-524eed35ff96" containerName="pull"
Jul 30 03:44:52.866965 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:52.866515    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ceed716-436d-4616-a637-524eed35ff96" containerName="pull"
Jul 30 03:44:52.866965 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:52.866520    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ceed716-436d-4616-a637-524eed35ff96" containerName="extract"
Jul 30 03:44:52.866965 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:52.866524    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ceed716-436d-4616-a637-524eed35ff96" containerName="extract"
Jul 30 03:44:52.866965 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:52.866534    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f32f58e2-cb21-499f-b72c-4ce90eed5bba" containerName="util"
Jul 30 03:44:52.866965 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:52.866542    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f32f58e2-cb21-499f-b72c-4ce90eed5bba" containerName="util"
Jul 30 03:44:52.866965 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:52.866550    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ada5a4e-7a8c-413b-a838-2d98660edf99" containerName="util"
Jul 30 03:44:52.866965 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:52.866556    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ada5a4e-7a8c-413b-a838-2d98660edf99" containerName="util"
Jul 30 03:44:52.866965 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:52.866563    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ceb213f-4e8b-4567-865d-5f370ebc38ea" containerName="extract"
Jul 30 03:44:52.866965 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:52.866567    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ceb213f-4e8b-4567-865d-5f370ebc38ea" containerName="extract"
Jul 30 03:44:52.866965 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:52.866612    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ada5a4e-7a8c-413b-a838-2d98660edf99" containerName="extract"
Jul 30 03:44:52.866965 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:52.866624    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ceb213f-4e8b-4567-865d-5f370ebc38ea" containerName="extract"
Jul 30 03:44:52.866965 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:52.866632    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f32f58e2-cb21-499f-b72c-4ce90eed5bba" containerName="extract"
Jul 30 03:44:52.866965 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:52.866639    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ceed716-436d-4616-a637-524eed35ff96" containerName="extract"
Jul 30 03:44:52.868438 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:52.868422    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5cl74"
Jul 30 03:44:52.870974 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:52.870949    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-6v4xr\""
Jul 30 03:44:52.890224 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:52.890198    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5cl74"]
Jul 30 03:44:52.984744 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:52.984710    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/425c18cd-881a-4cb8-babb-fd7210fd0e9f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5cl74\" (UID: \"425c18cd-881a-4cb8-babb-fd7210fd0e9f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5cl74"
Jul 30 03:44:52.984962 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:52.984786    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74flt\" (UniqueName: \"kubernetes.io/projected/425c18cd-881a-4cb8-babb-fd7210fd0e9f-kube-api-access-74flt\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5cl74\" (UID: \"425c18cd-881a-4cb8-babb-fd7210fd0e9f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5cl74"
Jul 30 03:44:53.066511 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:53.066482    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:44:53.085682 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:53.085654    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74flt\" (UniqueName: \"kubernetes.io/projected/425c18cd-881a-4cb8-babb-fd7210fd0e9f-kube-api-access-74flt\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5cl74\" (UID: \"425c18cd-881a-4cb8-babb-fd7210fd0e9f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5cl74"
Jul 30 03:44:53.085812 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:53.085733    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/425c18cd-881a-4cb8-babb-fd7210fd0e9f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5cl74\" (UID: \"425c18cd-881a-4cb8-babb-fd7210fd0e9f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5cl74"
Jul 30 03:44:53.086123 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:53.086103    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/425c18cd-881a-4cb8-babb-fd7210fd0e9f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5cl74\" (UID: \"425c18cd-881a-4cb8-babb-fd7210fd0e9f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5cl74"
Jul 30 03:44:53.096941 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:53.096909    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74flt\" (UniqueName: \"kubernetes.io/projected/425c18cd-881a-4cb8-babb-fd7210fd0e9f-kube-api-access-74flt\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5cl74\" (UID: \"425c18cd-881a-4cb8-babb-fd7210fd0e9f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5cl74"
Jul 30 03:44:53.178571 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:53.178491    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5cl74"
Jul 30 03:44:53.306664 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:53.306641    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5cl74"]
Jul 30 03:44:53.308298 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:44:53.308274    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod425c18cd_881a_4cb8_babb_fd7210fd0e9f.slice/crio-16003df7581f61d9250a3ac6319545b10a69a9a8d5ccd02016864cf326eef54c WatchSource:0}: Error finding container 16003df7581f61d9250a3ac6319545b10a69a9a8d5ccd02016864cf326eef54c: Status 404 returned error can't find the container with id 16003df7581f61d9250a3ac6319545b10a69a9a8d5ccd02016864cf326eef54c
Jul 30 03:44:53.425526 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:53.425493    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5cl74" event={"ID":"425c18cd-881a-4cb8-babb-fd7210fd0e9f","Type":"ContainerStarted","Data":"16003df7581f61d9250a3ac6319545b10a69a9a8d5ccd02016864cf326eef54c"}
Jul 30 03:44:53.477158 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:53.477075    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:44:54.514552 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:54.514518    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:44:55.369494 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:55.369450    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-8pv87"]
Jul 30 03:44:55.372561 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:55.372537    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8pv87"
Jul 30 03:44:55.375776 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:55.375754    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-7qzmp\""
Jul 30 03:44:55.385460 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:55.385435    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-8pv87"]
Jul 30 03:44:55.505776 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:55.505738    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxtl2\" (UniqueName: \"kubernetes.io/projected/2492ddbc-0304-4f0e-a209-3fb9e789899e-kube-api-access-zxtl2\") pod \"authorino-operator-75cc8d5cb7-8pv87\" (UID: \"2492ddbc-0304-4f0e-a209-3fb9e789899e\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-8pv87"
Jul 30 03:44:55.607036 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:55.606994    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxtl2\" (UniqueName: \"kubernetes.io/projected/2492ddbc-0304-4f0e-a209-3fb9e789899e-kube-api-access-zxtl2\") pod \"authorino-operator-75cc8d5cb7-8pv87\" (UID: \"2492ddbc-0304-4f0e-a209-3fb9e789899e\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-8pv87"
Jul 30 03:44:55.615167 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:55.615138    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxtl2\" (UniqueName: \"kubernetes.io/projected/2492ddbc-0304-4f0e-a209-3fb9e789899e-kube-api-access-zxtl2\") pod \"authorino-operator-75cc8d5cb7-8pv87\" (UID: \"2492ddbc-0304-4f0e-a209-3fb9e789899e\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-8pv87"
Jul 30 03:44:55.682115 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:55.682051    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8pv87"
Jul 30 03:44:55.803056 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:55.803036    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-8pv87"]
Jul 30 03:44:55.805844 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:44:55.805820    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2492ddbc_0304_4f0e_a209_3fb9e789899e.slice/crio-fcd248d7b81758858844c0f56cef15f141e36060b17f81db99a81971297a6493 WatchSource:0}: Error finding container fcd248d7b81758858844c0f56cef15f141e36060b17f81db99a81971297a6493: Status 404 returned error can't find the container with id fcd248d7b81758858844c0f56cef15f141e36060b17f81db99a81971297a6493
Jul 30 03:44:56.143057 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:56.143032    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:44:56.280353 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:56.280314    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:44:56.444899 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:56.444812    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8pv87" event={"ID":"2492ddbc-0304-4f0e-a209-3fb9e789899e","Type":"ContainerStarted","Data":"fcd248d7b81758858844c0f56cef15f141e36060b17f81db99a81971297a6493"}
Jul 30 03:44:56.596415 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:56.596380    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-594f54c687-9m247"]
Jul 30 03:44:56.599053 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:56.599030    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-594f54c687-9m247"
Jul 30 03:44:56.612203 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:56.612182    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-594f54c687-9m247"]
Jul 30 03:44:56.679145 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:56.679103    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:44:56.718022 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:56.717945    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9e2e2686-79f6-4374-8e8b-6374c28d8f82-console-config\") pod \"console-594f54c687-9m247\" (UID: \"9e2e2686-79f6-4374-8e8b-6374c28d8f82\") " pod="openshift-console/console-594f54c687-9m247"
Jul 30 03:44:56.718022 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:56.717998    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e2e2686-79f6-4374-8e8b-6374c28d8f82-trusted-ca-bundle\") pod \"console-594f54c687-9m247\" (UID: \"9e2e2686-79f6-4374-8e8b-6374c28d8f82\") " pod="openshift-console/console-594f54c687-9m247"
Jul 30 03:44:56.718242 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:56.718032    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9e2e2686-79f6-4374-8e8b-6374c28d8f82-console-serving-cert\") pod \"console-594f54c687-9m247\" (UID: \"9e2e2686-79f6-4374-8e8b-6374c28d8f82\") " pod="openshift-console/console-594f54c687-9m247"
Jul 30 03:44:56.718242 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:56.718056    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9e2e2686-79f6-4374-8e8b-6374c28d8f82-console-oauth-config\") pod \"console-594f54c687-9m247\" (UID: \"9e2e2686-79f6-4374-8e8b-6374c28d8f82\") " pod="openshift-console/console-594f54c687-9m247"
Jul 30 03:44:56.718242 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:56.718132    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9e2e2686-79f6-4374-8e8b-6374c28d8f82-oauth-serving-cert\") pod \"console-594f54c687-9m247\" (UID: \"9e2e2686-79f6-4374-8e8b-6374c28d8f82\") " pod="openshift-console/console-594f54c687-9m247"
Jul 30 03:44:56.718242 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:56.718227    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nvzv\" (UniqueName: \"kubernetes.io/projected/9e2e2686-79f6-4374-8e8b-6374c28d8f82-kube-api-access-6nvzv\") pod \"console-594f54c687-9m247\" (UID: \"9e2e2686-79f6-4374-8e8b-6374c28d8f82\") " pod="openshift-console/console-594f54c687-9m247"
Jul 30 03:44:56.718414 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:56.718264    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9e2e2686-79f6-4374-8e8b-6374c28d8f82-service-ca\") pod \"console-594f54c687-9m247\" (UID: \"9e2e2686-79f6-4374-8e8b-6374c28d8f82\") " pod="openshift-console/console-594f54c687-9m247"
Jul 30 03:44:56.768948 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:56.768888    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:44:56.819968 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:56.819242    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nvzv\" (UniqueName: \"kubernetes.io/projected/9e2e2686-79f6-4374-8e8b-6374c28d8f82-kube-api-access-6nvzv\") pod \"console-594f54c687-9m247\" (UID: \"9e2e2686-79f6-4374-8e8b-6374c28d8f82\") " pod="openshift-console/console-594f54c687-9m247"
Jul 30 03:44:56.819968 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:56.819299    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9e2e2686-79f6-4374-8e8b-6374c28d8f82-service-ca\") pod \"console-594f54c687-9m247\" (UID: \"9e2e2686-79f6-4374-8e8b-6374c28d8f82\") " pod="openshift-console/console-594f54c687-9m247"
Jul 30 03:44:56.819968 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:56.819352    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9e2e2686-79f6-4374-8e8b-6374c28d8f82-console-config\") pod \"console-594f54c687-9m247\" (UID: \"9e2e2686-79f6-4374-8e8b-6374c28d8f82\") " pod="openshift-console/console-594f54c687-9m247"
Jul 30 03:44:56.819968 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:56.819385    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e2e2686-79f6-4374-8e8b-6374c28d8f82-trusted-ca-bundle\") pod \"console-594f54c687-9m247\" (UID: \"9e2e2686-79f6-4374-8e8b-6374c28d8f82\") " pod="openshift-console/console-594f54c687-9m247"
Jul 30 03:44:56.819968 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:56.819409    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9e2e2686-79f6-4374-8e8b-6374c28d8f82-console-serving-cert\") pod \"console-594f54c687-9m247\" (UID: \"9e2e2686-79f6-4374-8e8b-6374c28d8f82\") " pod="openshift-console/console-594f54c687-9m247"
Jul 30 03:44:56.819968 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:56.819431    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9e2e2686-79f6-4374-8e8b-6374c28d8f82-console-oauth-config\") pod \"console-594f54c687-9m247\" (UID: \"9e2e2686-79f6-4374-8e8b-6374c28d8f82\") " pod="openshift-console/console-594f54c687-9m247"
Jul 30 03:44:56.819968 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:56.819460    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9e2e2686-79f6-4374-8e8b-6374c28d8f82-oauth-serving-cert\") pod \"console-594f54c687-9m247\" (UID: \"9e2e2686-79f6-4374-8e8b-6374c28d8f82\") " pod="openshift-console/console-594f54c687-9m247"
Jul 30 03:44:56.820452 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:56.820105    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9e2e2686-79f6-4374-8e8b-6374c28d8f82-service-ca\") pod \"console-594f54c687-9m247\" (UID: \"9e2e2686-79f6-4374-8e8b-6374c28d8f82\") " pod="openshift-console/console-594f54c687-9m247"
Jul 30 03:44:56.820452 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:56.820208    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9e2e2686-79f6-4374-8e8b-6374c28d8f82-oauth-serving-cert\") pod \"console-594f54c687-9m247\" (UID: \"9e2e2686-79f6-4374-8e8b-6374c28d8f82\") " pod="openshift-console/console-594f54c687-9m247"
Jul 30 03:44:56.820452 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:56.820408    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e2e2686-79f6-4374-8e8b-6374c28d8f82-trusted-ca-bundle\") pod \"console-594f54c687-9m247\" (UID: \"9e2e2686-79f6-4374-8e8b-6374c28d8f82\") " pod="openshift-console/console-594f54c687-9m247"
Jul 30 03:44:56.820772 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:56.820726    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9e2e2686-79f6-4374-8e8b-6374c28d8f82-console-config\") pod \"console-594f54c687-9m247\" (UID: \"9e2e2686-79f6-4374-8e8b-6374c28d8f82\") " pod="openshift-console/console-594f54c687-9m247"
Jul 30 03:44:56.824295 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:56.824250    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9e2e2686-79f6-4374-8e8b-6374c28d8f82-console-oauth-config\") pod \"console-594f54c687-9m247\" (UID: \"9e2e2686-79f6-4374-8e8b-6374c28d8f82\") " pod="openshift-console/console-594f54c687-9m247"
Jul 30 03:44:56.824943 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:56.824879    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9e2e2686-79f6-4374-8e8b-6374c28d8f82-console-serving-cert\") pod \"console-594f54c687-9m247\" (UID: \"9e2e2686-79f6-4374-8e8b-6374c28d8f82\") " pod="openshift-console/console-594f54c687-9m247"
Jul 30 03:44:56.828507 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:56.828483    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nvzv\" (UniqueName: \"kubernetes.io/projected/9e2e2686-79f6-4374-8e8b-6374c28d8f82-kube-api-access-6nvzv\") pod \"console-594f54c687-9m247\" (UID: \"9e2e2686-79f6-4374-8e8b-6374c28d8f82\") " pod="openshift-console/console-594f54c687-9m247"
Jul 30 03:44:56.910302 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:56.910264    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-594f54c687-9m247"
Jul 30 03:44:57.675963 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:57.675840    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-x5rnh"]
Jul 30 03:44:57.692192 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:57.678599    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x5rnh"
Jul 30 03:44:57.692192 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:57.681716    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 30 03:44:57.692192 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:57.681787    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-8q9vt\""
Jul 30 03:44:57.692827 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:57.692806    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-x5rnh"]
Jul 30 03:44:57.827972 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:57.827914    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvxml\" (UniqueName: \"kubernetes.io/projected/6e3c0ae4-6549-4214-a3cd-101ec3deaabf-kube-api-access-bvxml\") pod \"dns-operator-controller-manager-864dd78c4c-x5rnh\" (UID: \"6e3c0ae4-6549-4214-a3cd-101ec3deaabf\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x5rnh"
Jul 30 03:44:57.929090 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:57.929006    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvxml\" (UniqueName: \"kubernetes.io/projected/6e3c0ae4-6549-4214-a3cd-101ec3deaabf-kube-api-access-bvxml\") pod \"dns-operator-controller-manager-864dd78c4c-x5rnh\" (UID: \"6e3c0ae4-6549-4214-a3cd-101ec3deaabf\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x5rnh"
Jul 30 03:44:57.937897 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:57.937865    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvxml\" (UniqueName: \"kubernetes.io/projected/6e3c0ae4-6549-4214-a3cd-101ec3deaabf-kube-api-access-bvxml\") pod \"dns-operator-controller-manager-864dd78c4c-x5rnh\" (UID: \"6e3c0ae4-6549-4214-a3cd-101ec3deaabf\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x5rnh"
Jul 30 03:44:57.991656 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:57.991619    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x5rnh"
Jul 30 03:44:58.329207 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:58.329147    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:44:59.090620 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:59.090596    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-x5rnh"]
Jul 30 03:44:59.091779 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:44:59.091751    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e3c0ae4_6549_4214_a3cd_101ec3deaabf.slice/crio-83364d38f8c9f1ab880cb71edd417c12a9f076aeeeb38f582bc4ce12f9ece4e1 WatchSource:0}: Error finding container 83364d38f8c9f1ab880cb71edd417c12a9f076aeeeb38f582bc4ce12f9ece4e1: Status 404 returned error can't find the container with id 83364d38f8c9f1ab880cb71edd417c12a9f076aeeeb38f582bc4ce12f9ece4e1
Jul 30 03:44:59.115064 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:59.115042    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-594f54c687-9m247"]
Jul 30 03:44:59.117368 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:44:59.117346    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e2e2686_79f6_4374_8e8b_6374c28d8f82.slice/crio-8802809725f12e0f0b8711b54cefc13b669298f4405455c1ff7531a8452a9348 WatchSource:0}: Error finding container 8802809725f12e0f0b8711b54cefc13b669298f4405455c1ff7531a8452a9348: Status 404 returned error can't find the container with id 8802809725f12e0f0b8711b54cefc13b669298f4405455c1ff7531a8452a9348
Jul 30 03:44:59.461972 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:59.461917    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8pv87" event={"ID":"2492ddbc-0304-4f0e-a209-3fb9e789899e","Type":"ContainerStarted","Data":"9e7391cff3a1834ee8cf69c4e36ee72a8f603721cf9213d4ac610e70a00f8445"}
Jul 30 03:44:59.462172 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:59.462005    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8pv87"
Jul 30 03:44:59.463447 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:59.463420    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-594f54c687-9m247" event={"ID":"9e2e2686-79f6-4374-8e8b-6374c28d8f82","Type":"ContainerStarted","Data":"34e349bb4f8c51f0c17dc09398ab78f1b84eb4d26af79ea4e2c90636fd313b6b"}
Jul 30 03:44:59.463579 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:59.463464    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-594f54c687-9m247" event={"ID":"9e2e2686-79f6-4374-8e8b-6374c28d8f82","Type":"ContainerStarted","Data":"8802809725f12e0f0b8711b54cefc13b669298f4405455c1ff7531a8452a9348"}
Jul 30 03:44:59.464699 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:59.464669    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x5rnh" event={"ID":"6e3c0ae4-6549-4214-a3cd-101ec3deaabf","Type":"ContainerStarted","Data":"83364d38f8c9f1ab880cb71edd417c12a9f076aeeeb38f582bc4ce12f9ece4e1"}
Jul 30 03:44:59.466217 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:59.466196    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5cl74" event={"ID":"425c18cd-881a-4cb8-babb-fd7210fd0e9f","Type":"ContainerStarted","Data":"381e3198e4ce66d08621652dece56d7768e2530669e5ba13d468ab7e2b76616c"}
Jul 30 03:44:59.466321 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:59.466311    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5cl74"
Jul 30 03:44:59.478978 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:59.478911    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8pv87" podStartSLOduration=1.311683683 podStartE2EDuration="4.478894982s" totalImagesPullingTime="3.167211299s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:44:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:44:59.478653515 +0000 UTC m=+605.096885757" watchObservedRunningTime="2026-07-30 03:44:59.478894982 +0000 UTC m=+605.097127223"
Jul 30 03:44:59.502067 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:59.502025    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-594f54c687-9m247" podStartSLOduration=3.502011727 podStartE2EDuration="3.502011727s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:44:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:44:59.501310553 +0000 UTC m=+605.119542796" watchObservedRunningTime="2026-07-30 03:44:59.502011727 +0000 UTC m=+605.120244012"
Jul 30 03:44:59.528036 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:44:59.528001    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5cl74" podStartSLOduration=1.8938052349999999 podStartE2EDuration="7.527988717s" totalImagesPullingTime="5.634183482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:44:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:44:59.527159513 +0000 UTC m=+605.145391754" watchObservedRunningTime="2026-07-30 03:44:59.527988717 +0000 UTC m=+605.146220957"
Jul 30 03:45:00.126661 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:00.126625    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:01.476984 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:01.476876    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x5rnh" event={"ID":"6e3c0ae4-6549-4214-a3cd-101ec3deaabf","Type":"ContainerStarted","Data":"2f1d66b2a63a82c90b0fe825d421cd2a4e53bac4aadac03aef089eb338d20bf0"}
Jul 30 03:45:01.477405 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:01.477039    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x5rnh"
Jul 30 03:45:01.494331 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:01.494287    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x5rnh" podStartSLOduration=2.437415864 podStartE2EDuration="4.494273289s" totalImagesPullingTime="2.056857425s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:44:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:45:01.492620859 +0000 UTC m=+607.110853101" watchObservedRunningTime="2026-07-30 03:45:01.494273289 +0000 UTC m=+607.112505530"
Jul 30 03:45:01.877551 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:01.877514    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-vk9f7"]
Jul 30 03:45:01.880678 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:01.880654    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:01.880792 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:01.880759    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk9f7"
Jul 30 03:45:01.883388 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:01.883368    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 30 03:45:01.883470 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:01.883409    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kqcjr\""
Jul 30 03:45:01.883470 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:01.883462    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 30 03:45:01.888276 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:01.888255    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-vk9f7"]
Jul 30 03:45:01.980797 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:01.980764    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:02.065911 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:02.065868    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xg5b\" (UniqueName: \"kubernetes.io/projected/6e2fe08c-820d-4da0-91ca-752de17b3bf2-kube-api-access-2xg5b\") pod \"kuadrant-console-plugin-66b5774985-vk9f7\" (UID: \"6e2fe08c-820d-4da0-91ca-752de17b3bf2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk9f7"
Jul 30 03:45:02.066104 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:02.066024    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6e2fe08c-820d-4da0-91ca-752de17b3bf2-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-vk9f7\" (UID: \"6e2fe08c-820d-4da0-91ca-752de17b3bf2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk9f7"
Jul 30 03:45:02.066104 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:02.066088    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e2fe08c-820d-4da0-91ca-752de17b3bf2-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-vk9f7\" (UID: \"6e2fe08c-820d-4da0-91ca-752de17b3bf2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk9f7"
Jul 30 03:45:02.166805 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:02.166729    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xg5b\" (UniqueName: \"kubernetes.io/projected/6e2fe08c-820d-4da0-91ca-752de17b3bf2-kube-api-access-2xg5b\") pod \"kuadrant-console-plugin-66b5774985-vk9f7\" (UID: \"6e2fe08c-820d-4da0-91ca-752de17b3bf2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk9f7"
Jul 30 03:45:02.166990 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:02.166826    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6e2fe08c-820d-4da0-91ca-752de17b3bf2-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-vk9f7\" (UID: \"6e2fe08c-820d-4da0-91ca-752de17b3bf2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk9f7"
Jul 30 03:45:02.166990 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:02.166859    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e2fe08c-820d-4da0-91ca-752de17b3bf2-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-vk9f7\" (UID: \"6e2fe08c-820d-4da0-91ca-752de17b3bf2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk9f7"
Jul 30 03:45:02.166990 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:45:02.166985    2576 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 30 03:45:02.167103 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:45:02.167050    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e2fe08c-820d-4da0-91ca-752de17b3bf2-plugin-serving-cert podName:6e2fe08c-820d-4da0-91ca-752de17b3bf2 nodeName:}" failed. No retries permitted until 2026-07-30 03:45:02.667030103 +0000 UTC m=+608.285262326 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/6e2fe08c-820d-4da0-91ca-752de17b3bf2-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-vk9f7" (UID: "6e2fe08c-820d-4da0-91ca-752de17b3bf2") : secret "plugin-serving-cert" not found
Jul 30 03:45:02.167515 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:02.167495    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6e2fe08c-820d-4da0-91ca-752de17b3bf2-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-vk9f7\" (UID: \"6e2fe08c-820d-4da0-91ca-752de17b3bf2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk9f7"
Jul 30 03:45:02.175615 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:02.175596    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xg5b\" (UniqueName: \"kubernetes.io/projected/6e2fe08c-820d-4da0-91ca-752de17b3bf2-kube-api-access-2xg5b\") pod \"kuadrant-console-plugin-66b5774985-vk9f7\" (UID: \"6e2fe08c-820d-4da0-91ca-752de17b3bf2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk9f7"
Jul 30 03:45:02.405058 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:02.405026    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:02.670812 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:02.670779    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e2fe08c-820d-4da0-91ca-752de17b3bf2-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-vk9f7\" (UID: \"6e2fe08c-820d-4da0-91ca-752de17b3bf2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk9f7"
Jul 30 03:45:02.673144 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:02.673123    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e2fe08c-820d-4da0-91ca-752de17b3bf2-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-vk9f7\" (UID: \"6e2fe08c-820d-4da0-91ca-752de17b3bf2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk9f7"
Jul 30 03:45:02.790627 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:02.790588    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk9f7"
Jul 30 03:45:02.917156 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:02.917131    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-vk9f7"]
Jul 30 03:45:02.918715 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:45:02.918686    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6e2fe08c_820d_4da0_91ca_752de17b3bf2.slice/crio-536105799a86eae33d4e09b0d9ecec9dbfeeedb073e8c367cde79c05fa6323f9 WatchSource:0}: Error finding container 536105799a86eae33d4e09b0d9ecec9dbfeeedb073e8c367cde79c05fa6323f9: Status 404 returned error can't find the container with id 536105799a86eae33d4e09b0d9ecec9dbfeeedb073e8c367cde79c05fa6323f9
Jul 30 03:45:03.488977 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:03.488938    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk9f7" event={"ID":"6e2fe08c-820d-4da0-91ca-752de17b3bf2","Type":"ContainerStarted","Data":"536105799a86eae33d4e09b0d9ecec9dbfeeedb073e8c367cde79c05fa6323f9"}
Jul 30 03:45:04.193663 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:04.193634    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:04.254740 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:04.254707    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:06.911038 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:06.911004    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-594f54c687-9m247"
Jul 30 03:45:06.911555 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:06.911078    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-594f54c687-9m247"
Jul 30 03:45:06.917037 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:06.917010    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-594f54c687-9m247"
Jul 30 03:45:07.518175 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:07.518141    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-594f54c687-9m247"
Jul 30 03:45:07.587293 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:07.587259    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5fdbf99bdb-hw8kc"]
Jul 30 03:45:07.698259 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:07.698222    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:07.799376 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:07.799342    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:10.473830 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:10.473798    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5cl74"
Jul 30 03:45:10.474431 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:10.473885    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8pv87"
Jul 30 03:45:11.281131 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:11.281101    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:11.407730 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:11.407698    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:12.212823 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:12.212787    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5cl74"]
Jul 30 03:45:12.213294 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:12.212988    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5cl74" podUID="425c18cd-881a-4cb8-babb-fd7210fd0e9f" containerName="manager" containerID="cri-o://381e3198e4ce66d08621652dece56d7768e2530669e5ba13d468ab7e2b76616c" gracePeriod=2
Jul 30 03:45:12.222445 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:12.222416    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5cl74"]
Jul 30 03:45:12.223072 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:45:12.223028    2576 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"425c18cd-881a-4cb8-babb-fd7210fd0e9f\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-5cl74\": pods \"kuadrant-operator-controller-manager-8684c9ff49-5cl74\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5cl74"
Jul 30 03:45:12.235029 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:12.234999    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pnx7l"]
Jul 30 03:45:12.235500 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:12.235481    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="425c18cd-881a-4cb8-babb-fd7210fd0e9f" containerName="manager"
Jul 30 03:45:12.235594 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:12.235504    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="425c18cd-881a-4cb8-babb-fd7210fd0e9f" containerName="manager"
Jul 30 03:45:12.235633 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:12.235611    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="425c18cd-881a-4cb8-babb-fd7210fd0e9f" containerName="manager"
Jul 30 03:45:12.240120 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:12.240101    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pnx7l"
Jul 30 03:45:12.242716 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:45:12.242688    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-5cl74\" is forbidden: User \"system:node:ip-10-0-143-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-117.ec2.internal' and this object" podUID="425c18cd-881a-4cb8-babb-fd7210fd0e9f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5cl74"
Jul 30 03:45:12.253145 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:12.252945    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlwjb\" (UniqueName: \"kubernetes.io/projected/32965b1f-6bdf-41f5-8a48-831d145e92c0-kube-api-access-xlwjb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pnx7l\" (UID: \"32965b1f-6bdf-41f5-8a48-831d145e92c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pnx7l"
Jul 30 03:45:12.253145 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:12.253029    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/32965b1f-6bdf-41f5-8a48-831d145e92c0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pnx7l\" (UID: \"32965b1f-6bdf-41f5-8a48-831d145e92c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pnx7l"
Jul 30 03:45:12.253831 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:12.253810    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pnx7l"]
Jul 30 03:45:12.353893 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:12.353853    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/32965b1f-6bdf-41f5-8a48-831d145e92c0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pnx7l\" (UID: \"32965b1f-6bdf-41f5-8a48-831d145e92c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pnx7l"
Jul 30 03:45:12.354102 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:12.353979    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlwjb\" (UniqueName: \"kubernetes.io/projected/32965b1f-6bdf-41f5-8a48-831d145e92c0-kube-api-access-xlwjb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pnx7l\" (UID: \"32965b1f-6bdf-41f5-8a48-831d145e92c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pnx7l"
Jul 30 03:45:12.354278 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:12.354261    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/32965b1f-6bdf-41f5-8a48-831d145e92c0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pnx7l\" (UID: \"32965b1f-6bdf-41f5-8a48-831d145e92c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pnx7l"
Jul 30 03:45:12.363400 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:12.363376    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlwjb\" (UniqueName: \"kubernetes.io/projected/32965b1f-6bdf-41f5-8a48-831d145e92c0-kube-api-access-xlwjb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pnx7l\" (UID: \"32965b1f-6bdf-41f5-8a48-831d145e92c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pnx7l"
Jul 30 03:45:12.485782 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:12.485699    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x5rnh"
Jul 30 03:45:12.505077 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:45:12.505035    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-5cl74\" is forbidden: User \"system:node:ip-10-0-143-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-117.ec2.internal' and this object" podUID="425c18cd-881a-4cb8-babb-fd7210fd0e9f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5cl74"
Jul 30 03:45:12.574662 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:12.574628    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pnx7l"
Jul 30 03:45:13.582938 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:13.582886    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:13.684286 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:13.684245    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:18.293709 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:18.293617    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:18.414681 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:18.414652    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:22.098745 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:22.098709    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:22.160288 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:22.160257    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:24.383005 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:24.382975    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pnx7l"]
Jul 30 03:45:24.384411 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:45:24.384386    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32965b1f_6bdf_41f5_8a48_831d145e92c0.slice/crio-4695786018e7fcf6b1a5cc96f64833122bf29a6a4726ba0cee821769bc658427 WatchSource:0}: Error finding container 4695786018e7fcf6b1a5cc96f64833122bf29a6a4726ba0cee821769bc658427: Status 404 returned error can't find the container with id 4695786018e7fcf6b1a5cc96f64833122bf29a6a4726ba0cee821769bc658427
Jul 30 03:45:24.409166 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:24.409137    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5cl74"
Jul 30 03:45:24.411461 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:45:24.411434    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-5cl74\" is forbidden: User \"system:node:ip-10-0-143-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-117.ec2.internal' and this object" podUID="425c18cd-881a-4cb8-babb-fd7210fd0e9f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5cl74"
Jul 30 03:45:24.474558 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:24.474536    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74flt\" (UniqueName: \"kubernetes.io/projected/425c18cd-881a-4cb8-babb-fd7210fd0e9f-kube-api-access-74flt\") pod \"425c18cd-881a-4cb8-babb-fd7210fd0e9f\" (UID: \"425c18cd-881a-4cb8-babb-fd7210fd0e9f\") "
Jul 30 03:45:24.474680 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:24.474579    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/425c18cd-881a-4cb8-babb-fd7210fd0e9f-extensions-socket-volume\") pod \"425c18cd-881a-4cb8-babb-fd7210fd0e9f\" (UID: \"425c18cd-881a-4cb8-babb-fd7210fd0e9f\") "
Jul 30 03:45:24.475107 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:24.475079    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/425c18cd-881a-4cb8-babb-fd7210fd0e9f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "425c18cd-881a-4cb8-babb-fd7210fd0e9f" (UID: "425c18cd-881a-4cb8-babb-fd7210fd0e9f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 03:45:24.476425 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:24.476405    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/425c18cd-881a-4cb8-babb-fd7210fd0e9f-kube-api-access-74flt" (OuterVolumeSpecName: "kube-api-access-74flt") pod "425c18cd-881a-4cb8-babb-fd7210fd0e9f" (UID: "425c18cd-881a-4cb8-babb-fd7210fd0e9f"). InnerVolumeSpecName "kube-api-access-74flt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:45:24.575539 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:24.575465    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74flt\" (UniqueName: \"kubernetes.io/projected/425c18cd-881a-4cb8-babb-fd7210fd0e9f-kube-api-access-74flt\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:45:24.575539 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:24.575492    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/425c18cd-881a-4cb8-babb-fd7210fd0e9f-extensions-socket-volume\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:45:24.609043 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:24.609001    2576 generic.go:356] "Generic (PLEG): container finished" podID="425c18cd-881a-4cb8-babb-fd7210fd0e9f" containerID="381e3198e4ce66d08621652dece56d7768e2530669e5ba13d468ab7e2b76616c" exitCode=0
Jul 30 03:45:24.609043 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:24.609043    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5cl74"
Jul 30 03:45:24.609272 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:24.609056    2576 scope.go:117] "RemoveContainer" containerID="381e3198e4ce66d08621652dece56d7768e2530669e5ba13d468ab7e2b76616c"
Jul 30 03:45:24.611153 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:24.611128    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pnx7l" event={"ID":"32965b1f-6bdf-41f5-8a48-831d145e92c0","Type":"ContainerStarted","Data":"746af3122ac7d67b0175f6de96f386a94129c49784f6a4ef97961c62dbb70a69"}
Jul 30 03:45:24.611289 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:24.611182    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pnx7l" event={"ID":"32965b1f-6bdf-41f5-8a48-831d145e92c0","Type":"ContainerStarted","Data":"4695786018e7fcf6b1a5cc96f64833122bf29a6a4726ba0cee821769bc658427"}
Jul 30 03:45:24.611289 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:24.611226    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pnx7l"
Jul 30 03:45:24.611577 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:45:24.611553    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-5cl74\" is forbidden: User \"system:node:ip-10-0-143-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-117.ec2.internal' and this object" podUID="425c18cd-881a-4cb8-babb-fd7210fd0e9f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5cl74"
Jul 30 03:45:24.613645 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:45:24.613573    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-5cl74\" is forbidden: User \"system:node:ip-10-0-143-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-117.ec2.internal' and this object" podUID="425c18cd-881a-4cb8-babb-fd7210fd0e9f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5cl74"
Jul 30 03:45:24.619195 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:24.619178    2576 scope.go:117] "RemoveContainer" containerID="381e3198e4ce66d08621652dece56d7768e2530669e5ba13d468ab7e2b76616c"
Jul 30 03:45:24.619469 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:45:24.619444    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"381e3198e4ce66d08621652dece56d7768e2530669e5ba13d468ab7e2b76616c\": container with ID starting with 381e3198e4ce66d08621652dece56d7768e2530669e5ba13d468ab7e2b76616c not found: ID does not exist" containerID="381e3198e4ce66d08621652dece56d7768e2530669e5ba13d468ab7e2b76616c"
Jul 30 03:45:24.619469 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:24.619479    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"381e3198e4ce66d08621652dece56d7768e2530669e5ba13d468ab7e2b76616c"} err="failed to get container status \"381e3198e4ce66d08621652dece56d7768e2530669e5ba13d468ab7e2b76616c\": rpc error: code = NotFound desc = could not find container \"381e3198e4ce66d08621652dece56d7768e2530669e5ba13d468ab7e2b76616c\": container with ID starting with 381e3198e4ce66d08621652dece56d7768e2530669e5ba13d468ab7e2b76616c not found: ID does not exist"
Jul 30 03:45:24.632091 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:24.632053    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pnx7l" podStartSLOduration=12.6320405 podStartE2EDuration="12.6320405s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:45:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:45:24.630965626 +0000 UTC m=+630.249197872" watchObservedRunningTime="2026-07-30 03:45:24.6320405 +0000 UTC m=+630.250272737"
Jul 30 03:45:24.633168 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:45:24.633139    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-5cl74\" is forbidden: User \"system:node:ip-10-0-143-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-117.ec2.internal' and this object" podUID="425c18cd-881a-4cb8-babb-fd7210fd0e9f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5cl74"
Jul 30 03:45:25.050375 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:45:25.050339    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-5cl74\" is forbidden: User \"system:node:ip-10-0-143-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-117.ec2.internal' and this object" podUID="425c18cd-881a-4cb8-babb-fd7210fd0e9f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5cl74"
Jul 30 03:45:25.051093 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:25.051066    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="425c18cd-881a-4cb8-babb-fd7210fd0e9f" path="/var/lib/kubelet/pods/425c18cd-881a-4cb8-babb-fd7210fd0e9f/volumes"
Jul 30 03:45:25.620562 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:25.620521    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk9f7" event={"ID":"6e2fe08c-820d-4da0-91ca-752de17b3bf2","Type":"ContainerStarted","Data":"4c8507f1f4aeb103f1e5d5d92c851416016f92c69099ddaacc167ab65812ed71"}
Jul 30 03:45:25.639017 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:25.638974    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk9f7" podStartSLOduration=2.910445399 podStartE2EDuration="24.638959326s" totalImagesPullingTime="21.728513927s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:45:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:45:25.637478254 +0000 UTC m=+631.255710487" watchObservedRunningTime="2026-07-30 03:45:25.638959326 +0000 UTC m=+631.257191566"
Jul 30 03:45:29.187642 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:29.187607    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pnx7l"]
Jul 30 03:45:29.188192 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:29.187798    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pnx7l" podUID="32965b1f-6bdf-41f5-8a48-831d145e92c0" containerName="manager" containerID="cri-o://746af3122ac7d67b0175f6de96f386a94129c49784f6a4ef97961c62dbb70a69" gracePeriod=10
Jul 30 03:45:29.189705 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:29.189677    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pnx7l"
Jul 30 03:45:29.642309 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:29.642275    2576 generic.go:356] "Generic (PLEG): container finished" podID="32965b1f-6bdf-41f5-8a48-831d145e92c0" containerID="746af3122ac7d67b0175f6de96f386a94129c49784f6a4ef97961c62dbb70a69" exitCode=0
Jul 30 03:45:29.642484 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:29.642347    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pnx7l" event={"ID":"32965b1f-6bdf-41f5-8a48-831d145e92c0","Type":"ContainerDied","Data":"746af3122ac7d67b0175f6de96f386a94129c49784f6a4ef97961c62dbb70a69"}
Jul 30 03:45:29.974886 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:29.974860    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pnx7l"
Jul 30 03:45:30.022976 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:30.022949    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/32965b1f-6bdf-41f5-8a48-831d145e92c0-extensions-socket-volume\") pod \"32965b1f-6bdf-41f5-8a48-831d145e92c0\" (UID: \"32965b1f-6bdf-41f5-8a48-831d145e92c0\") "
Jul 30 03:45:30.023099 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:30.023012    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlwjb\" (UniqueName: \"kubernetes.io/projected/32965b1f-6bdf-41f5-8a48-831d145e92c0-kube-api-access-xlwjb\") pod \"32965b1f-6bdf-41f5-8a48-831d145e92c0\" (UID: \"32965b1f-6bdf-41f5-8a48-831d145e92c0\") "
Jul 30 03:45:30.023326 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:30.023306    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32965b1f-6bdf-41f5-8a48-831d145e92c0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "32965b1f-6bdf-41f5-8a48-831d145e92c0" (UID: "32965b1f-6bdf-41f5-8a48-831d145e92c0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 03:45:30.024909 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:30.024884    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32965b1f-6bdf-41f5-8a48-831d145e92c0-kube-api-access-xlwjb" (OuterVolumeSpecName: "kube-api-access-xlwjb") pod "32965b1f-6bdf-41f5-8a48-831d145e92c0" (UID: "32965b1f-6bdf-41f5-8a48-831d145e92c0"). InnerVolumeSpecName "kube-api-access-xlwjb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:45:30.124205 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:30.124171    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/32965b1f-6bdf-41f5-8a48-831d145e92c0-extensions-socket-volume\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:45:30.124205 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:30.124198    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlwjb\" (UniqueName: \"kubernetes.io/projected/32965b1f-6bdf-41f5-8a48-831d145e92c0-kube-api-access-xlwjb\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:45:30.648374 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:30.648338    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pnx7l" event={"ID":"32965b1f-6bdf-41f5-8a48-831d145e92c0","Type":"ContainerDied","Data":"4695786018e7fcf6b1a5cc96f64833122bf29a6a4726ba0cee821769bc658427"}
Jul 30 03:45:30.648814 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:30.648393    2576 scope.go:117] "RemoveContainer" containerID="746af3122ac7d67b0175f6de96f386a94129c49784f6a4ef97961c62dbb70a69"
Jul 30 03:45:30.648814 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:30.648346    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pnx7l"
Jul 30 03:45:30.669988 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:30.669960    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pnx7l"]
Jul 30 03:45:30.674067 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:30.674047    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pnx7l"]
Jul 30 03:45:30.806467 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:30.806433    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:30.908999 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:30.908906    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:31.050536 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:31.050506    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32965b1f-6bdf-41f5-8a48-831d145e92c0" path="/var/lib/kubelet/pods/32965b1f-6bdf-41f5-8a48-831d145e92c0/volumes"
Jul 30 03:45:32.611682 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:32.611642    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5fdbf99bdb-hw8kc" podUID="a95c6faa-d3b4-4311-83de-3e05bd962148" containerName="console" containerID="cri-o://2eefe7bb57c8448ed99f3b05418c6afd3f7e791209bf61cfa2dea5925267a6b3" gracePeriod=15
Jul 30 03:45:32.853828 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:32.853806    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fdbf99bdb-hw8kc"
Jul 30 03:45:32.945739 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:32.945655    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a95c6faa-d3b4-4311-83de-3e05bd962148-trusted-ca-bundle\") pod \"a95c6faa-d3b4-4311-83de-3e05bd962148\" (UID: \"a95c6faa-d3b4-4311-83de-3e05bd962148\") "
Jul 30 03:45:32.945739 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:32.945697    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a95c6faa-d3b4-4311-83de-3e05bd962148-oauth-serving-cert\") pod \"a95c6faa-d3b4-4311-83de-3e05bd962148\" (UID: \"a95c6faa-d3b4-4311-83de-3e05bd962148\") "
Jul 30 03:45:32.945739 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:32.945725    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a95c6faa-d3b4-4311-83de-3e05bd962148-console-oauth-config\") pod \"a95c6faa-d3b4-4311-83de-3e05bd962148\" (UID: \"a95c6faa-d3b4-4311-83de-3e05bd962148\") "
Jul 30 03:45:32.946037 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:32.945758    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a95c6faa-d3b4-4311-83de-3e05bd962148-console-config\") pod \"a95c6faa-d3b4-4311-83de-3e05bd962148\" (UID: \"a95c6faa-d3b4-4311-83de-3e05bd962148\") "
Jul 30 03:45:32.946037 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:32.945796    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a95c6faa-d3b4-4311-83de-3e05bd962148-console-serving-cert\") pod \"a95c6faa-d3b4-4311-83de-3e05bd962148\" (UID: \"a95c6faa-d3b4-4311-83de-3e05bd962148\") "
Jul 30 03:45:32.946037 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:32.945865    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nsl5r\" (UniqueName: \"kubernetes.io/projected/a95c6faa-d3b4-4311-83de-3e05bd962148-kube-api-access-nsl5r\") pod \"a95c6faa-d3b4-4311-83de-3e05bd962148\" (UID: \"a95c6faa-d3b4-4311-83de-3e05bd962148\") "
Jul 30 03:45:32.946037 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:32.945888    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a95c6faa-d3b4-4311-83de-3e05bd962148-service-ca\") pod \"a95c6faa-d3b4-4311-83de-3e05bd962148\" (UID: \"a95c6faa-d3b4-4311-83de-3e05bd962148\") "
Jul 30 03:45:32.946246 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:32.946201    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a95c6faa-d3b4-4311-83de-3e05bd962148-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a95c6faa-d3b4-4311-83de-3e05bd962148" (UID: "a95c6faa-d3b4-4311-83de-3e05bd962148"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 03:45:32.946246 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:32.946104    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a95c6faa-d3b4-4311-83de-3e05bd962148-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a95c6faa-d3b4-4311-83de-3e05bd962148" (UID: "a95c6faa-d3b4-4311-83de-3e05bd962148"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 03:45:32.946359 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:32.946331    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a95c6faa-d3b4-4311-83de-3e05bd962148-console-config" (OuterVolumeSpecName: "console-config") pod "a95c6faa-d3b4-4311-83de-3e05bd962148" (UID: "a95c6faa-d3b4-4311-83de-3e05bd962148"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 03:45:32.946428 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:32.946409    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a95c6faa-d3b4-4311-83de-3e05bd962148-service-ca" (OuterVolumeSpecName: "service-ca") pod "a95c6faa-d3b4-4311-83de-3e05bd962148" (UID: "a95c6faa-d3b4-4311-83de-3e05bd962148"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 03:45:32.948060 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:32.948020    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a95c6faa-d3b4-4311-83de-3e05bd962148-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a95c6faa-d3b4-4311-83de-3e05bd962148" (UID: "a95c6faa-d3b4-4311-83de-3e05bd962148"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 03:45:32.948194 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:32.948080    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a95c6faa-d3b4-4311-83de-3e05bd962148-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a95c6faa-d3b4-4311-83de-3e05bd962148" (UID: "a95c6faa-d3b4-4311-83de-3e05bd962148"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 03:45:32.948194 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:32.948097    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a95c6faa-d3b4-4311-83de-3e05bd962148-kube-api-access-nsl5r" (OuterVolumeSpecName: "kube-api-access-nsl5r") pod "a95c6faa-d3b4-4311-83de-3e05bd962148" (UID: "a95c6faa-d3b4-4311-83de-3e05bd962148"). InnerVolumeSpecName "kube-api-access-nsl5r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:45:33.047170 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:33.047136    2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a95c6faa-d3b4-4311-83de-3e05bd962148-trusted-ca-bundle\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:45:33.047170 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:33.047169    2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a95c6faa-d3b4-4311-83de-3e05bd962148-oauth-serving-cert\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:45:33.047350 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:33.047189    2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a95c6faa-d3b4-4311-83de-3e05bd962148-console-oauth-config\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:45:33.047350 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:33.047210    2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a95c6faa-d3b4-4311-83de-3e05bd962148-console-config\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:45:33.047350 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:33.047230    2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a95c6faa-d3b4-4311-83de-3e05bd962148-console-serving-cert\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:45:33.047350 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:33.047248    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nsl5r\" (UniqueName: \"kubernetes.io/projected/a95c6faa-d3b4-4311-83de-3e05bd962148-kube-api-access-nsl5r\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:45:33.047350 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:33.047269    2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a95c6faa-d3b4-4311-83de-3e05bd962148-service-ca\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:45:33.243163 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:33.243085    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:33.296891 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:33.296862    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:33.664730 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:33.664695    2576 generic.go:356] "Generic (PLEG): container finished" podID="a95c6faa-d3b4-4311-83de-3e05bd962148" containerID="2eefe7bb57c8448ed99f3b05418c6afd3f7e791209bf61cfa2dea5925267a6b3" exitCode=0
Jul 30 03:45:33.665148 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:33.664740    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fdbf99bdb-hw8kc"
Jul 30 03:45:33.665148 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:33.664783    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fdbf99bdb-hw8kc" event={"ID":"a95c6faa-d3b4-4311-83de-3e05bd962148","Type":"ContainerDied","Data":"2eefe7bb57c8448ed99f3b05418c6afd3f7e791209bf61cfa2dea5925267a6b3"}
Jul 30 03:45:33.665148 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:33.664835    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fdbf99bdb-hw8kc" event={"ID":"a95c6faa-d3b4-4311-83de-3e05bd962148","Type":"ContainerDied","Data":"93ade8d4a0eb1f8f259d7251544d8cbf5ecb96be7df375c99405b54792d33e30"}
Jul 30 03:45:33.665148 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:33.664862    2576 scope.go:117] "RemoveContainer" containerID="2eefe7bb57c8448ed99f3b05418c6afd3f7e791209bf61cfa2dea5925267a6b3"
Jul 30 03:45:33.673088 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:33.673074    2576 scope.go:117] "RemoveContainer" containerID="2eefe7bb57c8448ed99f3b05418c6afd3f7e791209bf61cfa2dea5925267a6b3"
Jul 30 03:45:33.673312 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:45:33.673288    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2eefe7bb57c8448ed99f3b05418c6afd3f7e791209bf61cfa2dea5925267a6b3\": container with ID starting with 2eefe7bb57c8448ed99f3b05418c6afd3f7e791209bf61cfa2dea5925267a6b3 not found: ID does not exist" containerID="2eefe7bb57c8448ed99f3b05418c6afd3f7e791209bf61cfa2dea5925267a6b3"
Jul 30 03:45:33.673371 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:33.673313    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eefe7bb57c8448ed99f3b05418c6afd3f7e791209bf61cfa2dea5925267a6b3"} err="failed to get container status \"2eefe7bb57c8448ed99f3b05418c6afd3f7e791209bf61cfa2dea5925267a6b3\": rpc error: code = NotFound desc = could not find container \"2eefe7bb57c8448ed99f3b05418c6afd3f7e791209bf61cfa2dea5925267a6b3\": container with ID starting with 2eefe7bb57c8448ed99f3b05418c6afd3f7e791209bf61cfa2dea5925267a6b3 not found: ID does not exist"
Jul 30 03:45:33.682121 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:33.682100    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5fdbf99bdb-hw8kc"]
Jul 30 03:45:33.686103 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:33.686083    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5fdbf99bdb-hw8kc"]
Jul 30 03:45:35.051395 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:35.051363    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a95c6faa-d3b4-4311-83de-3e05bd962148" path="/var/lib/kubelet/pods/a95c6faa-d3b4-4311-83de-3e05bd962148/volumes"
Jul 30 03:45:37.995973 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:37.995941    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:38.096687 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:38.096661    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:42.003409 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:42.003378    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:42.102698 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:42.102664    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:45.445477 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.445436    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"]
Jul 30 03:45:45.445821 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.445759    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32965b1f-6bdf-41f5-8a48-831d145e92c0" containerName="manager"
Jul 30 03:45:45.445821 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.445770    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="32965b1f-6bdf-41f5-8a48-831d145e92c0" containerName="manager"
Jul 30 03:45:45.445821 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.445787    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a95c6faa-d3b4-4311-83de-3e05bd962148" containerName="console"
Jul 30 03:45:45.445821 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.445792    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a95c6faa-d3b4-4311-83de-3e05bd962148" containerName="console"
Jul 30 03:45:45.445994 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.445838    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a95c6faa-d3b4-4311-83de-3e05bd962148" containerName="console"
Jul 30 03:45:45.445994 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.445844    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="32965b1f-6bdf-41f5-8a48-831d145e92c0" containerName="manager"
Jul 30 03:45:45.448682 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.448662    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"
Jul 30 03:45:45.450964 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.450944    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-dzq2k\""
Jul 30 03:45:45.462172 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.462147    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"]
Jul 30 03:45:45.476669 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.476645    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:45.548215 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.548187    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60518678-0996-4b4f-a74c-1221fecbedcc-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zvgfm\" (UID: \"60518678-0996-4b4f-a74c-1221fecbedcc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"
Jul 30 03:45:45.548389 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.548228    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgqnm\" (UniqueName: \"kubernetes.io/projected/60518678-0996-4b4f-a74c-1221fecbedcc-kube-api-access-qgqnm\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zvgfm\" (UID: \"60518678-0996-4b4f-a74c-1221fecbedcc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"
Jul 30 03:45:45.548389 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.548257    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60518678-0996-4b4f-a74c-1221fecbedcc-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zvgfm\" (UID: \"60518678-0996-4b4f-a74c-1221fecbedcc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"
Jul 30 03:45:45.548389 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.548336    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60518678-0996-4b4f-a74c-1221fecbedcc-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zvgfm\" (UID: \"60518678-0996-4b4f-a74c-1221fecbedcc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"
Jul 30 03:45:45.548499 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.548424    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60518678-0996-4b4f-a74c-1221fecbedcc-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zvgfm\" (UID: \"60518678-0996-4b4f-a74c-1221fecbedcc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"
Jul 30 03:45:45.548499 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.548458    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60518678-0996-4b4f-a74c-1221fecbedcc-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zvgfm\" (UID: \"60518678-0996-4b4f-a74c-1221fecbedcc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"
Jul 30 03:45:45.548499 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.548478    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60518678-0996-4b4f-a74c-1221fecbedcc-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zvgfm\" (UID: \"60518678-0996-4b4f-a74c-1221fecbedcc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"
Jul 30 03:45:45.548598 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.548513    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60518678-0996-4b4f-a74c-1221fecbedcc-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zvgfm\" (UID: \"60518678-0996-4b4f-a74c-1221fecbedcc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"
Jul 30 03:45:45.548598 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.548549    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60518678-0996-4b4f-a74c-1221fecbedcc-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zvgfm\" (UID: \"60518678-0996-4b4f-a74c-1221fecbedcc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"
Jul 30 03:45:45.649093 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.649063    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60518678-0996-4b4f-a74c-1221fecbedcc-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zvgfm\" (UID: \"60518678-0996-4b4f-a74c-1221fecbedcc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"
Jul 30 03:45:45.649093 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.649097    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60518678-0996-4b4f-a74c-1221fecbedcc-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zvgfm\" (UID: \"60518678-0996-4b4f-a74c-1221fecbedcc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"
Jul 30 03:45:45.649303 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.649119    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60518678-0996-4b4f-a74c-1221fecbedcc-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zvgfm\" (UID: \"60518678-0996-4b4f-a74c-1221fecbedcc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"
Jul 30 03:45:45.649303 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.649139    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60518678-0996-4b4f-a74c-1221fecbedcc-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zvgfm\" (UID: \"60518678-0996-4b4f-a74c-1221fecbedcc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"
Jul 30 03:45:45.649303 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.649177    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgqnm\" (UniqueName: \"kubernetes.io/projected/60518678-0996-4b4f-a74c-1221fecbedcc-kube-api-access-qgqnm\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zvgfm\" (UID: \"60518678-0996-4b4f-a74c-1221fecbedcc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"
Jul 30 03:45:45.649303 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.649207    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60518678-0996-4b4f-a74c-1221fecbedcc-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zvgfm\" (UID: \"60518678-0996-4b4f-a74c-1221fecbedcc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"
Jul 30 03:45:45.649303 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.649234    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60518678-0996-4b4f-a74c-1221fecbedcc-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zvgfm\" (UID: \"60518678-0996-4b4f-a74c-1221fecbedcc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"
Jul 30 03:45:45.649303 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.649278    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60518678-0996-4b4f-a74c-1221fecbedcc-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zvgfm\" (UID: \"60518678-0996-4b4f-a74c-1221fecbedcc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"
Jul 30 03:45:45.649629 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.649309    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60518678-0996-4b4f-a74c-1221fecbedcc-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zvgfm\" (UID: \"60518678-0996-4b4f-a74c-1221fecbedcc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"
Jul 30 03:45:45.649690 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.649657    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60518678-0996-4b4f-a74c-1221fecbedcc-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zvgfm\" (UID: \"60518678-0996-4b4f-a74c-1221fecbedcc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"
Jul 30 03:45:45.649742 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.649724    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60518678-0996-4b4f-a74c-1221fecbedcc-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zvgfm\" (UID: \"60518678-0996-4b4f-a74c-1221fecbedcc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"
Jul 30 03:45:45.649799 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.649765    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60518678-0996-4b4f-a74c-1221fecbedcc-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zvgfm\" (UID: \"60518678-0996-4b4f-a74c-1221fecbedcc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"
Jul 30 03:45:45.649975 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.649952    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60518678-0996-4b4f-a74c-1221fecbedcc-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zvgfm\" (UID: \"60518678-0996-4b4f-a74c-1221fecbedcc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"
Jul 30 03:45:45.650087 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.650067    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60518678-0996-4b4f-a74c-1221fecbedcc-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zvgfm\" (UID: \"60518678-0996-4b4f-a74c-1221fecbedcc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"
Jul 30 03:45:45.651393 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.651368    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60518678-0996-4b4f-a74c-1221fecbedcc-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zvgfm\" (UID: \"60518678-0996-4b4f-a74c-1221fecbedcc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"
Jul 30 03:45:45.651621 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.651604    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60518678-0996-4b4f-a74c-1221fecbedcc-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zvgfm\" (UID: \"60518678-0996-4b4f-a74c-1221fecbedcc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"
Jul 30 03:45:45.658607 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.658585    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgqnm\" (UniqueName: \"kubernetes.io/projected/60518678-0996-4b4f-a74c-1221fecbedcc-kube-api-access-qgqnm\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zvgfm\" (UID: \"60518678-0996-4b4f-a74c-1221fecbedcc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"
Jul 30 03:45:45.658730 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.658670    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60518678-0996-4b4f-a74c-1221fecbedcc-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zvgfm\" (UID: \"60518678-0996-4b4f-a74c-1221fecbedcc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"
Jul 30 03:45:45.758346 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.758284    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"
Jul 30 03:45:45.884872 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.884842    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:45.896429 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.896409    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"]
Jul 30 03:45:45.898051 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:45:45.898027    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60518678_0996_4b4f_a74c_1221fecbedcc.slice/crio-3681423a5038f4617950b6ca9740881de071cec6d6fa92ed575c57f2d24ecbf0 WatchSource:0}: Error finding container 3681423a5038f4617950b6ca9740881de071cec6d6fa92ed575c57f2d24ecbf0: Status 404 returned error can't find the container with id 3681423a5038f4617950b6ca9740881de071cec6d6fa92ed575c57f2d24ecbf0
Jul 30 03:45:45.900215 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.900186    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 03:45:45.900274 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.900252    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 03:45:45.900308 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:45.900283    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 03:45:46.730934 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:46.730898    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm" event={"ID":"60518678-0996-4b4f-a74c-1221fecbedcc","Type":"ContainerStarted","Data":"d0d24cb7ad604cdbd008169efdcf22401a2b8494dc4c7ba7965128599712887b"}
Jul 30 03:45:46.731393 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:46.730971    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm" event={"ID":"60518678-0996-4b4f-a74c-1221fecbedcc","Type":"ContainerStarted","Data":"3681423a5038f4617950b6ca9740881de071cec6d6fa92ed575c57f2d24ecbf0"}
Jul 30 03:45:46.750236 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:46.750195    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm" podStartSLOduration=1.7501814329999998 podStartE2EDuration="1.750181433s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:45:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:45:46.749540236 +0000 UTC m=+652.367772477" watchObservedRunningTime="2026-07-30 03:45:46.750181433 +0000 UTC m=+652.368413674"
Jul 30 03:45:46.758658 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:46.758635    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"
Jul 30 03:45:46.767762 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:46.767739    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"
Jul 30 03:45:47.736270 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:47.736235    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"
Jul 30 03:45:47.737098 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:47.737082    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"
Jul 30 03:45:47.746505 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:47.746480    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:47.757759 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:47.757732    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-7bkdz"]
Jul 30 03:45:47.803513 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:47.803486    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-7bkdz"]
Jul 30 03:45:47.803657 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:47.803584    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-7bkdz"
Jul 30 03:45:47.813085 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:47.813062    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 30 03:45:47.852508 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:47.852476    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-7bkdz"]
Jul 30 03:45:47.866359 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:47.866333    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:47.937470 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:47.937430    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 03:45:47.937621 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:47.937548    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:47.939652 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:47.939639    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-2ml29"
Jul 30 03:45:47.948180 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:47.948160    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 03:45:47.971309 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:47.971272    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0af99020-442e-41bd-9f8a-ac75c3df4e51-config-file\") pod \"limitador-limitador-64c94fd64f-7bkdz\" (UID: \"0af99020-442e-41bd-9f8a-ac75c3df4e51\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-7bkdz"
Jul 30 03:45:47.971490 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:47.971454    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78hml\" (UniqueName: \"kubernetes.io/projected/0af99020-442e-41bd-9f8a-ac75c3df4e51-kube-api-access-78hml\") pod \"limitador-limitador-64c94fd64f-7bkdz\" (UID: \"0af99020-442e-41bd-9f8a-ac75c3df4e51\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-7bkdz"
Jul 30 03:45:47.987360 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:47.987296    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 03:45:48.072080 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:48.072049    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78hml\" (UniqueName: \"kubernetes.io/projected/0af99020-442e-41bd-9f8a-ac75c3df4e51-kube-api-access-78hml\") pod \"limitador-limitador-64c94fd64f-7bkdz\" (UID: \"0af99020-442e-41bd-9f8a-ac75c3df4e51\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-7bkdz"
Jul 30 03:45:48.072221 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:48.072093    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rfrn\" (UniqueName: \"kubernetes.io/projected/e2d0eef0-b840-4496-aa36-e6a3fa885d70-kube-api-access-6rfrn\") pod \"limitador-limitador-54d6d988d5-2ml29\" (UID: \"e2d0eef0-b840-4496-aa36-e6a3fa885d70\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2ml29"
Jul 30 03:45:48.072221 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:48.072124    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0af99020-442e-41bd-9f8a-ac75c3df4e51-config-file\") pod \"limitador-limitador-64c94fd64f-7bkdz\" (UID: \"0af99020-442e-41bd-9f8a-ac75c3df4e51\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-7bkdz"
Jul 30 03:45:48.072221 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:48.072162    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e2d0eef0-b840-4496-aa36-e6a3fa885d70-config-file\") pod \"limitador-limitador-54d6d988d5-2ml29\" (UID: \"e2d0eef0-b840-4496-aa36-e6a3fa885d70\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2ml29"
Jul 30 03:45:48.072769 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:48.072752    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0af99020-442e-41bd-9f8a-ac75c3df4e51-config-file\") pod \"limitador-limitador-64c94fd64f-7bkdz\" (UID: \"0af99020-442e-41bd-9f8a-ac75c3df4e51\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-7bkdz"
Jul 30 03:45:48.080681 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:48.080659    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78hml\" (UniqueName: \"kubernetes.io/projected/0af99020-442e-41bd-9f8a-ac75c3df4e51-kube-api-access-78hml\") pod \"limitador-limitador-64c94fd64f-7bkdz\" (UID: \"0af99020-442e-41bd-9f8a-ac75c3df4e51\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-7bkdz"
Jul 30 03:45:48.114660 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:48.114634    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-7bkdz"
Jul 30 03:45:48.173267 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:48.173221    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e2d0eef0-b840-4496-aa36-e6a3fa885d70-config-file\") pod \"limitador-limitador-54d6d988d5-2ml29\" (UID: \"e2d0eef0-b840-4496-aa36-e6a3fa885d70\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2ml29"
Jul 30 03:45:48.173425 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:48.173320    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rfrn\" (UniqueName: \"kubernetes.io/projected/e2d0eef0-b840-4496-aa36-e6a3fa885d70-kube-api-access-6rfrn\") pod \"limitador-limitador-54d6d988d5-2ml29\" (UID: \"e2d0eef0-b840-4496-aa36-e6a3fa885d70\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2ml29"
Jul 30 03:45:48.174040 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:48.174016    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e2d0eef0-b840-4496-aa36-e6a3fa885d70-config-file\") pod \"limitador-limitador-54d6d988d5-2ml29\" (UID: \"e2d0eef0-b840-4496-aa36-e6a3fa885d70\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2ml29"
Jul 30 03:45:48.181348 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:48.181320    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rfrn\" (UniqueName: \"kubernetes.io/projected/e2d0eef0-b840-4496-aa36-e6a3fa885d70-kube-api-access-6rfrn\") pod \"limitador-limitador-54d6d988d5-2ml29\" (UID: \"e2d0eef0-b840-4496-aa36-e6a3fa885d70\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2ml29"
Jul 30 03:45:48.242553 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:48.242526    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-7bkdz"]
Jul 30 03:45:48.244330 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:45:48.244300    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0af99020_442e_41bd_9f8a_ac75c3df4e51.slice/crio-1f7f80cc7dc7e783e86385a3e1addffc88be5939c54e0469d52ec6a2a6612be5 WatchSource:0}: Error finding container 1f7f80cc7dc7e783e86385a3e1addffc88be5939c54e0469d52ec6a2a6612be5: Status 404 returned error can't find the container with id 1f7f80cc7dc7e783e86385a3e1addffc88be5939c54e0469d52ec6a2a6612be5
Jul 30 03:45:48.249907 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:48.249868    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-2ml29"
Jul 30 03:45:48.377981 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:48.377959    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 03:45:48.379340 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:45:48.379318    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2d0eef0_b840_4496_aa36_e6a3fa885d70.slice/crio-a2435f49f766be3f698f666a490fb5878117fef1859652e5b88a937b8dd30b9e WatchSource:0}: Error finding container a2435f49f766be3f698f666a490fb5878117fef1859652e5b88a937b8dd30b9e: Status 404 returned error can't find the container with id a2435f49f766be3f698f666a490fb5878117fef1859652e5b88a937b8dd30b9e
Jul 30 03:45:48.543872 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:48.543846    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-27gps"]
Jul 30 03:45:48.546731 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:48.546715    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-27gps"
Jul 30 03:45:48.549402 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:48.549380    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-v88vv\""
Jul 30 03:45:48.558314 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:48.558294    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-27gps"]
Jul 30 03:45:48.574798 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:48.574774    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:48.647455 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:48.647424    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:48.677826 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:48.677799    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhmxh\" (UniqueName: \"kubernetes.io/projected/a4606768-ed42-4b2d-87a1-8688566b8a12-kube-api-access-bhmxh\") pod \"authorino-58cd955675-27gps\" (UID: \"a4606768-ed42-4b2d-87a1-8688566b8a12\") " pod="kuadrant-system/authorino-58cd955675-27gps"
Jul 30 03:45:48.725981 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:48.725948    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-r9g82"]
Jul 30 03:45:48.728480 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:48.728465    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-r9g82"
Jul 30 03:45:48.737088 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:48.737060    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-r9g82"]
Jul 30 03:45:48.744183 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:48.744155    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-2ml29" event={"ID":"e2d0eef0-b840-4496-aa36-e6a3fa885d70","Type":"ContainerStarted","Data":"a2435f49f766be3f698f666a490fb5878117fef1859652e5b88a937b8dd30b9e"}
Jul 30 03:45:48.745697 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:48.745676    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-7bkdz" event={"ID":"0af99020-442e-41bd-9f8a-ac75c3df4e51","Type":"ContainerStarted","Data":"1f7f80cc7dc7e783e86385a3e1addffc88be5939c54e0469d52ec6a2a6612be5"}
Jul 30 03:45:48.778572 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:48.778545    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhmxh\" (UniqueName: \"kubernetes.io/projected/a4606768-ed42-4b2d-87a1-8688566b8a12-kube-api-access-bhmxh\") pod \"authorino-58cd955675-27gps\" (UID: \"a4606768-ed42-4b2d-87a1-8688566b8a12\") " pod="kuadrant-system/authorino-58cd955675-27gps"
Jul 30 03:45:48.786442 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:48.786413    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhmxh\" (UniqueName: \"kubernetes.io/projected/a4606768-ed42-4b2d-87a1-8688566b8a12-kube-api-access-bhmxh\") pod \"authorino-58cd955675-27gps\" (UID: \"a4606768-ed42-4b2d-87a1-8688566b8a12\") " pod="kuadrant-system/authorino-58cd955675-27gps"
Jul 30 03:45:48.855664 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:48.855603    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-27gps"
Jul 30 03:45:48.879784 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:48.879753    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5kz4\" (UniqueName: \"kubernetes.io/projected/62e46aa2-c703-47f5-b604-e5c3c2fa29b5-kube-api-access-g5kz4\") pod \"authorino-75d556bd77-r9g82\" (UID: \"62e46aa2-c703-47f5-b604-e5c3c2fa29b5\") " pod="kuadrant-system/authorino-75d556bd77-r9g82"
Jul 30 03:45:48.981207 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:48.980977    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5kz4\" (UniqueName: \"kubernetes.io/projected/62e46aa2-c703-47f5-b604-e5c3c2fa29b5-kube-api-access-g5kz4\") pod \"authorino-75d556bd77-r9g82\" (UID: \"62e46aa2-c703-47f5-b604-e5c3c2fa29b5\") " pod="kuadrant-system/authorino-75d556bd77-r9g82"
Jul 30 03:45:48.992462 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:48.992425    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5kz4\" (UniqueName: \"kubernetes.io/projected/62e46aa2-c703-47f5-b604-e5c3c2fa29b5-kube-api-access-g5kz4\") pod \"authorino-75d556bd77-r9g82\" (UID: \"62e46aa2-c703-47f5-b604-e5c3c2fa29b5\") " pod="kuadrant-system/authorino-75d556bd77-r9g82"
Jul 30 03:45:49.026321 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:49.026282    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-27gps"]
Jul 30 03:45:49.028196 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:45:49.028168    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda4606768_ed42_4b2d_87a1_8688566b8a12.slice/crio-4817bf42126663c95a9a5fa28f304bcfa606c51d59d497620ec87e027524e782 WatchSource:0}: Error finding container 4817bf42126663c95a9a5fa28f304bcfa606c51d59d497620ec87e027524e782: Status 404 returned error can't find the container with id 4817bf42126663c95a9a5fa28f304bcfa606c51d59d497620ec87e027524e782
Jul 30 03:45:49.038169 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:49.038105    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-r9g82"
Jul 30 03:45:49.246480 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:49.246447    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-r9g82"]
Jul 30 03:45:49.754699 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:49.754661    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-r9g82" event={"ID":"62e46aa2-c703-47f5-b604-e5c3c2fa29b5","Type":"ContainerStarted","Data":"d514ff25cb89d6d3dc6fdc6b85f24579c6251bf57f490416608ac9d66518e302"}
Jul 30 03:45:49.756285 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:49.756184    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-27gps" event={"ID":"a4606768-ed42-4b2d-87a1-8688566b8a12","Type":"ContainerStarted","Data":"4817bf42126663c95a9a5fa28f304bcfa606c51d59d497620ec87e027524e782"}
Jul 30 03:45:51.015083 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:51.015050    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:51.115963 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:51.115870    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:53.462989 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:53.462953    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:53.564538 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:53.564500    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:54.793713 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:54.793682    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-7bkdz" event={"ID":"0af99020-442e-41bd-9f8a-ac75c3df4e51","Type":"ContainerStarted","Data":"524ab98a2e2e0103148bc0d82d9e0377c70bc45927c1d2eec93c987ea0369f73"}
Jul 30 03:45:54.794209 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:54.793796    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-7bkdz"
Jul 30 03:45:54.795362 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:54.795338    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-r9g82" event={"ID":"62e46aa2-c703-47f5-b604-e5c3c2fa29b5","Type":"ContainerStarted","Data":"190b238c7698b1539ab776192348c60b690a9df8895c0f6e006cdd30831af98e"}
Jul 30 03:45:54.796621 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:54.796599    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-27gps" event={"ID":"a4606768-ed42-4b2d-87a1-8688566b8a12","Type":"ContainerStarted","Data":"9d0c08808a98ba4225bd02726a5c6535a6010c0d49db4af9eb61937cfb0552c1"}
Jul 30 03:45:54.798177 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:54.798155    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-2ml29" event={"ID":"e2d0eef0-b840-4496-aa36-e6a3fa885d70","Type":"ContainerStarted","Data":"e43b46db76e351518972afb21c3612482c241c2444783224fa948ce7058309c8"}
Jul 30 03:45:54.798331 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:54.798317    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-2ml29"
Jul 30 03:45:54.810344 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:54.810307    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-7bkdz" podStartSLOduration=1.8275719320000001 podStartE2EDuration="7.810295493s" totalImagesPullingTime="5.982723561s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:45:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:45:54.809604139 +0000 UTC m=+660.427836380" watchObservedRunningTime="2026-07-30 03:45:54.810295493 +0000 UTC m=+660.428527733"
Jul 30 03:45:54.829144 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:54.829101    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-2ml29" podStartSLOduration=1.970789039 podStartE2EDuration="7.829085266s" totalImagesPullingTime="5.858296227s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:45:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:45:54.827263311 +0000 UTC m=+660.445495552" watchObservedRunningTime="2026-07-30 03:45:54.829085266 +0000 UTC m=+660.447317507"
Jul 30 03:45:54.845427 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:54.845389    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-27gps" podStartSLOduration=1.645015498 podStartE2EDuration="6.845375841s" totalImagesPullingTime="5.200360343s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:45:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:45:54.843389634 +0000 UTC m=+660.461621876" watchObservedRunningTime="2026-07-30 03:45:54.845375841 +0000 UTC m=+660.463608134"
Jul 30 03:45:54.858522 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:54.858477    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-r9g82" podStartSLOduration=1.8787301699999999 podStartE2EDuration="6.858459128s" totalImagesPullingTime="4.979728958s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:45:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:45:54.857521901 +0000 UTC m=+660.475754147" watchObservedRunningTime="2026-07-30 03:45:54.858459128 +0000 UTC m=+660.476691373"
Jul 30 03:45:54.883496 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:54.883470    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-27gps"]
Jul 30 03:45:56.808564 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:56.808527    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-27gps" podUID="a4606768-ed42-4b2d-87a1-8688566b8a12" containerName="authorino" containerID="cri-o://9d0c08808a98ba4225bd02726a5c6535a6010c0d49db4af9eb61937cfb0552c1" gracePeriod=30
Jul 30 03:45:57.056454 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:57.056429    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-27gps"
Jul 30 03:45:57.157480 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:57.157386    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bhmxh\" (UniqueName: \"kubernetes.io/projected/a4606768-ed42-4b2d-87a1-8688566b8a12-kube-api-access-bhmxh\") pod \"a4606768-ed42-4b2d-87a1-8688566b8a12\" (UID: \"a4606768-ed42-4b2d-87a1-8688566b8a12\") "
Jul 30 03:45:57.159406 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:57.159375    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4606768-ed42-4b2d-87a1-8688566b8a12-kube-api-access-bhmxh" (OuterVolumeSpecName: "kube-api-access-bhmxh") pod "a4606768-ed42-4b2d-87a1-8688566b8a12" (UID: "a4606768-ed42-4b2d-87a1-8688566b8a12"). InnerVolumeSpecName "kube-api-access-bhmxh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:45:57.258650 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:57.258610    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhmxh\" (UniqueName: \"kubernetes.io/projected/a4606768-ed42-4b2d-87a1-8688566b8a12-kube-api-access-bhmxh\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:45:57.814503 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:57.814465    2576 generic.go:356] "Generic (PLEG): container finished" podID="a4606768-ed42-4b2d-87a1-8688566b8a12" containerID="9d0c08808a98ba4225bd02726a5c6535a6010c0d49db4af9eb61937cfb0552c1" exitCode=0
Jul 30 03:45:57.814904 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:57.814523    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-27gps"
Jul 30 03:45:57.814904 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:57.814548    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-27gps" event={"ID":"a4606768-ed42-4b2d-87a1-8688566b8a12","Type":"ContainerDied","Data":"9d0c08808a98ba4225bd02726a5c6535a6010c0d49db4af9eb61937cfb0552c1"}
Jul 30 03:45:57.814904 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:57.814593    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-27gps" event={"ID":"a4606768-ed42-4b2d-87a1-8688566b8a12","Type":"ContainerDied","Data":"4817bf42126663c95a9a5fa28f304bcfa606c51d59d497620ec87e027524e782"}
Jul 30 03:45:57.814904 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:57.814609    2576 scope.go:117] "RemoveContainer" containerID="9d0c08808a98ba4225bd02726a5c6535a6010c0d49db4af9eb61937cfb0552c1"
Jul 30 03:45:57.823606 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:57.823591    2576 scope.go:117] "RemoveContainer" containerID="9d0c08808a98ba4225bd02726a5c6535a6010c0d49db4af9eb61937cfb0552c1"
Jul 30 03:45:57.823838 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:45:57.823821    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d0c08808a98ba4225bd02726a5c6535a6010c0d49db4af9eb61937cfb0552c1\": container with ID starting with 9d0c08808a98ba4225bd02726a5c6535a6010c0d49db4af9eb61937cfb0552c1 not found: ID does not exist" containerID="9d0c08808a98ba4225bd02726a5c6535a6010c0d49db4af9eb61937cfb0552c1"
Jul 30 03:45:57.823897 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:57.823844    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d0c08808a98ba4225bd02726a5c6535a6010c0d49db4af9eb61937cfb0552c1"} err="failed to get container status \"9d0c08808a98ba4225bd02726a5c6535a6010c0d49db4af9eb61937cfb0552c1\": rpc error: code = NotFound desc = could not find container \"9d0c08808a98ba4225bd02726a5c6535a6010c0d49db4af9eb61937cfb0552c1\": container with ID starting with 9d0c08808a98ba4225bd02726a5c6535a6010c0d49db4af9eb61937cfb0552c1 not found: ID does not exist"
Jul 30 03:45:57.835417 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:57.835387    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-27gps"]
Jul 30 03:45:57.840366 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:57.840346    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-27gps"]
Jul 30 03:45:58.297358 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:58.297317    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:58.398344 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:58.398314    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:45:59.050659 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:45:59.050630    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4606768-ed42-4b2d-87a1-8688566b8a12" path="/var/lib/kubelet/pods/a4606768-ed42-4b2d-87a1-8688566b8a12/volumes"
Jul 30 03:46:02.277881 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:02.277852    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:05.805682 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:05.805648    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-2ml29"
Jul 30 03:46:05.806203 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:05.805723    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-7bkdz"
Jul 30 03:46:05.872454 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:05.872426    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-7bkdz"]
Jul 30 03:46:05.872646 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:05.872602    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-7bkdz" podUID="0af99020-442e-41bd-9f8a-ac75c3df4e51" containerName="limitador" containerID="cri-o://524ab98a2e2e0103148bc0d82d9e0377c70bc45927c1d2eec93c987ea0369f73" gracePeriod=30
Jul 30 03:46:06.425602 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:06.425580    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-7bkdz"
Jul 30 03:46:06.536638 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:06.536616    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0af99020-442e-41bd-9f8a-ac75c3df4e51-config-file\") pod \"0af99020-442e-41bd-9f8a-ac75c3df4e51\" (UID: \"0af99020-442e-41bd-9f8a-ac75c3df4e51\") "
Jul 30 03:46:06.536804 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:06.536666    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-78hml\" (UniqueName: \"kubernetes.io/projected/0af99020-442e-41bd-9f8a-ac75c3df4e51-kube-api-access-78hml\") pod \"0af99020-442e-41bd-9f8a-ac75c3df4e51\" (UID: \"0af99020-442e-41bd-9f8a-ac75c3df4e51\") "
Jul 30 03:46:06.537034 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:06.537005    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0af99020-442e-41bd-9f8a-ac75c3df4e51-config-file" (OuterVolumeSpecName: "config-file") pod "0af99020-442e-41bd-9f8a-ac75c3df4e51" (UID: "0af99020-442e-41bd-9f8a-ac75c3df4e51"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 03:46:06.538704 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:06.538684    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0af99020-442e-41bd-9f8a-ac75c3df4e51-kube-api-access-78hml" (OuterVolumeSpecName: "kube-api-access-78hml") pod "0af99020-442e-41bd-9f8a-ac75c3df4e51" (UID: "0af99020-442e-41bd-9f8a-ac75c3df4e51"). InnerVolumeSpecName "kube-api-access-78hml". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:46:06.637287 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:06.637252    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-78hml\" (UniqueName: \"kubernetes.io/projected/0af99020-442e-41bd-9f8a-ac75c3df4e51-kube-api-access-78hml\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:46:06.637287 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:06.637280    2576 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0af99020-442e-41bd-9f8a-ac75c3df4e51-config-file\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:46:06.861762 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:06.861679    2576 generic.go:356] "Generic (PLEG): container finished" podID="0af99020-442e-41bd-9f8a-ac75c3df4e51" containerID="524ab98a2e2e0103148bc0d82d9e0377c70bc45927c1d2eec93c987ea0369f73" exitCode=0
Jul 30 03:46:06.861762 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:06.861719    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-7bkdz"
Jul 30 03:46:06.861762 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:06.861747    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-7bkdz" event={"ID":"0af99020-442e-41bd-9f8a-ac75c3df4e51","Type":"ContainerDied","Data":"524ab98a2e2e0103148bc0d82d9e0377c70bc45927c1d2eec93c987ea0369f73"}
Jul 30 03:46:06.862309 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:06.861787    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-7bkdz" event={"ID":"0af99020-442e-41bd-9f8a-ac75c3df4e51","Type":"ContainerDied","Data":"1f7f80cc7dc7e783e86385a3e1addffc88be5939c54e0469d52ec6a2a6612be5"}
Jul 30 03:46:06.862309 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:06.861810    2576 scope.go:117] "RemoveContainer" containerID="524ab98a2e2e0103148bc0d82d9e0377c70bc45927c1d2eec93c987ea0369f73"
Jul 30 03:46:06.870699 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:06.870680    2576 scope.go:117] "RemoveContainer" containerID="524ab98a2e2e0103148bc0d82d9e0377c70bc45927c1d2eec93c987ea0369f73"
Jul 30 03:46:06.870967 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:46:06.870946    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"524ab98a2e2e0103148bc0d82d9e0377c70bc45927c1d2eec93c987ea0369f73\": container with ID starting with 524ab98a2e2e0103148bc0d82d9e0377c70bc45927c1d2eec93c987ea0369f73 not found: ID does not exist" containerID="524ab98a2e2e0103148bc0d82d9e0377c70bc45927c1d2eec93c987ea0369f73"
Jul 30 03:46:06.871041 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:06.870977    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"524ab98a2e2e0103148bc0d82d9e0377c70bc45927c1d2eec93c987ea0369f73"} err="failed to get container status \"524ab98a2e2e0103148bc0d82d9e0377c70bc45927c1d2eec93c987ea0369f73\": rpc error: code = NotFound desc = could not find container \"524ab98a2e2e0103148bc0d82d9e0377c70bc45927c1d2eec93c987ea0369f73\": container with ID starting with 524ab98a2e2e0103148bc0d82d9e0377c70bc45927c1d2eec93c987ea0369f73 not found: ID does not exist"
Jul 30 03:46:06.883651 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:06.883632    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-7bkdz"]
Jul 30 03:46:06.887141 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:06.887123    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-7bkdz"]
Jul 30 03:46:07.055599 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:07.055567    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0af99020-442e-41bd-9f8a-ac75c3df4e51" path="/var/lib/kubelet/pods/0af99020-442e-41bd-9f8a-ac75c3df4e51/volumes"
Jul 30 03:46:08.813382 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:08.813353    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:08.916813 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:08.916779    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:09.227028 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:09.226949    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-67vvd"]
Jul 30 03:46:09.227352 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:09.227335    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4606768-ed42-4b2d-87a1-8688566b8a12" containerName="authorino"
Jul 30 03:46:09.227441 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:09.227355    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4606768-ed42-4b2d-87a1-8688566b8a12" containerName="authorino"
Jul 30 03:46:09.227441 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:09.227382    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0af99020-442e-41bd-9f8a-ac75c3df4e51" containerName="limitador"
Jul 30 03:46:09.227441 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:09.227390    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0af99020-442e-41bd-9f8a-ac75c3df4e51" containerName="limitador"
Jul 30 03:46:09.227591 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:09.227468    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0af99020-442e-41bd-9f8a-ac75c3df4e51" containerName="limitador"
Jul 30 03:46:09.227591 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:09.227483    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4606768-ed42-4b2d-87a1-8688566b8a12" containerName="authorino"
Jul 30 03:46:09.231548 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:09.231526    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-67vvd"
Jul 30 03:46:09.234452 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:09.234431    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 30 03:46:09.234588 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:09.234552    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 30 03:46:09.235698 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:09.235676    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 03:46:09.235698 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:09.235692    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-cz2dh\""
Jul 30 03:46:09.240198 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:09.240179    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-67vvd"]
Jul 30 03:46:09.262229 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:09.262207    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f44c8\" (UniqueName: \"kubernetes.io/projected/a0530845-4a21-41c2-969e-daa4a0d1feba-kube-api-access-f44c8\") pod \"postgres-69fb4594cf-67vvd\" (UID: \"a0530845-4a21-41c2-969e-daa4a0d1feba\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-67vvd"
Jul 30 03:46:09.262321 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:09.262258    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a0530845-4a21-41c2-969e-daa4a0d1feba-data\") pod \"postgres-69fb4594cf-67vvd\" (UID: \"a0530845-4a21-41c2-969e-daa4a0d1feba\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-67vvd"
Jul 30 03:46:09.267562 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:09.267541    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:09.363231 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:09.363205    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f44c8\" (UniqueName: \"kubernetes.io/projected/a0530845-4a21-41c2-969e-daa4a0d1feba-kube-api-access-f44c8\") pod \"postgres-69fb4594cf-67vvd\" (UID: \"a0530845-4a21-41c2-969e-daa4a0d1feba\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-67vvd"
Jul 30 03:46:09.363521 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:09.363264    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a0530845-4a21-41c2-969e-daa4a0d1feba-data\") pod \"postgres-69fb4594cf-67vvd\" (UID: \"a0530845-4a21-41c2-969e-daa4a0d1feba\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-67vvd"
Jul 30 03:46:09.363613 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:09.363594    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a0530845-4a21-41c2-969e-daa4a0d1feba-data\") pod \"postgres-69fb4594cf-67vvd\" (UID: \"a0530845-4a21-41c2-969e-daa4a0d1feba\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-67vvd"
Jul 30 03:46:09.371727 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:09.371704    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f44c8\" (UniqueName: \"kubernetes.io/projected/a0530845-4a21-41c2-969e-daa4a0d1feba-kube-api-access-f44c8\") pod \"postgres-69fb4594cf-67vvd\" (UID: \"a0530845-4a21-41c2-969e-daa4a0d1feba\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-67vvd"
Jul 30 03:46:09.542792 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:09.542764    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-67vvd"
Jul 30 03:46:09.668476 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:09.668452    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-67vvd"]
Jul 30 03:46:09.670636 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:46:09.670607    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0530845_4a21_41c2_969e_daa4a0d1feba.slice/crio-83fa9b091069d2a2bf3d2edc53b746abe6ff15bcad752d0cdeee8fc4c7c85469 WatchSource:0}: Error finding container 83fa9b091069d2a2bf3d2edc53b746abe6ff15bcad752d0cdeee8fc4c7c85469: Status 404 returned error can't find the container with id 83fa9b091069d2a2bf3d2edc53b746abe6ff15bcad752d0cdeee8fc4c7c85469
Jul 30 03:46:09.671854 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:09.671837    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 03:46:09.878206 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:09.878129    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-67vvd" event={"ID":"a0530845-4a21-41c2-969e-daa4a0d1feba","Type":"ContainerStarted","Data":"83fa9b091069d2a2bf3d2edc53b746abe6ff15bcad752d0cdeee8fc4c7c85469"}
Jul 30 03:46:14.913079 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:14.913045    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-67vvd" event={"ID":"a0530845-4a21-41c2-969e-daa4a0d1feba","Type":"ContainerStarted","Data":"c7944ed8df6537c6a980bccf0d762a0cae32979cb7584961286aa207c6a06d84"}
Jul 30 03:46:14.913482 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:14.913178    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-67vvd"
Jul 30 03:46:14.930483 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:14.930400    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-67vvd" podStartSLOduration=0.921797463 podStartE2EDuration="5.930387809s" totalImagesPullingTime="5.008590346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:46:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:46:14.929377657 +0000 UTC m=+680.547609898" watchObservedRunningTime="2026-07-30 03:46:14.930387809 +0000 UTC m=+680.548620049"
Jul 30 03:46:20.946978 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:20.946937    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-67vvd"
Jul 30 03:46:21.271604 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:21.271570    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:21.470960 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:21.470913    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-txnlz"]
Jul 30 03:46:21.477067 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:21.477048    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-txnlz"
Jul 30 03:46:21.481839 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:21.481814    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-txnlz"]
Jul 30 03:46:21.571370 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:21.571291    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwcdl\" (UniqueName: \"kubernetes.io/projected/b94263bf-af20-4da2-ac49-5b1f6113bd54-kube-api-access-gwcdl\") pod \"authorino-5d4b67c446-txnlz\" (UID: \"b94263bf-af20-4da2-ac49-5b1f6113bd54\") " pod="kuadrant-system/authorino-5d4b67c446-txnlz"
Jul 30 03:46:21.671968 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:21.671908    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwcdl\" (UniqueName: \"kubernetes.io/projected/b94263bf-af20-4da2-ac49-5b1f6113bd54-kube-api-access-gwcdl\") pod \"authorino-5d4b67c446-txnlz\" (UID: \"b94263bf-af20-4da2-ac49-5b1f6113bd54\") " pod="kuadrant-system/authorino-5d4b67c446-txnlz"
Jul 30 03:46:21.680547 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:21.680522    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwcdl\" (UniqueName: \"kubernetes.io/projected/b94263bf-af20-4da2-ac49-5b1f6113bd54-kube-api-access-gwcdl\") pod \"authorino-5d4b67c446-txnlz\" (UID: \"b94263bf-af20-4da2-ac49-5b1f6113bd54\") " pod="kuadrant-system/authorino-5d4b67c446-txnlz"
Jul 30 03:46:21.708600 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:21.708571    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-txnlz"]
Jul 30 03:46:21.708817 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:21.708802    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-txnlz"
Jul 30 03:46:21.716214 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:21.716185    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:21.746909 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:21.746875    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7cb4f5f86c-qqfbk"]
Jul 30 03:46:21.751071 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:21.751050    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7cb4f5f86c-qqfbk"
Jul 30 03:46:21.757694 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:21.757674    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7cb4f5f86c-qqfbk"]
Jul 30 03:46:21.837500 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:21.837449    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-txnlz"]
Jul 30 03:46:21.840179 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:46:21.840151    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb94263bf_af20_4da2_ac49_5b1f6113bd54.slice/crio-e6ccec49f75d3074f7f9dde983e823daac224a477d0132f4351c20284ae04ce7 WatchSource:0}: Error finding container e6ccec49f75d3074f7f9dde983e823daac224a477d0132f4351c20284ae04ce7: Status 404 returned error can't find the container with id e6ccec49f75d3074f7f9dde983e823daac224a477d0132f4351c20284ae04ce7
Jul 30 03:46:21.873104 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:21.873076    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zmj4\" (UniqueName: \"kubernetes.io/projected/5568e06c-9fa8-4b89-9d11-5cb04fc0038b-kube-api-access-8zmj4\") pod \"authorino-7cb4f5f86c-qqfbk\" (UID: \"5568e06c-9fa8-4b89-9d11-5cb04fc0038b\") " pod="kuadrant-system/authorino-7cb4f5f86c-qqfbk"
Jul 30 03:46:21.949614 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:21.949591    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-txnlz" event={"ID":"b94263bf-af20-4da2-ac49-5b1f6113bd54","Type":"ContainerStarted","Data":"e6ccec49f75d3074f7f9dde983e823daac224a477d0132f4351c20284ae04ce7"}
Jul 30 03:46:21.968854 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:21.968829    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7cb4f5f86c-qqfbk"]
Jul 30 03:46:21.969072 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:46:21.969055    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-8zmj4], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-7cb4f5f86c-qqfbk" podUID="5568e06c-9fa8-4b89-9d11-5cb04fc0038b"
Jul 30 03:46:21.974043 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:21.974018    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zmj4\" (UniqueName: \"kubernetes.io/projected/5568e06c-9fa8-4b89-9d11-5cb04fc0038b-kube-api-access-8zmj4\") pod \"authorino-7cb4f5f86c-qqfbk\" (UID: \"5568e06c-9fa8-4b89-9d11-5cb04fc0038b\") " pod="kuadrant-system/authorino-7cb4f5f86c-qqfbk"
Jul 30 03:46:21.982076 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:21.982059    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zmj4\" (UniqueName: \"kubernetes.io/projected/5568e06c-9fa8-4b89-9d11-5cb04fc0038b-kube-api-access-8zmj4\") pod \"authorino-7cb4f5f86c-qqfbk\" (UID: \"5568e06c-9fa8-4b89-9d11-5cb04fc0038b\") " pod="kuadrant-system/authorino-7cb4f5f86c-qqfbk"
Jul 30 03:46:21.996388 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:21.996358    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-d4665d78c-dqsbg"]
Jul 30 03:46:22.000305 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:22.000286    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d4665d78c-dqsbg"
Jul 30 03:46:22.002733 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:22.002715    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 30 03:46:22.006614 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:22.006594    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-d4665d78c-dqsbg"]
Jul 30 03:46:22.074736 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:22.074704    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/53a15a1e-1cf7-4923-a486-8d8965c7da98-tls-cert\") pod \"authorino-d4665d78c-dqsbg\" (UID: \"53a15a1e-1cf7-4923-a486-8d8965c7da98\") " pod="kuadrant-system/authorino-d4665d78c-dqsbg"
Jul 30 03:46:22.074908 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:22.074825    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj666\" (UniqueName: \"kubernetes.io/projected/53a15a1e-1cf7-4923-a486-8d8965c7da98-kube-api-access-vj666\") pod \"authorino-d4665d78c-dqsbg\" (UID: \"53a15a1e-1cf7-4923-a486-8d8965c7da98\") " pod="kuadrant-system/authorino-d4665d78c-dqsbg"
Jul 30 03:46:22.176022 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:22.175952    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/53a15a1e-1cf7-4923-a486-8d8965c7da98-tls-cert\") pod \"authorino-d4665d78c-dqsbg\" (UID: \"53a15a1e-1cf7-4923-a486-8d8965c7da98\") " pod="kuadrant-system/authorino-d4665d78c-dqsbg"
Jul 30 03:46:22.176153 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:22.176037    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vj666\" (UniqueName: \"kubernetes.io/projected/53a15a1e-1cf7-4923-a486-8d8965c7da98-kube-api-access-vj666\") pod \"authorino-d4665d78c-dqsbg\" (UID: \"53a15a1e-1cf7-4923-a486-8d8965c7da98\") " pod="kuadrant-system/authorino-d4665d78c-dqsbg"
Jul 30 03:46:22.178312 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:22.178293    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/53a15a1e-1cf7-4923-a486-8d8965c7da98-tls-cert\") pod \"authorino-d4665d78c-dqsbg\" (UID: \"53a15a1e-1cf7-4923-a486-8d8965c7da98\") " pod="kuadrant-system/authorino-d4665d78c-dqsbg"
Jul 30 03:46:22.184072 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:22.184054    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj666\" (UniqueName: \"kubernetes.io/projected/53a15a1e-1cf7-4923-a486-8d8965c7da98-kube-api-access-vj666\") pod \"authorino-d4665d78c-dqsbg\" (UID: \"53a15a1e-1cf7-4923-a486-8d8965c7da98\") " pod="kuadrant-system/authorino-d4665d78c-dqsbg"
Jul 30 03:46:22.310759 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:22.310738    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d4665d78c-dqsbg"
Jul 30 03:46:22.439542 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:22.439518    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-d4665d78c-dqsbg"]
Jul 30 03:46:22.441297 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:46:22.441268    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod53a15a1e_1cf7_4923_a486_8d8965c7da98.slice/crio-6150a742a8d090902de0cd6682324a948544f498e5624c1d39550ad442bf95cd WatchSource:0}: Error finding container 6150a742a8d090902de0cd6682324a948544f498e5624c1d39550ad442bf95cd: Status 404 returned error can't find the container with id 6150a742a8d090902de0cd6682324a948544f498e5624c1d39550ad442bf95cd
Jul 30 03:46:22.956608 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:22.956575    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d4665d78c-dqsbg" event={"ID":"53a15a1e-1cf7-4923-a486-8d8965c7da98","Type":"ContainerStarted","Data":"e20bde47d13382b3c5bee89e8fdbce66507a148d189f11585ee7d7b43696e8d6"}
Jul 30 03:46:22.957076 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:22.956618    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d4665d78c-dqsbg" event={"ID":"53a15a1e-1cf7-4923-a486-8d8965c7da98","Type":"ContainerStarted","Data":"6150a742a8d090902de0cd6682324a948544f498e5624c1d39550ad442bf95cd"}
Jul 30 03:46:22.958107 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:22.958078    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-txnlz" event={"ID":"b94263bf-af20-4da2-ac49-5b1f6113bd54","Type":"ContainerStarted","Data":"01f314221f818e0f23e540df13c63ea842a9997e84522b05812a004dad234fa8"}
Jul 30 03:46:22.958210 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:22.958088    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7cb4f5f86c-qqfbk"
Jul 30 03:46:22.958210 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:22.958135    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-txnlz" podUID="b94263bf-af20-4da2-ac49-5b1f6113bd54" containerName="authorino" containerID="cri-o://01f314221f818e0f23e540df13c63ea842a9997e84522b05812a004dad234fa8" gracePeriod=30
Jul 30 03:46:22.965496 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:22.965480    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7cb4f5f86c-qqfbk"
Jul 30 03:46:22.971756 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:22.971685    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-d4665d78c-dqsbg" podStartSLOduration=1.610741872 podStartE2EDuration="1.971671941s" totalImagesPullingTime="360.930069ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:46:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:46:22.971525953 +0000 UTC m=+688.589758196" watchObservedRunningTime="2026-07-30 03:46:22.971671941 +0000 UTC m=+688.589904182"
Jul 30 03:46:23.000733 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:23.000704    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-r9g82"]
Jul 30 03:46:23.000963 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:23.000913    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-r9g82" podUID="62e46aa2-c703-47f5-b604-e5c3c2fa29b5" containerName="authorino" containerID="cri-o://190b238c7698b1539ab776192348c60b690a9df8895c0f6e006cdd30831af98e" gracePeriod=30
Jul 30 03:46:23.007773 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:23.007731    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-txnlz" podStartSLOduration=1.596414781 podStartE2EDuration="2.007719007s" totalImagesPullingTime="411.304226ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:46:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:46:23.004732312 +0000 UTC m=+688.622964554" watchObservedRunningTime="2026-07-30 03:46:23.007719007 +0000 UTC m=+688.625951247"
Jul 30 03:46:23.085014 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:23.084950    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8zmj4\" (UniqueName: \"kubernetes.io/projected/5568e06c-9fa8-4b89-9d11-5cb04fc0038b-kube-api-access-8zmj4\") pod \"5568e06c-9fa8-4b89-9d11-5cb04fc0038b\" (UID: \"5568e06c-9fa8-4b89-9d11-5cb04fc0038b\") "
Jul 30 03:46:23.086691 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:23.086667    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5568e06c-9fa8-4b89-9d11-5cb04fc0038b-kube-api-access-8zmj4" (OuterVolumeSpecName: "kube-api-access-8zmj4") pod "5568e06c-9fa8-4b89-9d11-5cb04fc0038b" (UID: "5568e06c-9fa8-4b89-9d11-5cb04fc0038b"). InnerVolumeSpecName "kube-api-access-8zmj4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:46:23.186635 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:23.186607    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8zmj4\" (UniqueName: \"kubernetes.io/projected/5568e06c-9fa8-4b89-9d11-5cb04fc0038b-kube-api-access-8zmj4\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:46:23.256412 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:23.256391    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-txnlz"
Jul 30 03:46:23.266863 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:23.266842    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-r9g82"
Jul 30 03:46:23.388167 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:23.388092    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5kz4\" (UniqueName: \"kubernetes.io/projected/62e46aa2-c703-47f5-b604-e5c3c2fa29b5-kube-api-access-g5kz4\") pod \"62e46aa2-c703-47f5-b604-e5c3c2fa29b5\" (UID: \"62e46aa2-c703-47f5-b604-e5c3c2fa29b5\") "
Jul 30 03:46:23.388336 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:23.388173    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gwcdl\" (UniqueName: \"kubernetes.io/projected/b94263bf-af20-4da2-ac49-5b1f6113bd54-kube-api-access-gwcdl\") pod \"b94263bf-af20-4da2-ac49-5b1f6113bd54\" (UID: \"b94263bf-af20-4da2-ac49-5b1f6113bd54\") "
Jul 30 03:46:23.389956 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:23.389909    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62e46aa2-c703-47f5-b604-e5c3c2fa29b5-kube-api-access-g5kz4" (OuterVolumeSpecName: "kube-api-access-g5kz4") pod "62e46aa2-c703-47f5-b604-e5c3c2fa29b5" (UID: "62e46aa2-c703-47f5-b604-e5c3c2fa29b5"). InnerVolumeSpecName "kube-api-access-g5kz4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:46:23.390061 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:23.389964    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b94263bf-af20-4da2-ac49-5b1f6113bd54-kube-api-access-gwcdl" (OuterVolumeSpecName: "kube-api-access-gwcdl") pod "b94263bf-af20-4da2-ac49-5b1f6113bd54" (UID: "b94263bf-af20-4da2-ac49-5b1f6113bd54"). InnerVolumeSpecName "kube-api-access-gwcdl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:46:23.489507 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:23.489479    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5kz4\" (UniqueName: \"kubernetes.io/projected/62e46aa2-c703-47f5-b604-e5c3c2fa29b5-kube-api-access-g5kz4\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:46:23.489507 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:23.489503    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwcdl\" (UniqueName: \"kubernetes.io/projected/b94263bf-af20-4da2-ac49-5b1f6113bd54-kube-api-access-gwcdl\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:46:23.967731 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:23.967649    2576 generic.go:356] "Generic (PLEG): container finished" podID="b94263bf-af20-4da2-ac49-5b1f6113bd54" containerID="01f314221f818e0f23e540df13c63ea842a9997e84522b05812a004dad234fa8" exitCode=0
Jul 30 03:46:23.967731 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:23.967713    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-txnlz"
Jul 30 03:46:23.968341 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:23.967734    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-txnlz" event={"ID":"b94263bf-af20-4da2-ac49-5b1f6113bd54","Type":"ContainerDied","Data":"01f314221f818e0f23e540df13c63ea842a9997e84522b05812a004dad234fa8"}
Jul 30 03:46:23.968341 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:23.967777    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-txnlz" event={"ID":"b94263bf-af20-4da2-ac49-5b1f6113bd54","Type":"ContainerDied","Data":"e6ccec49f75d3074f7f9dde983e823daac224a477d0132f4351c20284ae04ce7"}
Jul 30 03:46:23.968341 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:23.967799    2576 scope.go:117] "RemoveContainer" containerID="01f314221f818e0f23e540df13c63ea842a9997e84522b05812a004dad234fa8"
Jul 30 03:46:23.969222 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:23.968902    2576 generic.go:356] "Generic (PLEG): container finished" podID="62e46aa2-c703-47f5-b604-e5c3c2fa29b5" containerID="190b238c7698b1539ab776192348c60b690a9df8895c0f6e006cdd30831af98e" exitCode=0
Jul 30 03:46:23.969222 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:23.968989    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-r9g82"
Jul 30 03:46:23.969222 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:23.968987    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-r9g82" event={"ID":"62e46aa2-c703-47f5-b604-e5c3c2fa29b5","Type":"ContainerDied","Data":"190b238c7698b1539ab776192348c60b690a9df8895c0f6e006cdd30831af98e"}
Jul 30 03:46:23.969222 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:23.969029    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-r9g82" event={"ID":"62e46aa2-c703-47f5-b604-e5c3c2fa29b5","Type":"ContainerDied","Data":"d514ff25cb89d6d3dc6fdc6b85f24579c6251bf57f490416608ac9d66518e302"}
Jul 30 03:46:23.969222 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:23.969111    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7cb4f5f86c-qqfbk"
Jul 30 03:46:23.977574 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:23.977556    2576 scope.go:117] "RemoveContainer" containerID="01f314221f818e0f23e540df13c63ea842a9997e84522b05812a004dad234fa8"
Jul 30 03:46:23.978004 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:46:23.977818    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01f314221f818e0f23e540df13c63ea842a9997e84522b05812a004dad234fa8\": container with ID starting with 01f314221f818e0f23e540df13c63ea842a9997e84522b05812a004dad234fa8 not found: ID does not exist" containerID="01f314221f818e0f23e540df13c63ea842a9997e84522b05812a004dad234fa8"
Jul 30 03:46:23.978004 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:23.977843    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01f314221f818e0f23e540df13c63ea842a9997e84522b05812a004dad234fa8"} err="failed to get container status \"01f314221f818e0f23e540df13c63ea842a9997e84522b05812a004dad234fa8\": rpc error: code = NotFound desc = could not find container \"01f314221f818e0f23e540df13c63ea842a9997e84522b05812a004dad234fa8\": container with ID starting with 01f314221f818e0f23e540df13c63ea842a9997e84522b05812a004dad234fa8 not found: ID does not exist"
Jul 30 03:46:23.978004 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:23.977860    2576 scope.go:117] "RemoveContainer" containerID="190b238c7698b1539ab776192348c60b690a9df8895c0f6e006cdd30831af98e"
Jul 30 03:46:23.985645 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:23.985631    2576 scope.go:117] "RemoveContainer" containerID="190b238c7698b1539ab776192348c60b690a9df8895c0f6e006cdd30831af98e"
Jul 30 03:46:23.985875 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:46:23.985859    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"190b238c7698b1539ab776192348c60b690a9df8895c0f6e006cdd30831af98e\": container with ID starting with 190b238c7698b1539ab776192348c60b690a9df8895c0f6e006cdd30831af98e not found: ID does not exist" containerID="190b238c7698b1539ab776192348c60b690a9df8895c0f6e006cdd30831af98e"
Jul 30 03:46:23.985984 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:23.985881    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"190b238c7698b1539ab776192348c60b690a9df8895c0f6e006cdd30831af98e"} err="failed to get container status \"190b238c7698b1539ab776192348c60b690a9df8895c0f6e006cdd30831af98e\": rpc error: code = NotFound desc = could not find container \"190b238c7698b1539ab776192348c60b690a9df8895c0f6e006cdd30831af98e\": container with ID starting with 190b238c7698b1539ab776192348c60b690a9df8895c0f6e006cdd30831af98e not found: ID does not exist"
Jul 30 03:46:24.001314 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:24.001292    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7cb4f5f86c-qqfbk"]
Jul 30 03:46:24.005187 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:24.005165    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7cb4f5f86c-qqfbk"]
Jul 30 03:46:24.015957 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:24.015917    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-r9g82"]
Jul 30 03:46:24.017972 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:24.017953    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-r9g82"]
Jul 30 03:46:24.027932 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:24.027902    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-txnlz"]
Jul 30 03:46:24.033730 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:24.033712    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-txnlz"]
Jul 30 03:46:25.051112 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:25.051077    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5568e06c-9fa8-4b89-9d11-5cb04fc0038b" path="/var/lib/kubelet/pods/5568e06c-9fa8-4b89-9d11-5cb04fc0038b/volumes"
Jul 30 03:46:25.051468 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:25.051310    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62e46aa2-c703-47f5-b604-e5c3c2fa29b5" path="/var/lib/kubelet/pods/62e46aa2-c703-47f5-b604-e5c3c2fa29b5/volumes"
Jul 30 03:46:25.051626 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:25.051613    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b94263bf-af20-4da2-ac49-5b1f6113bd54" path="/var/lib/kubelet/pods/b94263bf-af20-4da2-ac49-5b1f6113bd54/volumes"
Jul 30 03:46:27.576121 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:27.576084    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:27.667048 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:27.667014    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:27.765551 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:27.765520    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:30.817051 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:30.817018    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:46:30.915097 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:46:30.915069    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:47:03.418098 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:03.418067    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:47:03.516568 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:03.516521    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:47:04.965006 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:04.964975    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:47:05.292425 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:05.292396    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:47:05.471751 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:05.471713    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:47:05.574913 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:05.574834    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:47:05.592227 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:05.592194    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-8qlzb"]
Jul 30 03:47:05.592525 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:05.592513    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b94263bf-af20-4da2-ac49-5b1f6113bd54" containerName="authorino"
Jul 30 03:47:05.592574 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:05.592527    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b94263bf-af20-4da2-ac49-5b1f6113bd54" containerName="authorino"
Jul 30 03:47:05.592574 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:05.592534    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62e46aa2-c703-47f5-b604-e5c3c2fa29b5" containerName="authorino"
Jul 30 03:47:05.592574 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:05.592539    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="62e46aa2-c703-47f5-b604-e5c3c2fa29b5" containerName="authorino"
Jul 30 03:47:05.592670 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:05.592595    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="62e46aa2-c703-47f5-b604-e5c3c2fa29b5" containerName="authorino"
Jul 30 03:47:05.592670 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:05.592604    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b94263bf-af20-4da2-ac49-5b1f6113bd54" containerName="authorino"
Jul 30 03:47:05.597128 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:05.597106    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-8qlzb"
Jul 30 03:47:05.599961 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:05.599943    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 30 03:47:05.600082 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:05.599948    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-vb6c4\""
Jul 30 03:47:05.606158 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:05.606139    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-8qlzb"]
Jul 30 03:47:05.633221 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:05.633201    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzjgn\" (UniqueName: \"kubernetes.io/projected/9b4ba3f1-07c7-4335-97ad-bf7c9269f7c3-kube-api-access-zzjgn\") pod \"payload-processing-5cc6cd444b-8qlzb\" (UID: \"9b4ba3f1-07c7-4335-97ad-bf7c9269f7c3\") " pod="openshift-ingress/payload-processing-5cc6cd444b-8qlzb"
Jul 30 03:47:05.633316 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:05.633246    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b4ba3f1-07c7-4335-97ad-bf7c9269f7c3-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-8qlzb\" (UID: \"9b4ba3f1-07c7-4335-97ad-bf7c9269f7c3\") " pod="openshift-ingress/payload-processing-5cc6cd444b-8qlzb"
Jul 30 03:47:05.655644 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:05.655616    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 03:47:05.734480 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:05.734450    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzjgn\" (UniqueName: \"kubernetes.io/projected/9b4ba3f1-07c7-4335-97ad-bf7c9269f7c3-kube-api-access-zzjgn\") pod \"payload-processing-5cc6cd444b-8qlzb\" (UID: \"9b4ba3f1-07c7-4335-97ad-bf7c9269f7c3\") " pod="openshift-ingress/payload-processing-5cc6cd444b-8qlzb"
Jul 30 03:47:05.734655 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:05.734531    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b4ba3f1-07c7-4335-97ad-bf7c9269f7c3-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-8qlzb\" (UID: \"9b4ba3f1-07c7-4335-97ad-bf7c9269f7c3\") " pod="openshift-ingress/payload-processing-5cc6cd444b-8qlzb"
Jul 30 03:47:05.735407 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:05.735373    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b4ba3f1-07c7-4335-97ad-bf7c9269f7c3-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-8qlzb\" (UID: \"9b4ba3f1-07c7-4335-97ad-bf7c9269f7c3\") " pod="openshift-ingress/payload-processing-5cc6cd444b-8qlzb"
Jul 30 03:47:05.743229 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:05.743205    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzjgn\" (UniqueName: \"kubernetes.io/projected/9b4ba3f1-07c7-4335-97ad-bf7c9269f7c3-kube-api-access-zzjgn\") pod \"payload-processing-5cc6cd444b-8qlzb\" (UID: \"9b4ba3f1-07c7-4335-97ad-bf7c9269f7c3\") " pod="openshift-ingress/payload-processing-5cc6cd444b-8qlzb"
Jul 30 03:47:05.908971 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:05.908876    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-8qlzb"
Jul 30 03:47:06.036591 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:06.036563    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-8qlzb"]
Jul 30 03:47:06.038497 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:47:06.038472    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b4ba3f1_07c7_4335_97ad_bf7c9269f7c3.slice/crio-6e4c0edd589760c75f41120964696d61c14c6d8dcdf66640aa37465875c015b5 WatchSource:0}: Error finding container 6e4c0edd589760c75f41120964696d61c14c6d8dcdf66640aa37465875c015b5: Status 404 returned error can't find the container with id 6e4c0edd589760c75f41120964696d61c14c6d8dcdf66640aa37465875c015b5
Jul 30 03:47:06.174452 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:06.174345    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-8qlzb" event={"ID":"9b4ba3f1-07c7-4335-97ad-bf7c9269f7c3","Type":"ContainerStarted","Data":"6e4c0edd589760c75f41120964696d61c14c6d8dcdf66640aa37465875c015b5"}
Jul 30 03:47:08.185655 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:08.185616    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-8qlzb" event={"ID":"9b4ba3f1-07c7-4335-97ad-bf7c9269f7c3","Type":"ContainerStarted","Data":"412b41546c8ea4b85c44650ba199dcdd9feb90edafad5e9234617b90ad513e1d"}
Jul 30 03:47:08.186110 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:08.185687    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-8qlzb"
Jul 30 03:47:08.202640 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:08.202598    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-8qlzb" podStartSLOduration=1.251230455 podStartE2EDuration="3.202584458s" totalImagesPullingTime="1.951354003s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:47:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:47:08.200270231 +0000 UTC m=+733.818502475" watchObservedRunningTime="2026-07-30 03:47:08.202584458 +0000 UTC m=+733.820816699"
Jul 30 03:47:13.212052 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:13.211975    2576 generic.go:356] "Generic (PLEG): container finished" podID="60518678-0996-4b4f-a74c-1221fecbedcc" containerID="d0d24cb7ad604cdbd008169efdcf22401a2b8494dc4c7ba7965128599712887b" exitCode=0
Jul 30 03:47:13.212052 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:13.212043    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm" event={"ID":"60518678-0996-4b4f-a74c-1221fecbedcc","Type":"ContainerDied","Data":"d0d24cb7ad604cdbd008169efdcf22401a2b8494dc4c7ba7965128599712887b"}
Jul 30 03:47:13.212472 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:13.212391    2576 scope.go:117] "RemoveContainer" containerID="d0d24cb7ad604cdbd008169efdcf22401a2b8494dc4c7ba7965128599712887b"
Jul 30 03:47:13.213361 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:13.213332    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 03:47:13.213427 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:13.213414    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 03:47:13.213485 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:13.213470    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 03:47:14.218676 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:14.218642    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm" event={"ID":"60518678-0996-4b4f-a74c-1221fecbedcc","Type":"ContainerStarted","Data":"e8326cc801b6f834a55178359fccab789e6a60a19cbc138b73718bb0fabebde0"}
Jul 30 03:47:14.611812 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:14.611782    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:47:14.714710 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:14.714675    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:47:14.759216 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:14.759178    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"
Jul 30 03:47:14.760219 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:14.760189    2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-zvgfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Jul 30 03:47:14.760360 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:14.760241    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm" podUID="60518678-0996-4b4f-a74c-1221fecbedcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Jul 30 03:47:15.758667 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:15.758627    2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-zvgfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Jul 30 03:47:15.759169 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:15.758678    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm" podUID="60518678-0996-4b4f-a74c-1221fecbedcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Jul 30 03:47:16.759370 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:16.759327    2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-zvgfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Jul 30 03:47:16.759768 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:16.759437    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm" podUID="60518678-0996-4b4f-a74c-1221fecbedcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Jul 30 03:47:17.737169 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:17.737129    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"
Jul 30 03:47:17.759510 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:17.759475    2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-zvgfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Jul 30 03:47:17.759870 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:17.759582    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm" podUID="60518678-0996-4b4f-a74c-1221fecbedcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Jul 30 03:47:18.763005 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:18.762979    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"
Jul 30 03:47:18.763976 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:18.763959    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zvgfm"
Jul 30 03:47:19.192918 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:19.192891    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-8qlzb"
Jul 30 03:47:30.736761 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:30.736725    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk"]
Jul 30 03:47:30.739331 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:30.739311    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk"
Jul 30 03:47:30.742049 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:30.742026    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 30 03:47:30.742049 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:30.742040    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 30 03:47:30.742195 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:30.742025    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 30 03:47:30.742961 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:30.742946    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-65bpz\""
Jul 30 03:47:30.755694 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:30.755671    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk"]
Jul 30 03:47:30.766240 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:30.766220    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:47:30.874819 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:30.874782    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abcac6f7-ad4c-4b47-8b18-348d98bc3c87-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk\" (UID: \"abcac6f7-ad4c-4b47-8b18-348d98bc3c87\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk"
Jul 30 03:47:30.874993 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:30.874838    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abcac6f7-ad4c-4b47-8b18-348d98bc3c87-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk\" (UID: \"abcac6f7-ad4c-4b47-8b18-348d98bc3c87\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk"
Jul 30 03:47:30.874993 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:30.874901    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abcac6f7-ad4c-4b47-8b18-348d98bc3c87-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk\" (UID: \"abcac6f7-ad4c-4b47-8b18-348d98bc3c87\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk"
Jul 30 03:47:30.875087 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:30.874997    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntlst\" (UniqueName: \"kubernetes.io/projected/abcac6f7-ad4c-4b47-8b18-348d98bc3c87-kube-api-access-ntlst\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk\" (UID: \"abcac6f7-ad4c-4b47-8b18-348d98bc3c87\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk"
Jul 30 03:47:30.875087 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:30.875033    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abcac6f7-ad4c-4b47-8b18-348d98bc3c87-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk\" (UID: \"abcac6f7-ad4c-4b47-8b18-348d98bc3c87\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk"
Jul 30 03:47:30.875087 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:30.875067    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abcac6f7-ad4c-4b47-8b18-348d98bc3c87-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk\" (UID: \"abcac6f7-ad4c-4b47-8b18-348d98bc3c87\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk"
Jul 30 03:47:30.976062 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:30.976025    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntlst\" (UniqueName: \"kubernetes.io/projected/abcac6f7-ad4c-4b47-8b18-348d98bc3c87-kube-api-access-ntlst\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk\" (UID: \"abcac6f7-ad4c-4b47-8b18-348d98bc3c87\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk"
Jul 30 03:47:30.976246 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:30.976077    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abcac6f7-ad4c-4b47-8b18-348d98bc3c87-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk\" (UID: \"abcac6f7-ad4c-4b47-8b18-348d98bc3c87\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk"
Jul 30 03:47:30.976246 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:30.976135    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abcac6f7-ad4c-4b47-8b18-348d98bc3c87-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk\" (UID: \"abcac6f7-ad4c-4b47-8b18-348d98bc3c87\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk"
Jul 30 03:47:30.976246 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:30.976171    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abcac6f7-ad4c-4b47-8b18-348d98bc3c87-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk\" (UID: \"abcac6f7-ad4c-4b47-8b18-348d98bc3c87\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk"
Jul 30 03:47:30.976246 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:30.976205    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abcac6f7-ad4c-4b47-8b18-348d98bc3c87-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk\" (UID: \"abcac6f7-ad4c-4b47-8b18-348d98bc3c87\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk"
Jul 30 03:47:30.976246 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:30.976231    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abcac6f7-ad4c-4b47-8b18-348d98bc3c87-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk\" (UID: \"abcac6f7-ad4c-4b47-8b18-348d98bc3c87\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk"
Jul 30 03:47:30.976530 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:30.976510    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abcac6f7-ad4c-4b47-8b18-348d98bc3c87-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk\" (UID: \"abcac6f7-ad4c-4b47-8b18-348d98bc3c87\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk"
Jul 30 03:47:30.976602 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:30.976578    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abcac6f7-ad4c-4b47-8b18-348d98bc3c87-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk\" (UID: \"abcac6f7-ad4c-4b47-8b18-348d98bc3c87\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk"
Jul 30 03:47:30.976654 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:30.976602    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abcac6f7-ad4c-4b47-8b18-348d98bc3c87-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk\" (UID: \"abcac6f7-ad4c-4b47-8b18-348d98bc3c87\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk"
Jul 30 03:47:30.978328 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:30.978306    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abcac6f7-ad4c-4b47-8b18-348d98bc3c87-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk\" (UID: \"abcac6f7-ad4c-4b47-8b18-348d98bc3c87\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk"
Jul 30 03:47:30.978462 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:30.978445    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abcac6f7-ad4c-4b47-8b18-348d98bc3c87-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk\" (UID: \"abcac6f7-ad4c-4b47-8b18-348d98bc3c87\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk"
Jul 30 03:47:30.984261 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:30.984245    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntlst\" (UniqueName: \"kubernetes.io/projected/abcac6f7-ad4c-4b47-8b18-348d98bc3c87-kube-api-access-ntlst\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk\" (UID: \"abcac6f7-ad4c-4b47-8b18-348d98bc3c87\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk"
Jul 30 03:47:31.049570 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:31.049546    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk"
Jul 30 03:47:31.176961 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:31.176510    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk"]
Jul 30 03:47:31.178369 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:47:31.178342    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabcac6f7_ad4c_4b47_8b18_348d98bc3c87.slice/crio-092683898fe3f033bc76988740b5daac059093221e2ba25d1b9975228c17c18e WatchSource:0}: Error finding container 092683898fe3f033bc76988740b5daac059093221e2ba25d1b9975228c17c18e: Status 404 returned error can't find the container with id 092683898fe3f033bc76988740b5daac059093221e2ba25d1b9975228c17c18e
Jul 30 03:47:31.305641 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:31.305560    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk" event={"ID":"abcac6f7-ad4c-4b47-8b18-348d98bc3c87","Type":"ContainerStarted","Data":"092683898fe3f033bc76988740b5daac059093221e2ba25d1b9975228c17c18e"}
Jul 30 03:47:31.393345 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:31.393310    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 03:47:33.316673 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:33.316635    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk" event={"ID":"abcac6f7-ad4c-4b47-8b18-348d98bc3c87","Type":"ContainerStarted","Data":"c8c0244bf493ad2545c05a1a7876c3fc0c2e32b50515437b8f4ed49617f2aece"}
Jul 30 03:47:33.335655 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:33.335602    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk" podStartSLOduration=2.248910658 podStartE2EDuration="3.335586807s" totalImagesPullingTime="1.086676149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:47:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:47:33.332911965 +0000 UTC m=+758.951144222" watchObservedRunningTime="2026-07-30 03:47:33.335586807 +0000 UTC m=+758.953819049"
Jul 30 03:47:38.333200 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:38.333167    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:47:39.576570 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:39.576537    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:47:40.030126 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:40.030095    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 03:47:41.050493 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:41.050466    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk"
Jul 30 03:47:41.050973 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:41.050518    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk"
Jul 30 03:47:41.061919 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:41.061895    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk"
Jul 30 03:47:41.368666 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:47:41.368578    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk"
Jul 30 03:48:10.537842 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:10.537808    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-qwnzr"]
Jul 30 03:48:10.540667 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:10.540648    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qwnzr"
Jul 30 03:48:10.543455 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:10.543432    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 30 03:48:10.550099 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:10.550076    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-qwnzr"]
Jul 30 03:48:10.555696 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:10.555653    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:48:10.618089 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:10.618060    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58bb4966-2e49-491b-abde-744efcb0d35f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qwnzr\" (UID: \"58bb4966-2e49-491b-abde-744efcb0d35f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qwnzr"
Jul 30 03:48:10.618228 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:10.618100    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58bb4966-2e49-491b-abde-744efcb0d35f-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qwnzr\" (UID: \"58bb4966-2e49-491b-abde-744efcb0d35f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qwnzr"
Jul 30 03:48:10.618228 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:10.618122    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58bb4966-2e49-491b-abde-744efcb0d35f-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qwnzr\" (UID: \"58bb4966-2e49-491b-abde-744efcb0d35f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qwnzr"
Jul 30 03:48:10.618228 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:10.618217    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58bb4966-2e49-491b-abde-744efcb0d35f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qwnzr\" (UID: \"58bb4966-2e49-491b-abde-744efcb0d35f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qwnzr"
Jul 30 03:48:10.618333 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:10.618249    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhb8v\" (UniqueName: \"kubernetes.io/projected/58bb4966-2e49-491b-abde-744efcb0d35f-kube-api-access-lhb8v\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qwnzr\" (UID: \"58bb4966-2e49-491b-abde-744efcb0d35f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qwnzr"
Jul 30 03:48:10.618333 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:10.618315    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58bb4966-2e49-491b-abde-744efcb0d35f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qwnzr\" (UID: \"58bb4966-2e49-491b-abde-744efcb0d35f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qwnzr"
Jul 30 03:48:10.719372 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:10.719334    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58bb4966-2e49-491b-abde-744efcb0d35f-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qwnzr\" (UID: \"58bb4966-2e49-491b-abde-744efcb0d35f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qwnzr"
Jul 30 03:48:10.719533 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:10.719388    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58bb4966-2e49-491b-abde-744efcb0d35f-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qwnzr\" (UID: \"58bb4966-2e49-491b-abde-744efcb0d35f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qwnzr"
Jul 30 03:48:10.719533 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:10.719450    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58bb4966-2e49-491b-abde-744efcb0d35f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qwnzr\" (UID: \"58bb4966-2e49-491b-abde-744efcb0d35f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qwnzr"
Jul 30 03:48:10.719533 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:10.719484    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhb8v\" (UniqueName: \"kubernetes.io/projected/58bb4966-2e49-491b-abde-744efcb0d35f-kube-api-access-lhb8v\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qwnzr\" (UID: \"58bb4966-2e49-491b-abde-744efcb0d35f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qwnzr"
Jul 30 03:48:10.719660 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:10.719548    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58bb4966-2e49-491b-abde-744efcb0d35f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qwnzr\" (UID: \"58bb4966-2e49-491b-abde-744efcb0d35f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qwnzr"
Jul 30 03:48:10.719660 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:10.719599    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58bb4966-2e49-491b-abde-744efcb0d35f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qwnzr\" (UID: \"58bb4966-2e49-491b-abde-744efcb0d35f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qwnzr"
Jul 30 03:48:10.719835 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:10.719810    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58bb4966-2e49-491b-abde-744efcb0d35f-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qwnzr\" (UID: \"58bb4966-2e49-491b-abde-744efcb0d35f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qwnzr"
Jul 30 03:48:10.719946 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:10.719892    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58bb4966-2e49-491b-abde-744efcb0d35f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qwnzr\" (UID: \"58bb4966-2e49-491b-abde-744efcb0d35f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qwnzr"
Jul 30 03:48:10.720022 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:10.719982    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58bb4966-2e49-491b-abde-744efcb0d35f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qwnzr\" (UID: \"58bb4966-2e49-491b-abde-744efcb0d35f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qwnzr"
Jul 30 03:48:10.721553 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:10.721532    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58bb4966-2e49-491b-abde-744efcb0d35f-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qwnzr\" (UID: \"58bb4966-2e49-491b-abde-744efcb0d35f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qwnzr"
Jul 30 03:48:10.721906 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:10.721888    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58bb4966-2e49-491b-abde-744efcb0d35f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qwnzr\" (UID: \"58bb4966-2e49-491b-abde-744efcb0d35f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qwnzr"
Jul 30 03:48:10.727422 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:10.727402    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhb8v\" (UniqueName: \"kubernetes.io/projected/58bb4966-2e49-491b-abde-744efcb0d35f-kube-api-access-lhb8v\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qwnzr\" (UID: \"58bb4966-2e49-491b-abde-744efcb0d35f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qwnzr"
Jul 30 03:48:10.851828 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:10.851751    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qwnzr"
Jul 30 03:48:10.978894 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:10.978862    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-qwnzr"]
Jul 30 03:48:10.980052 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:48:10.980022    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58bb4966_2e49_491b_abde_744efcb0d35f.slice/crio-13d917cd75d0108ad943df22f3548d7d345389059519218dde1a0ee86b492704 WatchSource:0}: Error finding container 13d917cd75d0108ad943df22f3548d7d345389059519218dde1a0ee86b492704: Status 404 returned error can't find the container with id 13d917cd75d0108ad943df22f3548d7d345389059519218dde1a0ee86b492704
Jul 30 03:48:11.509799 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:11.509718    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qwnzr" event={"ID":"58bb4966-2e49-491b-abde-744efcb0d35f","Type":"ContainerStarted","Data":"26c1a100925a2baaeaba02258c4e54f5f68136a50dcdbf1ec7fe48a71f5d6aca"}
Jul 30 03:48:11.509799 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:11.509772    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qwnzr" event={"ID":"58bb4966-2e49-491b-abde-744efcb0d35f","Type":"ContainerStarted","Data":"13d917cd75d0108ad943df22f3548d7d345389059519218dde1a0ee86b492704"}
Jul 30 03:48:11.529749 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:11.529699    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qwnzr" podStartSLOduration=1.326351385 podStartE2EDuration="1.529684717s" totalImagesPullingTime="203.333332ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:48:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:48:11.527591536 +0000 UTC m=+797.145823800" watchObservedRunningTime="2026-07-30 03:48:11.529684717 +0000 UTC m=+797.147916958"
Jul 30 03:48:11.732451 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:11.732413    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 03:48:17.062899 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:17.062817    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:48:18.525349 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:18.525312    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 03:48:20.851986 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:20.851948    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qwnzr"
Jul 30 03:48:20.852401 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:20.852035    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qwnzr"
Jul 30 03:48:20.864836 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:20.864815    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qwnzr"
Jul 30 03:48:21.568606 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:21.568572    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qwnzr"
Jul 30 03:48:24.120020 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:24.119992    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:48:26.728817 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:26.728783    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 03:48:37.346854 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:37.346804    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:48:39.733202 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:39.733165    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk"]
Jul 30 03:48:39.736506 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:39.736491    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk"
Jul 30 03:48:39.739365 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:39.739342    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 30 03:48:39.748039 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:39.748015    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk"]
Jul 30 03:48:39.764996 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:39.764970    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:48:39.776162 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:39.776139    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6kjf\" (UniqueName: \"kubernetes.io/projected/a5f48781-5841-4e0c-946a-af5beac71c94-kube-api-access-d6kjf\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk\" (UID: \"a5f48781-5841-4e0c-946a-af5beac71c94\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk"
Jul 30 03:48:39.776251 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:39.776168    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5f48781-5841-4e0c-946a-af5beac71c94-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk\" (UID: \"a5f48781-5841-4e0c-946a-af5beac71c94\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk"
Jul 30 03:48:39.776251 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:39.776188    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5f48781-5841-4e0c-946a-af5beac71c94-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk\" (UID: \"a5f48781-5841-4e0c-946a-af5beac71c94\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk"
Jul 30 03:48:39.776251 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:39.776228    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5f48781-5841-4e0c-946a-af5beac71c94-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk\" (UID: \"a5f48781-5841-4e0c-946a-af5beac71c94\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk"
Jul 30 03:48:39.776359 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:39.776320    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5f48781-5841-4e0c-946a-af5beac71c94-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk\" (UID: \"a5f48781-5841-4e0c-946a-af5beac71c94\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk"
Jul 30 03:48:39.776359 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:39.776338    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5f48781-5841-4e0c-946a-af5beac71c94-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk\" (UID: \"a5f48781-5841-4e0c-946a-af5beac71c94\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk"
Jul 30 03:48:39.877609 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:39.877584    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5f48781-5841-4e0c-946a-af5beac71c94-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk\" (UID: \"a5f48781-5841-4e0c-946a-af5beac71c94\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk"
Jul 30 03:48:39.877743 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:39.877646    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6kjf\" (UniqueName: \"kubernetes.io/projected/a5f48781-5841-4e0c-946a-af5beac71c94-kube-api-access-d6kjf\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk\" (UID: \"a5f48781-5841-4e0c-946a-af5beac71c94\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk"
Jul 30 03:48:39.877791 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:39.877769    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5f48781-5841-4e0c-946a-af5beac71c94-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk\" (UID: \"a5f48781-5841-4e0c-946a-af5beac71c94\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk"
Jul 30 03:48:39.877845 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:39.877814    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5f48781-5841-4e0c-946a-af5beac71c94-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk\" (UID: \"a5f48781-5841-4e0c-946a-af5beac71c94\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk"
Jul 30 03:48:39.877891 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:39.877845    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5f48781-5841-4e0c-946a-af5beac71c94-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk\" (UID: \"a5f48781-5841-4e0c-946a-af5beac71c94\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk"
Jul 30 03:48:39.878139 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:39.877966    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5f48781-5841-4e0c-946a-af5beac71c94-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk\" (UID: \"a5f48781-5841-4e0c-946a-af5beac71c94\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk"
Jul 30 03:48:39.878139 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:39.878103    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5f48781-5841-4e0c-946a-af5beac71c94-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk\" (UID: \"a5f48781-5841-4e0c-946a-af5beac71c94\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk"
Jul 30 03:48:39.878350 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:39.878327    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5f48781-5841-4e0c-946a-af5beac71c94-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk\" (UID: \"a5f48781-5841-4e0c-946a-af5beac71c94\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk"
Jul 30 03:48:39.878594 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:39.878575    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5f48781-5841-4e0c-946a-af5beac71c94-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk\" (UID: \"a5f48781-5841-4e0c-946a-af5beac71c94\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk"
Jul 30 03:48:39.880835 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:39.880810    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5f48781-5841-4e0c-946a-af5beac71c94-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk\" (UID: \"a5f48781-5841-4e0c-946a-af5beac71c94\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk"
Jul 30 03:48:39.881013 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:39.880948    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5f48781-5841-4e0c-946a-af5beac71c94-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk\" (UID: \"a5f48781-5841-4e0c-946a-af5beac71c94\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk"
Jul 30 03:48:39.885505 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:39.885479    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6kjf\" (UniqueName: \"kubernetes.io/projected/a5f48781-5841-4e0c-946a-af5beac71c94-kube-api-access-d6kjf\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk\" (UID: \"a5f48781-5841-4e0c-946a-af5beac71c94\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk"
Jul 30 03:48:40.045450 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:40.045421    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk"
Jul 30 03:48:40.176072 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:40.176047    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk"]
Jul 30 03:48:40.177855 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:48:40.177826    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda5f48781_5841_4e0c_946a_af5beac71c94.slice/crio-cc83cb53e4a270ef6ad9a22e9ced77bad5e2e008b65e448624ecf08efb24237a WatchSource:0}: Error finding container cc83cb53e4a270ef6ad9a22e9ced77bad5e2e008b65e448624ecf08efb24237a: Status 404 returned error can't find the container with id cc83cb53e4a270ef6ad9a22e9ced77bad5e2e008b65e448624ecf08efb24237a
Jul 30 03:48:40.660777 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:40.660688    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk" event={"ID":"a5f48781-5841-4e0c-946a-af5beac71c94","Type":"ContainerStarted","Data":"c71c9479e4da43c057175a9c72895450765ce87bdf0797da3b689aa709863eec"}
Jul 30 03:48:40.660777 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:40.660729    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk" event={"ID":"a5f48781-5841-4e0c-946a-af5beac71c94","Type":"ContainerStarted","Data":"cc83cb53e4a270ef6ad9a22e9ced77bad5e2e008b65e448624ecf08efb24237a"}
Jul 30 03:48:40.681250 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:40.681207    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk" podStartSLOduration=1.504184795 podStartE2EDuration="1.681189669s" totalImagesPullingTime="177.004874ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:48:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:48:40.679133928 +0000 UTC m=+826.297366168" watchObservedRunningTime="2026-07-30 03:48:40.681189669 +0000 UTC m=+826.299421909"
Jul 30 03:48:42.236021 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:42.235975    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 03:48:49.135751 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:49.135706    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:48:50.045563 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:50.045508    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk"
Jul 30 03:48:50.045563 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:50.045565    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk"
Jul 30 03:48:50.058043 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:50.058021    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk"
Jul 30 03:48:50.723962 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:48:50.723918    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk"
Jul 30 03:49:36.031793 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:36.031738    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7ddbfffd7d-pgt8d"]
Jul 30 03:49:36.035222 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:36.035206    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7ddbfffd7d-pgt8d"
Jul 30 03:49:36.043820 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:36.043797    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7ddbfffd7d-pgt8d"]
Jul 30 03:49:36.192605 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:36.192561    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4slb\" (UniqueName: \"kubernetes.io/projected/d4249a5e-2ed9-44a7-ba88-793ebe1306c9-kube-api-access-q4slb\") pod \"authorino-7ddbfffd7d-pgt8d\" (UID: \"d4249a5e-2ed9-44a7-ba88-793ebe1306c9\") " pod="kuadrant-system/authorino-7ddbfffd7d-pgt8d"
Jul 30 03:49:36.192847 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:36.192628    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d4249a5e-2ed9-44a7-ba88-793ebe1306c9-tls-cert\") pod \"authorino-7ddbfffd7d-pgt8d\" (UID: \"d4249a5e-2ed9-44a7-ba88-793ebe1306c9\") " pod="kuadrant-system/authorino-7ddbfffd7d-pgt8d"
Jul 30 03:49:36.293200 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:36.293124    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4slb\" (UniqueName: \"kubernetes.io/projected/d4249a5e-2ed9-44a7-ba88-793ebe1306c9-kube-api-access-q4slb\") pod \"authorino-7ddbfffd7d-pgt8d\" (UID: \"d4249a5e-2ed9-44a7-ba88-793ebe1306c9\") " pod="kuadrant-system/authorino-7ddbfffd7d-pgt8d"
Jul 30 03:49:36.293200 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:36.293160    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d4249a5e-2ed9-44a7-ba88-793ebe1306c9-tls-cert\") pod \"authorino-7ddbfffd7d-pgt8d\" (UID: \"d4249a5e-2ed9-44a7-ba88-793ebe1306c9\") " pod="kuadrant-system/authorino-7ddbfffd7d-pgt8d"
Jul 30 03:49:36.295549 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:36.295524    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d4249a5e-2ed9-44a7-ba88-793ebe1306c9-tls-cert\") pod \"authorino-7ddbfffd7d-pgt8d\" (UID: \"d4249a5e-2ed9-44a7-ba88-793ebe1306c9\") " pod="kuadrant-system/authorino-7ddbfffd7d-pgt8d"
Jul 30 03:49:36.300768 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:36.300747    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4slb\" (UniqueName: \"kubernetes.io/projected/d4249a5e-2ed9-44a7-ba88-793ebe1306c9-kube-api-access-q4slb\") pod \"authorino-7ddbfffd7d-pgt8d\" (UID: \"d4249a5e-2ed9-44a7-ba88-793ebe1306c9\") " pod="kuadrant-system/authorino-7ddbfffd7d-pgt8d"
Jul 30 03:49:36.344428 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:36.344399    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7ddbfffd7d-pgt8d"
Jul 30 03:49:36.403566 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:36.403524    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:36.477651 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:36.477594    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7ddbfffd7d-pgt8d"]
Jul 30 03:49:36.479184 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:36.479155    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:36.480551 ip-10-0-143-117 kubenswrapper[2576]: W0730 03:49:36.480527    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd4249a5e_2ed9_44a7_ba88_793ebe1306c9.slice/crio-26e0d1213a823944f6f469f38618b6d06b9683d1b93ed8399ced2ae4c7b6d825 WatchSource:0}: Error finding container 26e0d1213a823944f6f469f38618b6d06b9683d1b93ed8399ced2ae4c7b6d825: Status 404 returned error can't find the container with id 26e0d1213a823944f6f469f38618b6d06b9683d1b93ed8399ced2ae4c7b6d825
Jul 30 03:49:36.948664 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:36.948634    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7ddbfffd7d-pgt8d" event={"ID":"d4249a5e-2ed9-44a7-ba88-793ebe1306c9","Type":"ContainerStarted","Data":"26e0d1213a823944f6f469f38618b6d06b9683d1b93ed8399ced2ae4c7b6d825"}
Jul 30 03:49:37.482758 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:37.482706    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:37.587654 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:37.587595    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:37.958237 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:37.958203    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7ddbfffd7d-pgt8d" event={"ID":"d4249a5e-2ed9-44a7-ba88-793ebe1306c9","Type":"ContainerStarted","Data":"1c179a1481fc1364480317a18e7e6189e420c5bb775994d1a599c7f5b121a87f"}
Jul 30 03:49:37.983859 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:37.983804    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7ddbfffd7d-pgt8d" podStartSLOduration=1.564037262 podStartE2EDuration="1.983786909s" totalImagesPullingTime="419.749647ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:49:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:49:37.980513395 +0000 UTC m=+883.598745636" watchObservedRunningTime="2026-07-30 03:49:37.983786909 +0000 UTC m=+883.602019150"
Jul 30 03:49:38.010255 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:38.010226    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-d4665d78c-dqsbg"]
Jul 30 03:49:38.010852 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:38.010485    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-d4665d78c-dqsbg" podUID="53a15a1e-1cf7-4923-a486-8d8965c7da98" containerName="authorino" containerID="cri-o://e20bde47d13382b3c5bee89e8fdbce66507a148d189f11585ee7d7b43696e8d6" gracePeriod=30
Jul 30 03:49:38.284251 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:38.284229    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d4665d78c-dqsbg"
Jul 30 03:49:38.413686 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:38.413651    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vj666\" (UniqueName: \"kubernetes.io/projected/53a15a1e-1cf7-4923-a486-8d8965c7da98-kube-api-access-vj666\") pod \"53a15a1e-1cf7-4923-a486-8d8965c7da98\" (UID: \"53a15a1e-1cf7-4923-a486-8d8965c7da98\") "
Jul 30 03:49:38.413872 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:38.413715    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/53a15a1e-1cf7-4923-a486-8d8965c7da98-tls-cert\") pod \"53a15a1e-1cf7-4923-a486-8d8965c7da98\" (UID: \"53a15a1e-1cf7-4923-a486-8d8965c7da98\") "
Jul 30 03:49:38.415823 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:38.415790    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53a15a1e-1cf7-4923-a486-8d8965c7da98-kube-api-access-vj666" (OuterVolumeSpecName: "kube-api-access-vj666") pod "53a15a1e-1cf7-4923-a486-8d8965c7da98" (UID: "53a15a1e-1cf7-4923-a486-8d8965c7da98"). InnerVolumeSpecName "kube-api-access-vj666". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 03:49:38.424478 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:38.424444    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53a15a1e-1cf7-4923-a486-8d8965c7da98-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "53a15a1e-1cf7-4923-a486-8d8965c7da98" (UID: "53a15a1e-1cf7-4923-a486-8d8965c7da98"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 03:49:38.514479 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:38.514405    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vj666\" (UniqueName: \"kubernetes.io/projected/53a15a1e-1cf7-4923-a486-8d8965c7da98-kube-api-access-vj666\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:49:38.514479 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:38.514428    2576 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/53a15a1e-1cf7-4923-a486-8d8965c7da98-tls-cert\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 03:49:38.964046 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:38.964009    2576 generic.go:356] "Generic (PLEG): container finished" podID="53a15a1e-1cf7-4923-a486-8d8965c7da98" containerID="e20bde47d13382b3c5bee89e8fdbce66507a148d189f11585ee7d7b43696e8d6" exitCode=0
Jul 30 03:49:38.964204 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:38.964054    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d4665d78c-dqsbg"
Jul 30 03:49:38.964204 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:38.964069    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d4665d78c-dqsbg" event={"ID":"53a15a1e-1cf7-4923-a486-8d8965c7da98","Type":"ContainerDied","Data":"e20bde47d13382b3c5bee89e8fdbce66507a148d189f11585ee7d7b43696e8d6"}
Jul 30 03:49:38.964204 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:38.964115    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d4665d78c-dqsbg" event={"ID":"53a15a1e-1cf7-4923-a486-8d8965c7da98","Type":"ContainerDied","Data":"6150a742a8d090902de0cd6682324a948544f498e5624c1d39550ad442bf95cd"}
Jul 30 03:49:38.964204 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:38.964134    2576 scope.go:117] "RemoveContainer" containerID="e20bde47d13382b3c5bee89e8fdbce66507a148d189f11585ee7d7b43696e8d6"
Jul 30 03:49:38.981505 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:38.981486    2576 scope.go:117] "RemoveContainer" containerID="e20bde47d13382b3c5bee89e8fdbce66507a148d189f11585ee7d7b43696e8d6"
Jul 30 03:49:38.981767 ip-10-0-143-117 kubenswrapper[2576]: E0730 03:49:38.981746    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e20bde47d13382b3c5bee89e8fdbce66507a148d189f11585ee7d7b43696e8d6\": container with ID starting with e20bde47d13382b3c5bee89e8fdbce66507a148d189f11585ee7d7b43696e8d6 not found: ID does not exist" containerID="e20bde47d13382b3c5bee89e8fdbce66507a148d189f11585ee7d7b43696e8d6"
Jul 30 03:49:38.981841 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:38.981775    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e20bde47d13382b3c5bee89e8fdbce66507a148d189f11585ee7d7b43696e8d6"} err="failed to get container status \"e20bde47d13382b3c5bee89e8fdbce66507a148d189f11585ee7d7b43696e8d6\": rpc error: code = NotFound desc = could not find container \"e20bde47d13382b3c5bee89e8fdbce66507a148d189f11585ee7d7b43696e8d6\": container with ID starting with e20bde47d13382b3c5bee89e8fdbce66507a148d189f11585ee7d7b43696e8d6 not found: ID does not exist"
Jul 30 03:49:38.994782 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:38.994759    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-d4665d78c-dqsbg"]
Jul 30 03:49:38.999952 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:38.999933    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-d4665d78c-dqsbg"]
Jul 30 03:49:39.051182 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:39.051154    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53a15a1e-1cf7-4923-a486-8d8965c7da98" path="/var/lib/kubelet/pods/53a15a1e-1cf7-4923-a486-8d8965c7da98/volumes"
Jul 30 03:49:52.262491 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:52.262408    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:49:53.420580 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:49:53.420527    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:50:04.029566 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:50:04.029529    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 03:50:15.126382 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:50:15.126345    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 03:50:25.432057 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:50:25.432013    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 03:50:52.862505 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:50:52.862467    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:50:52.959995 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:50:52.959960    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:51:41.465271 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:51:41.465235    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:51:42.653774 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:51:42.653740    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:51:45.786105 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:51:45.786067    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:52:07.623471 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:52:07.623433    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 03:52:15.738130 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:52:15.738092    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 03:52:55.586153 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:52:55.586079    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:52:56.883970 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:52:56.883918    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:52:56.982163 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:52:56.982130    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:52:57.081818 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:52:57.081782    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:53:02.926137 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:53:02.926105    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:53:02.986976 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:53:02.986943    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:53:03.060857 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:53:03.060826    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:53:03.160873 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:53:03.160837    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:53:03.262888 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:53:03.262806    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:53:08.913756 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:53:08.913721    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:53:09.013983 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:53:09.013954    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:53:25.924884 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:53:25.924845    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 03:53:35.523542 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:53:35.523509    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 03:53:50.636802 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:53:50.636770    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 03:54:00.022402 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:54:00.022372    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 03:54:08.633041 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:54:08.633007    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 03:54:18.227805 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:54:18.227708    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 03:54:30.326268 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:54:30.326234    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 03:54:40.233198 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:54:40.233164    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 03:55:02.425129 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:55:02.425093    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 03:55:13.922556 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:55:13.922517    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 03:55:35.327514 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:55:35.327479    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 03:55:48.822490 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:55:48.822407    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 03:56:03.232945 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:56:03.231761    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 03:56:23.125149 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:56:23.125114    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 03:56:29.959747 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:56:29.959710    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:56:38.133845 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:56:38.133805    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 03:56:38.769593 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:56:38.769561    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:56:38.870935 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:56:38.870891    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:56:46.523721 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:56:46.523685    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 03:56:58.431146 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:56:58.431107    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 03:57:07.626458 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:57:07.626423    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 03:57:15.228083 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:57:15.228054    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 03:57:37.825778 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:57:37.825690    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 03:57:47.133493 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:57:47.133462    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 03:57:55.424655 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:57:55.424613    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 03:59:59.705457 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:59:59.705421    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 03:59:59.760804 ip-10-0-143-117 kubenswrapper[2576]: I0730 03:59:59.760770    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:00:07.517388 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:00:07.517352    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:00:15.325241 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:00:15.325207    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:00:38.633319 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:00:38.633233    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:00:50.672151 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:00:50.672114    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:01:01.123174 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:01:01.123128    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:01:05.955571 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:01:05.955541    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:01:09.844385 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:01:09.844354    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:01:20.922048 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:01:20.922016    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:01:31.522395 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:01:31.522363    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:01:39.826337 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:01:39.826300    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:01:50.733720 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:01:50.733642    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:01:59.834749 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:01:59.834714    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:02:00.987853 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:00.987812    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:02:02.661544 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:02.661509    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:02:08.824511 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:08.824478    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:02:31.824394 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:31.824357    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:02:41.532710 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:41.532668    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:02:48.917402 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:48.917371    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt"]
Jul 30 04:02:48.917771 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:48.917731    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53a15a1e-1cf7-4923-a486-8d8965c7da98" containerName="authorino"
Jul 30 04:02:48.917771 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:48.917742    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="53a15a1e-1cf7-4923-a486-8d8965c7da98" containerName="authorino"
Jul 30 04:02:48.917847 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:48.917795    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="53a15a1e-1cf7-4923-a486-8d8965c7da98" containerName="authorino"
Jul 30 04:02:48.920743 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:48.920729    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt"
Jul 30 04:02:48.923451 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:48.923431    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-2c4e6901-kserve-self-signed-certs\""
Jul 30 04:02:48.937359 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:48.937339    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt"]
Jul 30 04:02:48.964941 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:48.964899    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:02:48.973091 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:48.973066    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50-home\") pod \"e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt\" (UID: \"9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50\") " pod="llm/e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt"
Jul 30 04:02:48.973208 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:48.973106    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50-tmp-dir\") pod \"e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt\" (UID: \"9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50\") " pod="llm/e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt"
Jul 30 04:02:48.973208 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:48.973175    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50-model-cache\") pod \"e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt\" (UID: \"9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50\") " pod="llm/e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt"
Jul 30 04:02:48.973324 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:48.973218    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50-tls-certs\") pod \"e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt\" (UID: \"9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50\") " pod="llm/e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt"
Jul 30 04:02:48.973324 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:48.973245    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wbzt\" (UniqueName: \"kubernetes.io/projected/9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50-kube-api-access-9wbzt\") pod \"e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt\" (UID: \"9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50\") " pod="llm/e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt"
Jul 30 04:02:48.973324 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:48.973281    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50-dshm\") pod \"e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt\" (UID: \"9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50\") " pod="llm/e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt"
Jul 30 04:02:49.073733 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:49.073701    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50-home\") pod \"e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt\" (UID: \"9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50\") " pod="llm/e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt"
Jul 30 04:02:49.073918 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:49.073739    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50-tmp-dir\") pod \"e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt\" (UID: \"9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50\") " pod="llm/e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt"
Jul 30 04:02:49.073918 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:49.073786    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50-model-cache\") pod \"e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt\" (UID: \"9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50\") " pod="llm/e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt"
Jul 30 04:02:49.073918 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:49.073820    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50-tls-certs\") pod \"e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt\" (UID: \"9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50\") " pod="llm/e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt"
Jul 30 04:02:49.074133 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:49.073957    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wbzt\" (UniqueName: \"kubernetes.io/projected/9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50-kube-api-access-9wbzt\") pod \"e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt\" (UID: \"9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50\") " pod="llm/e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt"
Jul 30 04:02:49.074133 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:49.074015    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50-dshm\") pod \"e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt\" (UID: \"9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50\") " pod="llm/e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt"
Jul 30 04:02:49.074133 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:49.074127    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50-home\") pod \"e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt\" (UID: \"9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50\") " pod="llm/e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt"
Jul 30 04:02:49.074310 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:49.074170    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50-model-cache\") pod \"e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt\" (UID: \"9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50\") " pod="llm/e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt"
Jul 30 04:02:49.074310 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:49.074191    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50-tmp-dir\") pod \"e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt\" (UID: \"9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50\") " pod="llm/e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt"
Jul 30 04:02:49.076125 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:49.076104    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50-dshm\") pod \"e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt\" (UID: \"9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50\") " pod="llm/e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt"
Jul 30 04:02:49.076340 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:49.076322    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50-tls-certs\") pod \"e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt\" (UID: \"9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50\") " pod="llm/e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt"
Jul 30 04:02:49.081406 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:49.081388    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wbzt\" (UniqueName: \"kubernetes.io/projected/9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50-kube-api-access-9wbzt\") pod \"e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt\" (UID: \"9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50\") " pod="llm/e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt"
Jul 30 04:02:49.229724 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:49.229643    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt"
Jul 30 04:02:49.356706 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:49.356681    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt"]
Jul 30 04:02:49.358806 ip-10-0-143-117 kubenswrapper[2576]: W0730 04:02:49.358782    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9855f5d2_9e39_4c40_81ed_c2d3dc2e8d50.slice/crio-2de4e6025a055fe927aaf889de380e0d1947dad95cf3837c7a465480c9a0766d WatchSource:0}: Error finding container 2de4e6025a055fe927aaf889de380e0d1947dad95cf3837c7a465480c9a0766d: Status 404 returned error can't find the container with id 2de4e6025a055fe927aaf889de380e0d1947dad95cf3837c7a465480c9a0766d
Jul 30 04:02:50.026714 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:50.026676    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt" event={"ID":"9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50","Type":"ContainerStarted","Data":"0925059b86b64fda62a6582c8268f64e6d2c2e606f62c4faa5ae86860fb19b69"}
Jul 30 04:02:50.026714 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:50.026716    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt" event={"ID":"9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50","Type":"ContainerStarted","Data":"2de4e6025a055fe927aaf889de380e0d1947dad95cf3837c7a465480c9a0766d"}
Jul 30 04:02:50.046859 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:50.046810    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt" podStartSLOduration=2.04679502 podStartE2EDuration="2.04679502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:02:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:02:50.045968526 +0000 UTC m=+1675.664200768" watchObservedRunningTime="2026-07-30 04:02:50.04679502 +0000 UTC m=+1675.665027261"
Jul 30 04:02:52.228537 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:52.228497    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:02:55.928379 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:55.928349    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g"]
Jul 30 04:02:55.931613 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:55.931596    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g"
Jul 30 04:02:55.935641 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:55.935613    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-2c4e6901-kserve-self-signed-certs\""
Jul 30 04:02:55.954240 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:55.954217    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g"]
Jul 30 04:02:56.035370 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:56.035342    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:02:56.041512 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:56.041490    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bcab1f49-1277-4824-9a91-92715e684cc3-tmp-dir\") pod \"e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g\" (UID: \"bcab1f49-1277-4824-9a91-92715e684cc3\") " pod="llm/e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g"
Jul 30 04:02:56.041593 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:56.041534    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbxkq\" (UniqueName: \"kubernetes.io/projected/bcab1f49-1277-4824-9a91-92715e684cc3-kube-api-access-qbxkq\") pod \"e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g\" (UID: \"bcab1f49-1277-4824-9a91-92715e684cc3\") " pod="llm/e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g"
Jul 30 04:02:56.041593 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:56.041564    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bcab1f49-1277-4824-9a91-92715e684cc3-dshm\") pod \"e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g\" (UID: \"bcab1f49-1277-4824-9a91-92715e684cc3\") " pod="llm/e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g"
Jul 30 04:02:56.041593 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:56.041585    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bcab1f49-1277-4824-9a91-92715e684cc3-tls-certs\") pod \"e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g\" (UID: \"bcab1f49-1277-4824-9a91-92715e684cc3\") " pod="llm/e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g"
Jul 30 04:02:56.041740 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:56.041631    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bcab1f49-1277-4824-9a91-92715e684cc3-home\") pod \"e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g\" (UID: \"bcab1f49-1277-4824-9a91-92715e684cc3\") " pod="llm/e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g"
Jul 30 04:02:56.041740 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:56.041705    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bcab1f49-1277-4824-9a91-92715e684cc3-model-cache\") pod \"e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g\" (UID: \"bcab1f49-1277-4824-9a91-92715e684cc3\") " pod="llm/e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g"
Jul 30 04:02:56.142226 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:56.142189    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bcab1f49-1277-4824-9a91-92715e684cc3-tmp-dir\") pod \"e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g\" (UID: \"bcab1f49-1277-4824-9a91-92715e684cc3\") " pod="llm/e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g"
Jul 30 04:02:56.142408 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:56.142241    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbxkq\" (UniqueName: \"kubernetes.io/projected/bcab1f49-1277-4824-9a91-92715e684cc3-kube-api-access-qbxkq\") pod \"e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g\" (UID: \"bcab1f49-1277-4824-9a91-92715e684cc3\") " pod="llm/e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g"
Jul 30 04:02:56.142408 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:56.142266    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bcab1f49-1277-4824-9a91-92715e684cc3-dshm\") pod \"e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g\" (UID: \"bcab1f49-1277-4824-9a91-92715e684cc3\") " pod="llm/e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g"
Jul 30 04:02:56.142408 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:56.142283    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bcab1f49-1277-4824-9a91-92715e684cc3-tls-certs\") pod \"e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g\" (UID: \"bcab1f49-1277-4824-9a91-92715e684cc3\") " pod="llm/e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g"
Jul 30 04:02:56.142408 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:56.142351    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bcab1f49-1277-4824-9a91-92715e684cc3-home\") pod \"e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g\" (UID: \"bcab1f49-1277-4824-9a91-92715e684cc3\") " pod="llm/e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g"
Jul 30 04:02:56.142408 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:56.142377    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bcab1f49-1277-4824-9a91-92715e684cc3-model-cache\") pod \"e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g\" (UID: \"bcab1f49-1277-4824-9a91-92715e684cc3\") " pod="llm/e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g"
Jul 30 04:02:56.142685 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:56.142632    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bcab1f49-1277-4824-9a91-92715e684cc3-tmp-dir\") pod \"e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g\" (UID: \"bcab1f49-1277-4824-9a91-92715e684cc3\") " pod="llm/e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g"
Jul 30 04:02:56.142786 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:56.142762    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bcab1f49-1277-4824-9a91-92715e684cc3-home\") pod \"e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g\" (UID: \"bcab1f49-1277-4824-9a91-92715e684cc3\") " pod="llm/e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g"
Jul 30 04:02:56.142918 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:56.142901    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bcab1f49-1277-4824-9a91-92715e684cc3-model-cache\") pod \"e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g\" (UID: \"bcab1f49-1277-4824-9a91-92715e684cc3\") " pod="llm/e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g"
Jul 30 04:02:56.144499 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:56.144467    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bcab1f49-1277-4824-9a91-92715e684cc3-dshm\") pod \"e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g\" (UID: \"bcab1f49-1277-4824-9a91-92715e684cc3\") " pod="llm/e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g"
Jul 30 04:02:56.144847 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:56.144823    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bcab1f49-1277-4824-9a91-92715e684cc3-tls-certs\") pod \"e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g\" (UID: \"bcab1f49-1277-4824-9a91-92715e684cc3\") " pod="llm/e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g"
Jul 30 04:02:56.150724 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:56.150701    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbxkq\" (UniqueName: \"kubernetes.io/projected/bcab1f49-1277-4824-9a91-92715e684cc3-kube-api-access-qbxkq\") pod \"e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g\" (UID: \"bcab1f49-1277-4824-9a91-92715e684cc3\") " pod="llm/e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g"
Jul 30 04:02:56.242611 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:56.242520    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g"
Jul 30 04:02:56.377346 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:56.377323    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g"]
Jul 30 04:02:56.379311 ip-10-0-143-117 kubenswrapper[2576]: W0730 04:02:56.379277    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbcab1f49_1277_4824_9a91_92715e684cc3.slice/crio-de86ff549478d6fa7ae95337ed56e9f3ccdd92bdab268af3dcfa60601aaace77 WatchSource:0}: Error finding container de86ff549478d6fa7ae95337ed56e9f3ccdd92bdab268af3dcfa60601aaace77: Status 404 returned error can't find the container with id de86ff549478d6fa7ae95337ed56e9f3ccdd92bdab268af3dcfa60601aaace77
Jul 30 04:02:57.065380 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:57.065342    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g" event={"ID":"bcab1f49-1277-4824-9a91-92715e684cc3","Type":"ContainerStarted","Data":"6bdf9de67184312e264f918cc84dab4320c137fd5a421bdc1389a9cb436fcdd4"}
Jul 30 04:02:57.065873 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:57.065391    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g" event={"ID":"bcab1f49-1277-4824-9a91-92715e684cc3","Type":"ContainerStarted","Data":"de86ff549478d6fa7ae95337ed56e9f3ccdd92bdab268af3dcfa60601aaace77"}
Jul 30 04:02:57.083100 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:57.083050    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g" podStartSLOduration=2.083035129 podStartE2EDuration="2.083035129s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:02:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:02:57.080909653 +0000 UTC m=+1682.699141895" watchObservedRunningTime="2026-07-30 04:02:57.083035129 +0000 UTC m=+1682.701267369"
Jul 30 04:02:59.230396 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:59.230311    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt"
Jul 30 04:02:59.230739 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:59.230401    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt"
Jul 30 04:02:59.242677 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:02:59.242653    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt"
Jul 30 04:03:00.091679 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:00.091641    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt"
Jul 30 04:03:05.946641 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:05.946610    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:03:06.242713 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:06.242619    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g"
Jul 30 04:03:06.242713 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:06.242678    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g"
Jul 30 04:03:06.255078 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:06.255055    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g"
Jul 30 04:03:07.126183 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:07.126148    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g"
Jul 30 04:03:17.245106 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:17.245018    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:03:18.615878 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:18.615845    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt"]
Jul 30 04:03:18.636563 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:18.636537    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:03:19.700829 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:19.700795    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g"]
Jul 30 04:03:19.714634 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:19.714608    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:03:22.949853 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:22.949815    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:03:33.635240 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:33.635174    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt" podUID="9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50" containerName="main" containerID="cri-o://0925059b86b64fda62a6582c8268f64e6d2c2e606f62c4faa5ae86860fb19b69" gracePeriod=45
Jul 30 04:03:33.879292 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:33.879269    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt"
Jul 30 04:03:33.975020 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:33.974918    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50-tmp-dir\") pod \"9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50\" (UID: \"9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50\") "
Jul 30 04:03:33.975020 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:33.974983    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50-home\") pod \"9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50\" (UID: \"9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50\") "
Jul 30 04:03:33.975242 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:33.975032    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50-dshm\") pod \"9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50\" (UID: \"9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50\") "
Jul 30 04:03:33.975242 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:33.975078    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50-tls-certs\") pod \"9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50\" (UID: \"9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50\") "
Jul 30 04:03:33.975242 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:33.975111    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50-model-cache\") pod \"9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50\" (UID: \"9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50\") "
Jul 30 04:03:33.975242 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:33.975142    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9wbzt\" (UniqueName: \"kubernetes.io/projected/9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50-kube-api-access-9wbzt\") pod \"9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50\" (UID: \"9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50\") "
Jul 30 04:03:33.975242 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:33.975214    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50" (UID: "9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:03:33.975519 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:33.975268    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50-home" (OuterVolumeSpecName: "home") pod "9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50" (UID: "9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:03:33.975519 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:33.975368    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50-model-cache" (OuterVolumeSpecName: "model-cache") pod "9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50" (UID: "9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:03:33.975519 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:33.975458    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50-model-cache\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:03:33.975519 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:33.975473    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50-tmp-dir\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:03:33.975519 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:33.975482    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50-home\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:03:33.977350 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:33.977321    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50-dshm" (OuterVolumeSpecName: "dshm") pod "9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50" (UID: "9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:03:33.977476 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:33.977359    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50-kube-api-access-9wbzt" (OuterVolumeSpecName: "kube-api-access-9wbzt") pod "9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50" (UID: "9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50"). InnerVolumeSpecName "kube-api-access-9wbzt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:03:33.977476 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:33.977393    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50" (UID: "9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 04:03:34.076618 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:34.076580    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50-dshm\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:03:34.076618 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:34.076611    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50-tls-certs\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:03:34.076618 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:34.076623    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9wbzt\" (UniqueName: \"kubernetes.io/projected/9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50-kube-api-access-9wbzt\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:03:34.260989 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:34.260864    2576 generic.go:356] "Generic (PLEG): container finished" podID="9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50" containerID="0925059b86b64fda62a6582c8268f64e6d2c2e606f62c4faa5ae86860fb19b69" exitCode=0
Jul 30 04:03:34.260989 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:34.260910    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt"
Jul 30 04:03:34.260989 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:34.260948    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt" event={"ID":"9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50","Type":"ContainerDied","Data":"0925059b86b64fda62a6582c8268f64e6d2c2e606f62c4faa5ae86860fb19b69"}
Jul 30 04:03:34.260989 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:34.260990    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt" event={"ID":"9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50","Type":"ContainerDied","Data":"2de4e6025a055fe927aaf889de380e0d1947dad95cf3837c7a465480c9a0766d"}
Jul 30 04:03:34.261350 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:34.261010    2576 scope.go:117] "RemoveContainer" containerID="0925059b86b64fda62a6582c8268f64e6d2c2e606f62c4faa5ae86860fb19b69"
Jul 30 04:03:34.270221 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:34.270201    2576 scope.go:117] "RemoveContainer" containerID="0925059b86b64fda62a6582c8268f64e6d2c2e606f62c4faa5ae86860fb19b69"
Jul 30 04:03:34.270471 ip-10-0-143-117 kubenswrapper[2576]: E0730 04:03:34.270454    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0925059b86b64fda62a6582c8268f64e6d2c2e606f62c4faa5ae86860fb19b69\": container with ID starting with 0925059b86b64fda62a6582c8268f64e6d2c2e606f62c4faa5ae86860fb19b69 not found: ID does not exist" containerID="0925059b86b64fda62a6582c8268f64e6d2c2e606f62c4faa5ae86860fb19b69"
Jul 30 04:03:34.270522 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:34.270479    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0925059b86b64fda62a6582c8268f64e6d2c2e606f62c4faa5ae86860fb19b69"} err="failed to get container status \"0925059b86b64fda62a6582c8268f64e6d2c2e606f62c4faa5ae86860fb19b69\": rpc error: code = NotFound desc = could not find container \"0925059b86b64fda62a6582c8268f64e6d2c2e606f62c4faa5ae86860fb19b69\": container with ID starting with 0925059b86b64fda62a6582c8268f64e6d2c2e606f62c4faa5ae86860fb19b69 not found: ID does not exist"
Jul 30 04:03:34.282359 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:34.282335    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt"]
Jul 30 04:03:34.285763 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:34.285743    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-2c4e6901-kserve-796788c49f-f4rmt"]
Jul 30 04:03:34.719904 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:34.719860    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g" podUID="bcab1f49-1277-4824-9a91-92715e684cc3" containerName="main" containerID="cri-o://6bdf9de67184312e264f918cc84dab4320c137fd5a421bdc1389a9cb436fcdd4" gracePeriod=45
Jul 30 04:03:34.968011 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:34.967984    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g"
Jul 30 04:03:35.052965 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:35.052910    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50" path="/var/lib/kubelet/pods/9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50/volumes"
Jul 30 04:03:35.084662 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:35.084636    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bcab1f49-1277-4824-9a91-92715e684cc3-tmp-dir\") pod \"bcab1f49-1277-4824-9a91-92715e684cc3\" (UID: \"bcab1f49-1277-4824-9a91-92715e684cc3\") "
Jul 30 04:03:35.084662 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:35.084665    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bcab1f49-1277-4824-9a91-92715e684cc3-home\") pod \"bcab1f49-1277-4824-9a91-92715e684cc3\" (UID: \"bcab1f49-1277-4824-9a91-92715e684cc3\") "
Jul 30 04:03:35.084859 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:35.084694    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bcab1f49-1277-4824-9a91-92715e684cc3-model-cache\") pod \"bcab1f49-1277-4824-9a91-92715e684cc3\" (UID: \"bcab1f49-1277-4824-9a91-92715e684cc3\") "
Jul 30 04:03:35.084859 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:35.084716    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bcab1f49-1277-4824-9a91-92715e684cc3-dshm\") pod \"bcab1f49-1277-4824-9a91-92715e684cc3\" (UID: \"bcab1f49-1277-4824-9a91-92715e684cc3\") "
Jul 30 04:03:35.084859 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:35.084758    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qbxkq\" (UniqueName: \"kubernetes.io/projected/bcab1f49-1277-4824-9a91-92715e684cc3-kube-api-access-qbxkq\") pod \"bcab1f49-1277-4824-9a91-92715e684cc3\" (UID: \"bcab1f49-1277-4824-9a91-92715e684cc3\") "
Jul 30 04:03:35.084859 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:35.084810    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bcab1f49-1277-4824-9a91-92715e684cc3-tls-certs\") pod \"bcab1f49-1277-4824-9a91-92715e684cc3\" (UID: \"bcab1f49-1277-4824-9a91-92715e684cc3\") "
Jul 30 04:03:35.085107 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:35.084912    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcab1f49-1277-4824-9a91-92715e684cc3-model-cache" (OuterVolumeSpecName: "model-cache") pod "bcab1f49-1277-4824-9a91-92715e684cc3" (UID: "bcab1f49-1277-4824-9a91-92715e684cc3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:03:35.085107 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:35.084957    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcab1f49-1277-4824-9a91-92715e684cc3-home" (OuterVolumeSpecName: "home") pod "bcab1f49-1277-4824-9a91-92715e684cc3" (UID: "bcab1f49-1277-4824-9a91-92715e684cc3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:03:35.085107 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:35.084966    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcab1f49-1277-4824-9a91-92715e684cc3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bcab1f49-1277-4824-9a91-92715e684cc3" (UID: "bcab1f49-1277-4824-9a91-92715e684cc3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:03:35.085247 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:35.085136    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bcab1f49-1277-4824-9a91-92715e684cc3-tmp-dir\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:03:35.085247 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:35.085158    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bcab1f49-1277-4824-9a91-92715e684cc3-home\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:03:35.085247 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:35.085172    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bcab1f49-1277-4824-9a91-92715e684cc3-model-cache\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:03:35.086958 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:35.086934    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcab1f49-1277-4824-9a91-92715e684cc3-dshm" (OuterVolumeSpecName: "dshm") pod "bcab1f49-1277-4824-9a91-92715e684cc3" (UID: "bcab1f49-1277-4824-9a91-92715e684cc3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:03:35.086958 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:35.086942    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bcab1f49-1277-4824-9a91-92715e684cc3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bcab1f49-1277-4824-9a91-92715e684cc3" (UID: "bcab1f49-1277-4824-9a91-92715e684cc3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 04:03:35.087124 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:35.086956    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcab1f49-1277-4824-9a91-92715e684cc3-kube-api-access-qbxkq" (OuterVolumeSpecName: "kube-api-access-qbxkq") pod "bcab1f49-1277-4824-9a91-92715e684cc3" (UID: "bcab1f49-1277-4824-9a91-92715e684cc3"). InnerVolumeSpecName "kube-api-access-qbxkq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:03:35.186331 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:35.186286    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbxkq\" (UniqueName: \"kubernetes.io/projected/bcab1f49-1277-4824-9a91-92715e684cc3-kube-api-access-qbxkq\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:03:35.186331 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:35.186327    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bcab1f49-1277-4824-9a91-92715e684cc3-tls-certs\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:03:35.186331 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:35.186340    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bcab1f49-1277-4824-9a91-92715e684cc3-dshm\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:03:35.268274 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:35.268241    2576 generic.go:356] "Generic (PLEG): container finished" podID="bcab1f49-1277-4824-9a91-92715e684cc3" containerID="6bdf9de67184312e264f918cc84dab4320c137fd5a421bdc1389a9cb436fcdd4" exitCode=0
Jul 30 04:03:35.268434 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:35.268305    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g"
Jul 30 04:03:35.268434 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:35.268329    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g" event={"ID":"bcab1f49-1277-4824-9a91-92715e684cc3","Type":"ContainerDied","Data":"6bdf9de67184312e264f918cc84dab4320c137fd5a421bdc1389a9cb436fcdd4"}
Jul 30 04:03:35.268434 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:35.268378    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g" event={"ID":"bcab1f49-1277-4824-9a91-92715e684cc3","Type":"ContainerDied","Data":"de86ff549478d6fa7ae95337ed56e9f3ccdd92bdab268af3dcfa60601aaace77"}
Jul 30 04:03:35.268434 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:35.268395    2576 scope.go:117] "RemoveContainer" containerID="6bdf9de67184312e264f918cc84dab4320c137fd5a421bdc1389a9cb436fcdd4"
Jul 30 04:03:35.277702 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:35.277688    2576 scope.go:117] "RemoveContainer" containerID="6bdf9de67184312e264f918cc84dab4320c137fd5a421bdc1389a9cb436fcdd4"
Jul 30 04:03:35.277953 ip-10-0-143-117 kubenswrapper[2576]: E0730 04:03:35.277917    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bdf9de67184312e264f918cc84dab4320c137fd5a421bdc1389a9cb436fcdd4\": container with ID starting with 6bdf9de67184312e264f918cc84dab4320c137fd5a421bdc1389a9cb436fcdd4 not found: ID does not exist" containerID="6bdf9de67184312e264f918cc84dab4320c137fd5a421bdc1389a9cb436fcdd4"
Jul 30 04:03:35.278005 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:35.277962    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bdf9de67184312e264f918cc84dab4320c137fd5a421bdc1389a9cb436fcdd4"} err="failed to get container status \"6bdf9de67184312e264f918cc84dab4320c137fd5a421bdc1389a9cb436fcdd4\": rpc error: code = NotFound desc = could not find container \"6bdf9de67184312e264f918cc84dab4320c137fd5a421bdc1389a9cb436fcdd4\": container with ID starting with 6bdf9de67184312e264f918cc84dab4320c137fd5a421bdc1389a9cb436fcdd4 not found: ID does not exist"
Jul 30 04:03:35.289670 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:35.289649    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g"]
Jul 30 04:03:35.293346 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:35.293326    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-2c4e6901-kserve-7ddf97459-2j92g"]
Jul 30 04:03:37.035228 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:37.035188    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:03:37.052083 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:37.052045    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcab1f49-1277-4824-9a91-92715e684cc3" path="/var/lib/kubelet/pods/bcab1f49-1277-4824-9a91-92715e684cc3/volumes"
Jul 30 04:03:48.133756 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:48.133721    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:03:59.938349 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:03:59.938311    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:04:12.835984 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:04:12.835950    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:04:23.922171 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:04:23.922137    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:04:34.227455 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:04:34.227409    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:04:43.421075 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:04:43.421039    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:04:53.222382 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:04:53.222349    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:05:05.924843 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:05:05.924805    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:05:15.129128 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:05:15.129084    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:05:24.930126 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:05:24.930088    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:05:35.227996 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:05:35.227959    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:05:46.589985 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:05:46.589944    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:05:46.640678 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:05:46.640641    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:05:46.920438 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:05:46.920359    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:05:56.238383 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:05:56.238339    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:06:18.725817 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:06:18.725722    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:06:31.223191 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:06:31.223156    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:06:46.222524 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:06:46.222486    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:07:01.018295 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:01.018258    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:07:10.228324 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:10.228288    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:07:12.121333 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:12.121304    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:07:23.024098 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:23.024063    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:07:37.928602 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:37.928570    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:07:47.720688 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:47.720608    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:07:48.385514 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:48.385480    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:07:51.830655 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:51.830620    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:07:51.879838 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:51.879802    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:07:57.527714 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:57.527668    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:07:57.578880 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:57.578848    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:07:57.740324 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:57.740289    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:07:58.005993 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.005898    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"]
Jul 30 04:07:58.007022 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.006998    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcab1f49-1277-4824-9a91-92715e684cc3" containerName="main"
Jul 30 04:07:58.007154 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.007143    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcab1f49-1277-4824-9a91-92715e684cc3" containerName="main"
Jul 30 04:07:58.007254 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.007244    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50" containerName="main"
Jul 30 04:07:58.007324 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.007315    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50" containerName="main"
Jul 30 04:07:58.007618 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.007604    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcab1f49-1277-4824-9a91-92715e684cc3" containerName="main"
Jul 30 04:07:58.007712 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.007703    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9855f5d2-9e39-4c40-81ed-c2d3dc2e8d50" containerName="main"
Jul 30 04:07:58.013024 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.013001    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:07:58.015772 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.015750    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-0a082c3b-openshift-default-dockercfg-2jfkd\""
Jul 30 04:07:58.026435 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.026412    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"]
Jul 30 04:07:58.030728 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.030703    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:07:58.062532 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.062506    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75cmr\" (UniqueName: \"kubernetes.io/projected/94f98593-6e2c-4605-8f12-cc0ec0b05546-kube-api-access-75cmr\") pod \"e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw\" (UID: \"94f98593-6e2c-4605-8f12-cc0ec0b05546\") " pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:07:58.062643 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.062596    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94f98593-6e2c-4605-8f12-cc0ec0b05546-istiod-ca-cert\") pod \"e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw\" (UID: \"94f98593-6e2c-4605-8f12-cc0ec0b05546\") " pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:07:58.062700 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.062655    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94f98593-6e2c-4605-8f12-cc0ec0b05546-istio-podinfo\") pod \"e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw\" (UID: \"94f98593-6e2c-4605-8f12-cc0ec0b05546\") " pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:07:58.062745 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.062711    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94f98593-6e2c-4605-8f12-cc0ec0b05546-istio-envoy\") pod \"e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw\" (UID: \"94f98593-6e2c-4605-8f12-cc0ec0b05546\") " pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:07:58.062745 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.062741    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94f98593-6e2c-4605-8f12-cc0ec0b05546-istio-token\") pod \"e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw\" (UID: \"94f98593-6e2c-4605-8f12-cc0ec0b05546\") " pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:07:58.062831 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.062769    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94f98593-6e2c-4605-8f12-cc0ec0b05546-istio-data\") pod \"e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw\" (UID: \"94f98593-6e2c-4605-8f12-cc0ec0b05546\") " pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:07:58.062831 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.062810    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94f98593-6e2c-4605-8f12-cc0ec0b05546-credential-socket\") pod \"e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw\" (UID: \"94f98593-6e2c-4605-8f12-cc0ec0b05546\") " pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:07:58.062982 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.062855    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94f98593-6e2c-4605-8f12-cc0ec0b05546-workload-socket\") pod \"e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw\" (UID: \"94f98593-6e2c-4605-8f12-cc0ec0b05546\") " pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:07:58.062982 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.062877    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94f98593-6e2c-4605-8f12-cc0ec0b05546-workload-certs\") pod \"e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw\" (UID: \"94f98593-6e2c-4605-8f12-cc0ec0b05546\") " pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:07:58.141669 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.141638    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:07:58.163563 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.163536    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94f98593-6e2c-4605-8f12-cc0ec0b05546-istio-envoy\") pod \"e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw\" (UID: \"94f98593-6e2c-4605-8f12-cc0ec0b05546\") " pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:07:58.163710 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.163570    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94f98593-6e2c-4605-8f12-cc0ec0b05546-istio-token\") pod \"e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw\" (UID: \"94f98593-6e2c-4605-8f12-cc0ec0b05546\") " pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:07:58.163710 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.163595    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94f98593-6e2c-4605-8f12-cc0ec0b05546-istio-data\") pod \"e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw\" (UID: \"94f98593-6e2c-4605-8f12-cc0ec0b05546\") " pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:07:58.163710 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.163629    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94f98593-6e2c-4605-8f12-cc0ec0b05546-credential-socket\") pod \"e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw\" (UID: \"94f98593-6e2c-4605-8f12-cc0ec0b05546\") " pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:07:58.163710 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.163650    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94f98593-6e2c-4605-8f12-cc0ec0b05546-workload-socket\") pod \"e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw\" (UID: \"94f98593-6e2c-4605-8f12-cc0ec0b05546\") " pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:07:58.163710 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.163668    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94f98593-6e2c-4605-8f12-cc0ec0b05546-workload-certs\") pod \"e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw\" (UID: \"94f98593-6e2c-4605-8f12-cc0ec0b05546\") " pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:07:58.163710 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.163692    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75cmr\" (UniqueName: \"kubernetes.io/projected/94f98593-6e2c-4605-8f12-cc0ec0b05546-kube-api-access-75cmr\") pod \"e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw\" (UID: \"94f98593-6e2c-4605-8f12-cc0ec0b05546\") " pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:07:58.164053 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.163755    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94f98593-6e2c-4605-8f12-cc0ec0b05546-istiod-ca-cert\") pod \"e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw\" (UID: \"94f98593-6e2c-4605-8f12-cc0ec0b05546\") " pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:07:58.164053 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.163797    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94f98593-6e2c-4605-8f12-cc0ec0b05546-istio-podinfo\") pod \"e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw\" (UID: \"94f98593-6e2c-4605-8f12-cc0ec0b05546\") " pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:07:58.164175 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.164053    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94f98593-6e2c-4605-8f12-cc0ec0b05546-workload-socket\") pod \"e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw\" (UID: \"94f98593-6e2c-4605-8f12-cc0ec0b05546\") " pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:07:58.164236 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.164186    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94f98593-6e2c-4605-8f12-cc0ec0b05546-credential-socket\") pod \"e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw\" (UID: \"94f98593-6e2c-4605-8f12-cc0ec0b05546\") " pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:07:58.164294 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.164260    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94f98593-6e2c-4605-8f12-cc0ec0b05546-istio-data\") pod \"e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw\" (UID: \"94f98593-6e2c-4605-8f12-cc0ec0b05546\") " pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:07:58.164356 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.164336    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94f98593-6e2c-4605-8f12-cc0ec0b05546-workload-certs\") pod \"e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw\" (UID: \"94f98593-6e2c-4605-8f12-cc0ec0b05546\") " pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:07:58.164491 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.164476    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94f98593-6e2c-4605-8f12-cc0ec0b05546-istiod-ca-cert\") pod \"e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw\" (UID: \"94f98593-6e2c-4605-8f12-cc0ec0b05546\") " pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:07:58.165791 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.165768    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94f98593-6e2c-4605-8f12-cc0ec0b05546-istio-envoy\") pod \"e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw\" (UID: \"94f98593-6e2c-4605-8f12-cc0ec0b05546\") " pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:07:58.166286 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.166269    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94f98593-6e2c-4605-8f12-cc0ec0b05546-istio-podinfo\") pod \"e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw\" (UID: \"94f98593-6e2c-4605-8f12-cc0ec0b05546\") " pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:07:58.172020 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.171999    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94f98593-6e2c-4605-8f12-cc0ec0b05546-istio-token\") pod \"e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw\" (UID: \"94f98593-6e2c-4605-8f12-cc0ec0b05546\") " pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:07:58.172463 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.172447    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75cmr\" (UniqueName: \"kubernetes.io/projected/94f98593-6e2c-4605-8f12-cc0ec0b05546-kube-api-access-75cmr\") pod \"e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw\" (UID: \"94f98593-6e2c-4605-8f12-cc0ec0b05546\") " pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:07:58.323002 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.322964    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:07:58.457916 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.457885    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"]
Jul 30 04:07:58.459233 ip-10-0-143-117 kubenswrapper[2576]: W0730 04:07:58.459204    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94f98593_6e2c_4605_8f12_cc0ec0b05546.slice/crio-a2ea695617fee485afb6e56c3ac26ddc84c28b213c5a5de1183e17143478ccd5 WatchSource:0}: Error finding container a2ea695617fee485afb6e56c3ac26ddc84c28b213c5a5de1183e17143478ccd5: Status 404 returned error can't find the container with id a2ea695617fee485afb6e56c3ac26ddc84c28b213c5a5de1183e17143478ccd5
Jul 30 04:07:58.461385 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.461357    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:07:58.461495 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.461446    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:07:58.461558 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.461493    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:07:58.529569 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.529544    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:07:58.639339 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.639249    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw" event={"ID":"94f98593-6e2c-4605-8f12-cc0ec0b05546","Type":"ContainerStarted","Data":"9bdcbb4f93d05f35afd57985720d4ca08d6318ae3477b4cce17750d841ad77c8"}
Jul 30 04:07:58.639339 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.639308    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw" event={"ID":"94f98593-6e2c-4605-8f12-cc0ec0b05546","Type":"ContainerStarted","Data":"a2ea695617fee485afb6e56c3ac26ddc84c28b213c5a5de1183e17143478ccd5"}
Jul 30 04:07:58.657526 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:58.657480    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw" podStartSLOduration=1.6574642179999999 podStartE2EDuration="1.657464218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:07:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:07:58.656440994 +0000 UTC m=+1984.274673246" watchObservedRunningTime="2026-07-30 04:07:58.657464218 +0000 UTC m=+1984.275696459"
Jul 30 04:07:59.323388 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:59.323349    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:07:59.328285 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:59.328262    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:07:59.645038 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:59.644954    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:07:59.645779 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:07:59.645762    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:08:00.003553 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:00.003469    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:08:00.104292 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:00.104256    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:08:01.199858 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:01.199816    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:08:01.314237 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:01.314201    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:08:01.645860 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:01.645826    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:08:01.749267 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:01.749237    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:08:01.750640 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:01.750621    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0a082c3b-86c79d7cc6-wzmb7"]
Jul 30 04:08:01.754111 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:01.754097    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0a082c3b-86c79d7cc6-wzmb7"
Jul 30 04:08:01.756636 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:01.756614    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-0a082c3b-dockercfg-prrmj\""
Jul 30 04:08:01.756846 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:01.756657    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-0a082c3b\""
Jul 30 04:08:01.764420 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:01.764396    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0a082c3b-86c79d7cc6-wzmb7"]
Jul 30 04:08:01.799014 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:01.798980    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2zmm\" (UniqueName: \"kubernetes.io/projected/a4910631-a2cf-4461-b30b-e6297898dcb5-kube-api-access-c2zmm\") pod \"payload-pre-processing-e2e-ait-0a082c3b-86c79d7cc6-wzmb7\" (UID: \"a4910631-a2cf-4461-b30b-e6297898dcb5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0a082c3b-86c79d7cc6-wzmb7"
Jul 30 04:08:01.799197 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:01.799176    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4910631-a2cf-4461-b30b-e6297898dcb5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-0a082c3b-86c79d7cc6-wzmb7\" (UID: \"a4910631-a2cf-4461-b30b-e6297898dcb5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0a082c3b-86c79d7cc6-wzmb7"
Jul 30 04:08:01.899867 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:01.899783    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4910631-a2cf-4461-b30b-e6297898dcb5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-0a082c3b-86c79d7cc6-wzmb7\" (UID: \"a4910631-a2cf-4461-b30b-e6297898dcb5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0a082c3b-86c79d7cc6-wzmb7"
Jul 30 04:08:01.899867 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:01.899861    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2zmm\" (UniqueName: \"kubernetes.io/projected/a4910631-a2cf-4461-b30b-e6297898dcb5-kube-api-access-c2zmm\") pod \"payload-pre-processing-e2e-ait-0a082c3b-86c79d7cc6-wzmb7\" (UID: \"a4910631-a2cf-4461-b30b-e6297898dcb5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0a082c3b-86c79d7cc6-wzmb7"
Jul 30 04:08:01.900494 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:01.900475    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4910631-a2cf-4461-b30b-e6297898dcb5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-0a082c3b-86c79d7cc6-wzmb7\" (UID: \"a4910631-a2cf-4461-b30b-e6297898dcb5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0a082c3b-86c79d7cc6-wzmb7"
Jul 30 04:08:01.910265 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:01.910238    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2zmm\" (UniqueName: \"kubernetes.io/projected/a4910631-a2cf-4461-b30b-e6297898dcb5-kube-api-access-c2zmm\") pod \"payload-pre-processing-e2e-ait-0a082c3b-86c79d7cc6-wzmb7\" (UID: \"a4910631-a2cf-4461-b30b-e6297898dcb5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0a082c3b-86c79d7cc6-wzmb7"
Jul 30 04:08:02.066542 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:02.066509    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0a082c3b-86c79d7cc6-wzmb7"
Jul 30 04:08:02.199756 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:02.199730    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0a082c3b-86c79d7cc6-wzmb7"]
Jul 30 04:08:02.200984 ip-10-0-143-117 kubenswrapper[2576]: W0730 04:08:02.200948    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4910631_a2cf_4461_b30b_e6297898dcb5.slice/crio-bbd759e1190f4737ab597beec703ce12676619d648cfe2a2675f87cc8abe98cb WatchSource:0}: Error finding container bbd759e1190f4737ab597beec703ce12676619d648cfe2a2675f87cc8abe98cb: Status 404 returned error can't find the container with id bbd759e1190f4737ab597beec703ce12676619d648cfe2a2675f87cc8abe98cb
Jul 30 04:08:02.660489 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:02.660453    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0a082c3b-86c79d7cc6-wzmb7" event={"ID":"a4910631-a2cf-4461-b30b-e6297898dcb5","Type":"ContainerStarted","Data":"ba99d9c9cd0859ef2c0e8d64d75b5e7451166466f776a93c89c8ff22b758c13e"}
Jul 30 04:08:02.660701 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:02.660505    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0a082c3b-86c79d7cc6-wzmb7" event={"ID":"a4910631-a2cf-4461-b30b-e6297898dcb5","Type":"ContainerStarted","Data":"bbd759e1190f4737ab597beec703ce12676619d648cfe2a2675f87cc8abe98cb"}
Jul 30 04:08:02.660701 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:02.660590    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-0a082c3b-86c79d7cc6-wzmb7"
Jul 30 04:08:02.675461 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:02.675409    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-0a082c3b-86c79d7cc6-wzmb7" podStartSLOduration=1.675393064 podStartE2EDuration="1.675393064s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:08:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:08:02.674517653 +0000 UTC m=+1988.292749893" watchObservedRunningTime="2026-07-30 04:08:02.675393064 +0000 UTC m=+1988.293625306"
Jul 30 04:08:06.826394 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:06.826358    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:08:13.668054 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:13.668025    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-0a082c3b-86c79d7cc6-wzmb7"
Jul 30 04:08:15.054799 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:15.054762    2576 patch_prober.go:28] interesting pod/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": read tcp 10.134.0.2:58196->10.134.0.52:15021: read: connection reset by peer" start-of-body=
Jul 30 04:08:15.055197 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:15.054829    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw" podUID="94f98593-6e2c-4605-8f12-cc0ec0b05546" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": read tcp 10.134.0.2:58196->10.134.0.52:15021: read: connection reset by peer"
Jul 30 04:08:15.729716 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:15.729680    2576 generic.go:356] "Generic (PLEG): container finished" podID="94f98593-6e2c-4605-8f12-cc0ec0b05546" containerID="9bdcbb4f93d05f35afd57985720d4ca08d6318ae3477b4cce17750d841ad77c8" exitCode=0
Jul 30 04:08:15.729893 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:15.729754    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw" event={"ID":"94f98593-6e2c-4605-8f12-cc0ec0b05546","Type":"ContainerDied","Data":"9bdcbb4f93d05f35afd57985720d4ca08d6318ae3477b4cce17750d841ad77c8"}
Jul 30 04:08:15.730162 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:15.730149    2576 scope.go:117] "RemoveContainer" containerID="9bdcbb4f93d05f35afd57985720d4ca08d6318ae3477b4cce17750d841ad77c8"
Jul 30 04:08:15.731192 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:15.731147    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:08:15.731290 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:15.731254    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:08:15.731329 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:15.731295    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:08:16.737023 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:16.736993    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw" event={"ID":"94f98593-6e2c-4605-8f12-cc0ec0b05546","Type":"ContainerStarted","Data":"d1ef355a8de9131e63d6552c3b50a1ec24a5674bfda1d3e873167d6b7c21ab51"}
Jul 30 04:08:17.323979 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:17.323940    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:08:17.324239 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:17.324219    2576 patch_prober.go:28] interesting pod/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 30 04:08:17.324312 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:17.324260    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw" podUID="94f98593-6e2c-4605-8f12-cc0ec0b05546" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 30 04:08:18.324117 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:18.324078    2576 patch_prober.go:28] interesting pod/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 30 04:08:18.324596 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:18.324141    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw" podUID="94f98593-6e2c-4605-8f12-cc0ec0b05546" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 30 04:08:19.324098 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:19.324059    2576 patch_prober.go:28] interesting pod/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 30 04:08:19.324288 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:19.324117    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw" podUID="94f98593-6e2c-4605-8f12-cc0ec0b05546" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 30 04:08:20.323437 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:20.323395    2576 patch_prober.go:28] interesting pod/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 30 04:08:20.323649 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:20.323450    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw" podUID="94f98593-6e2c-4605-8f12-cc0ec0b05546" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 30 04:08:21.326863 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:21.326826    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:08:21.327354 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:21.327266    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:08:21.327776 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:21.327757    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:08:31.557712 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:31.557674    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:08:31.594566 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:31.594523    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0a082c3b-86c79d7cc6-wzmb7"]
Jul 30 04:08:31.594794 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:31.594763    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-0a082c3b-86c79d7cc6-wzmb7" podUID="a4910631-a2cf-4461-b30b-e6297898dcb5" containerName="payload-pre-processing" containerID="cri-o://ba99d9c9cd0859ef2c0e8d64d75b5e7451166466f776a93c89c8ff22b758c13e" gracePeriod=30
Jul 30 04:08:31.658552 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:31.658521    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:08:31.757289 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:31.757168    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:08:31.808539 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:31.808430    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:08:31.825580 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:31.825547    2576 generic.go:356] "Generic (PLEG): container finished" podID="a4910631-a2cf-4461-b30b-e6297898dcb5" containerID="ba99d9c9cd0859ef2c0e8d64d75b5e7451166466f776a93c89c8ff22b758c13e" exitCode=0
Jul 30 04:08:31.825725 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:31.825616    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0a082c3b-86c79d7cc6-wzmb7" event={"ID":"a4910631-a2cf-4461-b30b-e6297898dcb5","Type":"ContainerDied","Data":"ba99d9c9cd0859ef2c0e8d64d75b5e7451166466f776a93c89c8ff22b758c13e"}
Jul 30 04:08:31.856196 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:31.856171    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0a082c3b-86c79d7cc6-wzmb7"
Jul 30 04:08:31.877287 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:31.877145    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c2zmm\" (UniqueName: \"kubernetes.io/projected/a4910631-a2cf-4461-b30b-e6297898dcb5-kube-api-access-c2zmm\") pod \"a4910631-a2cf-4461-b30b-e6297898dcb5\" (UID: \"a4910631-a2cf-4461-b30b-e6297898dcb5\") "
Jul 30 04:08:31.877287 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:31.877269    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4910631-a2cf-4461-b30b-e6297898dcb5-plugins-config-volume\") pod \"a4910631-a2cf-4461-b30b-e6297898dcb5\" (UID: \"a4910631-a2cf-4461-b30b-e6297898dcb5\") "
Jul 30 04:08:31.878105 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:31.877903    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4910631-a2cf-4461-b30b-e6297898dcb5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a4910631-a2cf-4461-b30b-e6297898dcb5" (UID: "a4910631-a2cf-4461-b30b-e6297898dcb5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:08:31.881149 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:31.881123    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4910631-a2cf-4461-b30b-e6297898dcb5-kube-api-access-c2zmm" (OuterVolumeSpecName: "kube-api-access-c2zmm") pod "a4910631-a2cf-4461-b30b-e6297898dcb5" (UID: "a4910631-a2cf-4461-b30b-e6297898dcb5"). InnerVolumeSpecName "kube-api-access-c2zmm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:08:31.888756 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:31.888726    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"]
Jul 30 04:08:31.889114 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:31.888990    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw" podUID="94f98593-6e2c-4605-8f12-cc0ec0b05546" containerName="istio-proxy" containerID="cri-o://d1ef355a8de9131e63d6552c3b50a1ec24a5674bfda1d3e873167d6b7c21ab51" gracePeriod=30
Jul 30 04:08:31.911215 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:31.911189    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:08:31.978264 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:31.978230    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2zmm\" (UniqueName: \"kubernetes.io/projected/a4910631-a2cf-4461-b30b-e6297898dcb5-kube-api-access-c2zmm\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:08:31.978264 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:31.978264    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4910631-a2cf-4461-b30b-e6297898dcb5-plugins-config-volume\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:08:32.108205 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.108107    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:08:32.385765 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.385682    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:08:32.482118 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.482089    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:08:32.580112 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.580066    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:08:32.817150 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.817118    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"]
Jul 30 04:08:32.817538 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.817522    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4910631-a2cf-4461-b30b-e6297898dcb5" containerName="payload-pre-processing"
Jul 30 04:08:32.817615 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.817538    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4910631-a2cf-4461-b30b-e6297898dcb5" containerName="payload-pre-processing"
Jul 30 04:08:32.817615 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.817603    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4910631-a2cf-4461-b30b-e6297898dcb5" containerName="payload-pre-processing"
Jul 30 04:08:32.820978 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.820959    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:08:32.823657 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.823642    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-e1a4694c-gw-openshift-default-dockercfg-cc9pk\""
Jul 30 04:08:32.832371 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.832345    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"]
Jul 30 04:08:32.834024 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.834005    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0a082c3b-86c79d7cc6-wzmb7"
Jul 30 04:08:32.834131 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.834009    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0a082c3b-86c79d7cc6-wzmb7" event={"ID":"a4910631-a2cf-4461-b30b-e6297898dcb5","Type":"ContainerDied","Data":"bbd759e1190f4737ab597beec703ce12676619d648cfe2a2675f87cc8abe98cb"}
Jul 30 04:08:32.834222 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.834129    2576 scope.go:117] "RemoveContainer" containerID="ba99d9c9cd0859ef2c0e8d64d75b5e7451166466f776a93c89c8ff22b758c13e"
Jul 30 04:08:32.844844 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.844815    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:08:32.856027 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.856003    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0a082c3b-86c79d7cc6-wzmb7"]
Jul 30 04:08:32.860944 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.860909    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0a082c3b-86c79d7cc6-wzmb7"]
Jul 30 04:08:32.892791 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.892764    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-istio-token\") pod \"e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9\" (UID: \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\") " pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:08:32.892895 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.892796    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-credential-socket\") pod \"e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9\" (UID: \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\") " pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:08:32.892895 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.892817    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-istio-envoy\") pod \"e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9\" (UID: \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\") " pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:08:32.892895 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.892834    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-workload-socket\") pod \"e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9\" (UID: \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\") " pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:08:32.892895 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.892856    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-istio-podinfo\") pod \"e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9\" (UID: \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\") " pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:08:32.892895 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.892890    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-istio-data\") pod \"e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9\" (UID: \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\") " pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:08:32.893085 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.892912    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-workload-certs\") pod \"e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9\" (UID: \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\") " pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:08:32.893085 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.892999    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-istiod-ca-cert\") pod \"e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9\" (UID: \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\") " pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:08:32.893085 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.893052    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btcn6\" (UniqueName: \"kubernetes.io/projected/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-kube-api-access-btcn6\") pod \"e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9\" (UID: \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\") " pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:08:32.993809 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.993770    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-istiod-ca-cert\") pod \"e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9\" (UID: \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\") " pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:08:32.993809 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.993813    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btcn6\" (UniqueName: \"kubernetes.io/projected/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-kube-api-access-btcn6\") pod \"e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9\" (UID: \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\") " pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:08:32.994071 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.993858    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-istio-token\") pod \"e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9\" (UID: \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\") " pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:08:32.994071 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.993877    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-credential-socket\") pod \"e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9\" (UID: \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\") " pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:08:32.994071 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.993896    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-istio-envoy\") pod \"e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9\" (UID: \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\") " pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:08:32.994071 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.993914    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-workload-socket\") pod \"e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9\" (UID: \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\") " pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:08:32.994290 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.994091    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-istio-podinfo\") pod \"e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9\" (UID: \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\") " pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:08:32.994290 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.994173    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-istio-data\") pod \"e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9\" (UID: \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\") " pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:08:32.994290 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.994213    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-workload-certs\") pod \"e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9\" (UID: \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\") " pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:08:32.994290 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.994280    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-credential-socket\") pod \"e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9\" (UID: \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\") " pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:08:32.994492 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.994410    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-workload-socket\") pod \"e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9\" (UID: \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\") " pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:08:32.994551 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.994508    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-istio-data\") pod \"e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9\" (UID: \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\") " pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:08:32.994614 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.994600    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-workload-certs\") pod \"e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9\" (UID: \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\") " pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:08:32.994661 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.994634    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-istiod-ca-cert\") pod \"e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9\" (UID: \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\") " pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:08:32.996339 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.996321    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-istio-envoy\") pod \"e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9\" (UID: \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\") " pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:08:32.996401 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:32.996384    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-istio-podinfo\") pod \"e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9\" (UID: \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\") " pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:08:33.002395 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:33.002370    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-istio-token\") pod \"e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9\" (UID: \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\") " pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:08:33.002559 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:33.002540    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btcn6\" (UniqueName: \"kubernetes.io/projected/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-kube-api-access-btcn6\") pod \"e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9\" (UID: \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\") " pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:08:33.050830 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:33.050795    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4910631-a2cf-4461-b30b-e6297898dcb5" path="/var/lib/kubelet/pods/a4910631-a2cf-4461-b30b-e6297898dcb5/volumes"
Jul 30 04:08:33.132002 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:33.131900    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:08:33.160815 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:33.160766    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:08:33.265863 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:33.265822    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"]
Jul 30 04:08:33.266909 ip-10-0-143-117 kubenswrapper[2576]: W0730 04:08:33.266883    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85a0ec97_0b7e_42f3_8ef0_bf97a54aa004.slice/crio-62a512fd76ecf3d2964feaf8a952692fb057abd0f2655e75acda2a4a9d77e8ba WatchSource:0}: Error finding container 62a512fd76ecf3d2964feaf8a952692fb057abd0f2655e75acda2a4a9d77e8ba: Status 404 returned error can't find the container with id 62a512fd76ecf3d2964feaf8a952692fb057abd0f2655e75acda2a4a9d77e8ba
Jul 30 04:08:33.269113 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:33.269082    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:08:33.269210 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:33.269143    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:08:33.269210 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:33.269173    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:08:33.842059 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:33.842026    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9" event={"ID":"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004","Type":"ContainerStarted","Data":"5e1dd70d87dc29ae6ca47d1f984d65181a5b9662fb10b42b31506ee3a985ee23"}
Jul 30 04:08:33.842399 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:33.842080    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9" event={"ID":"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004","Type":"ContainerStarted","Data":"62a512fd76ecf3d2964feaf8a952692fb057abd0f2655e75acda2a4a9d77e8ba"}
Jul 30 04:08:33.860643 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:33.860594    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9" podStartSLOduration=1.860578293 podStartE2EDuration="1.860578293s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:08:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:08:33.858443779 +0000 UTC m=+2019.476676019" watchObservedRunningTime="2026-07-30 04:08:33.860578293 +0000 UTC m=+2019.478810534"
Jul 30 04:08:34.132692 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:34.132610    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:08:34.137535 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:34.137510    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:08:34.847730 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:34.847706    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:08:34.848707 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:34.848688    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:08:35.287515 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:35.287485    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:08:35.404454 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:35.404422    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:08:35.830787 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:35.830752    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:08:35.903475 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:35.903446    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:08:36.000702 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:36.000673    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:08:36.330005 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:36.329968    2576 patch_prober.go:28] interesting pod/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Jul 30 04:08:36.330197 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:36.330033    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw" podUID="94f98593-6e2c-4605-8f12-cc0ec0b05546" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 04:08:36.384526 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:36.384493    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:08:37.147317 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.147293    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:08:37.236898 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.236817    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94f98593-6e2c-4605-8f12-cc0ec0b05546-istiod-ca-cert\") pod \"94f98593-6e2c-4605-8f12-cc0ec0b05546\" (UID: \"94f98593-6e2c-4605-8f12-cc0ec0b05546\") "
Jul 30 04:08:37.236898 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.236871    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94f98593-6e2c-4605-8f12-cc0ec0b05546-istio-podinfo\") pod \"94f98593-6e2c-4605-8f12-cc0ec0b05546\" (UID: \"94f98593-6e2c-4605-8f12-cc0ec0b05546\") "
Jul 30 04:08:37.237154 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.236910    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94f98593-6e2c-4605-8f12-cc0ec0b05546-istio-envoy\") pod \"94f98593-6e2c-4605-8f12-cc0ec0b05546\" (UID: \"94f98593-6e2c-4605-8f12-cc0ec0b05546\") "
Jul 30 04:08:37.237154 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.237009    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94f98593-6e2c-4605-8f12-cc0ec0b05546-workload-socket\") pod \"94f98593-6e2c-4605-8f12-cc0ec0b05546\" (UID: \"94f98593-6e2c-4605-8f12-cc0ec0b05546\") "
Jul 30 04:08:37.237154 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.237075    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94f98593-6e2c-4605-8f12-cc0ec0b05546-workload-certs\") pod \"94f98593-6e2c-4605-8f12-cc0ec0b05546\" (UID: \"94f98593-6e2c-4605-8f12-cc0ec0b05546\") "
Jul 30 04:08:37.237154 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.237114    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94f98593-6e2c-4605-8f12-cc0ec0b05546-istio-data\") pod \"94f98593-6e2c-4605-8f12-cc0ec0b05546\" (UID: \"94f98593-6e2c-4605-8f12-cc0ec0b05546\") "
Jul 30 04:08:37.237154 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.237152    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94f98593-6e2c-4605-8f12-cc0ec0b05546-credential-socket\") pod \"94f98593-6e2c-4605-8f12-cc0ec0b05546\" (UID: \"94f98593-6e2c-4605-8f12-cc0ec0b05546\") "
Jul 30 04:08:37.237422 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.237230    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-75cmr\" (UniqueName: \"kubernetes.io/projected/94f98593-6e2c-4605-8f12-cc0ec0b05546-kube-api-access-75cmr\") pod \"94f98593-6e2c-4605-8f12-cc0ec0b05546\" (UID: \"94f98593-6e2c-4605-8f12-cc0ec0b05546\") "
Jul 30 04:08:37.237422 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.237257    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94f98593-6e2c-4605-8f12-cc0ec0b05546-istio-token\") pod \"94f98593-6e2c-4605-8f12-cc0ec0b05546\" (UID: \"94f98593-6e2c-4605-8f12-cc0ec0b05546\") "
Jul 30 04:08:37.237422 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.237262    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94f98593-6e2c-4605-8f12-cc0ec0b05546-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "94f98593-6e2c-4605-8f12-cc0ec0b05546" (UID: "94f98593-6e2c-4605-8f12-cc0ec0b05546"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:08:37.237422 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.237333    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94f98593-6e2c-4605-8f12-cc0ec0b05546-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "94f98593-6e2c-4605-8f12-cc0ec0b05546" (UID: "94f98593-6e2c-4605-8f12-cc0ec0b05546"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:08:37.237636 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.237430    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94f98593-6e2c-4605-8f12-cc0ec0b05546-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "94f98593-6e2c-4605-8f12-cc0ec0b05546" (UID: "94f98593-6e2c-4605-8f12-cc0ec0b05546"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:08:37.237636 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.237500    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94f98593-6e2c-4605-8f12-cc0ec0b05546-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "94f98593-6e2c-4605-8f12-cc0ec0b05546" (UID: "94f98593-6e2c-4605-8f12-cc0ec0b05546"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:08:37.237636 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.237616    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94f98593-6e2c-4605-8f12-cc0ec0b05546-workload-socket\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:08:37.237636 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.237634    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94f98593-6e2c-4605-8f12-cc0ec0b05546-workload-certs\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:08:37.237850 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.237649    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94f98593-6e2c-4605-8f12-cc0ec0b05546-credential-socket\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:08:37.237850 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.237662    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94f98593-6e2c-4605-8f12-cc0ec0b05546-istiod-ca-cert\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:08:37.237990 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.237964    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94f98593-6e2c-4605-8f12-cc0ec0b05546-istio-data" (OuterVolumeSpecName: "istio-data") pod "94f98593-6e2c-4605-8f12-cc0ec0b05546" (UID: "94f98593-6e2c-4605-8f12-cc0ec0b05546"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:08:37.239265 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.239236    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94f98593-6e2c-4605-8f12-cc0ec0b05546-kube-api-access-75cmr" (OuterVolumeSpecName: "kube-api-access-75cmr") pod "94f98593-6e2c-4605-8f12-cc0ec0b05546" (UID: "94f98593-6e2c-4605-8f12-cc0ec0b05546"). InnerVolumeSpecName "kube-api-access-75cmr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:08:37.239402 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.239266    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/94f98593-6e2c-4605-8f12-cc0ec0b05546-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "94f98593-6e2c-4605-8f12-cc0ec0b05546" (UID: "94f98593-6e2c-4605-8f12-cc0ec0b05546"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 04:08:37.239466 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.239445    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94f98593-6e2c-4605-8f12-cc0ec0b05546-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "94f98593-6e2c-4605-8f12-cc0ec0b05546" (UID: "94f98593-6e2c-4605-8f12-cc0ec0b05546"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:08:37.239543 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.239529    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94f98593-6e2c-4605-8f12-cc0ec0b05546-istio-token" (OuterVolumeSpecName: "istio-token") pod "94f98593-6e2c-4605-8f12-cc0ec0b05546" (UID: "94f98593-6e2c-4605-8f12-cc0ec0b05546"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:08:37.339076 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.339033    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75cmr\" (UniqueName: \"kubernetes.io/projected/94f98593-6e2c-4605-8f12-cc0ec0b05546-kube-api-access-75cmr\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:08:37.339076 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.339070    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94f98593-6e2c-4605-8f12-cc0ec0b05546-istio-token\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:08:37.339076 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.339084    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94f98593-6e2c-4605-8f12-cc0ec0b05546-istio-podinfo\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:08:37.339318 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.339096    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94f98593-6e2c-4605-8f12-cc0ec0b05546-istio-envoy\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:08:37.339318 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.339108    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94f98593-6e2c-4605-8f12-cc0ec0b05546-istio-data\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:08:37.868338 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.868299    2576 generic.go:356] "Generic (PLEG): container finished" podID="94f98593-6e2c-4605-8f12-cc0ec0b05546" containerID="d1ef355a8de9131e63d6552c3b50a1ec24a5674bfda1d3e873167d6b7c21ab51" exitCode=0
Jul 30 04:08:37.868511 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.868346    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"
Jul 30 04:08:37.868511 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.868365    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw" event={"ID":"94f98593-6e2c-4605-8f12-cc0ec0b05546","Type":"ContainerDied","Data":"d1ef355a8de9131e63d6552c3b50a1ec24a5674bfda1d3e873167d6b7c21ab51"}
Jul 30 04:08:37.868511 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.868413    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw" event={"ID":"94f98593-6e2c-4605-8f12-cc0ec0b05546","Type":"ContainerDied","Data":"a2ea695617fee485afb6e56c3ac26ddc84c28b213c5a5de1183e17143478ccd5"}
Jul 30 04:08:37.868511 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.868436    2576 scope.go:117] "RemoveContainer" containerID="d1ef355a8de9131e63d6552c3b50a1ec24a5674bfda1d3e873167d6b7c21ab51"
Jul 30 04:08:37.878165 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.878148    2576 scope.go:117] "RemoveContainer" containerID="9bdcbb4f93d05f35afd57985720d4ca08d6318ae3477b4cce17750d841ad77c8"
Jul 30 04:08:37.886276 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.886259    2576 scope.go:117] "RemoveContainer" containerID="d1ef355a8de9131e63d6552c3b50a1ec24a5674bfda1d3e873167d6b7c21ab51"
Jul 30 04:08:37.886517 ip-10-0-143-117 kubenswrapper[2576]: E0730 04:08:37.886494    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1ef355a8de9131e63d6552c3b50a1ec24a5674bfda1d3e873167d6b7c21ab51\": container with ID starting with d1ef355a8de9131e63d6552c3b50a1ec24a5674bfda1d3e873167d6b7c21ab51 not found: ID does not exist" containerID="d1ef355a8de9131e63d6552c3b50a1ec24a5674bfda1d3e873167d6b7c21ab51"
Jul 30 04:08:37.886602 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.886520    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1ef355a8de9131e63d6552c3b50a1ec24a5674bfda1d3e873167d6b7c21ab51"} err="failed to get container status \"d1ef355a8de9131e63d6552c3b50a1ec24a5674bfda1d3e873167d6b7c21ab51\": rpc error: code = NotFound desc = could not find container \"d1ef355a8de9131e63d6552c3b50a1ec24a5674bfda1d3e873167d6b7c21ab51\": container with ID starting with d1ef355a8de9131e63d6552c3b50a1ec24a5674bfda1d3e873167d6b7c21ab51 not found: ID does not exist"
Jul 30 04:08:37.886602 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.886535    2576 scope.go:117] "RemoveContainer" containerID="9bdcbb4f93d05f35afd57985720d4ca08d6318ae3477b4cce17750d841ad77c8"
Jul 30 04:08:37.886791 ip-10-0-143-117 kubenswrapper[2576]: E0730 04:08:37.886774    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bdcbb4f93d05f35afd57985720d4ca08d6318ae3477b4cce17750d841ad77c8\": container with ID starting with 9bdcbb4f93d05f35afd57985720d4ca08d6318ae3477b4cce17750d841ad77c8 not found: ID does not exist" containerID="9bdcbb4f93d05f35afd57985720d4ca08d6318ae3477b4cce17750d841ad77c8"
Jul 30 04:08:37.886878 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.886800    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bdcbb4f93d05f35afd57985720d4ca08d6318ae3477b4cce17750d841ad77c8"} err="failed to get container status \"9bdcbb4f93d05f35afd57985720d4ca08d6318ae3477b4cce17750d841ad77c8\": rpc error: code = NotFound desc = could not find container \"9bdcbb4f93d05f35afd57985720d4ca08d6318ae3477b4cce17750d841ad77c8\": container with ID starting with 9bdcbb4f93d05f35afd57985720d4ca08d6318ae3477b4cce17750d841ad77c8 not found: ID does not exist"
Jul 30 04:08:37.889490 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.889472    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"]
Jul 30 04:08:37.893069 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:37.893050    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-0a082c3b-openshift-default-58dd877498-t7cgw"]
Jul 30 04:08:39.051391 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:39.051358    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94f98593-6e2c-4605-8f12-cc0ec0b05546" path="/var/lib/kubelet/pods/94f98593-6e2c-4605-8f12-cc0ec0b05546/volumes"
Jul 30 04:08:47.128681 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:47.128647    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:08:50.455983 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:50.455416    2576 patch_prober.go:28] interesting pod/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": read tcp 10.134.0.2:49312->10.134.0.54:15021: read: connection reset by peer" start-of-body=
Jul 30 04:08:50.455983 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:50.455489    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9" podUID="85a0ec97-0b7e-42f3-8ef0-bf97a54aa004" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": read tcp 10.134.0.2:49312->10.134.0.54:15021: read: connection reset by peer"
Jul 30 04:08:50.937817 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:50.937782    2576 generic.go:356] "Generic (PLEG): container finished" podID="85a0ec97-0b7e-42f3-8ef0-bf97a54aa004" containerID="5e1dd70d87dc29ae6ca47d1f984d65181a5b9662fb10b42b31506ee3a985ee23" exitCode=0
Jul 30 04:08:50.937994 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:50.937853    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9" event={"ID":"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004","Type":"ContainerDied","Data":"5e1dd70d87dc29ae6ca47d1f984d65181a5b9662fb10b42b31506ee3a985ee23"}
Jul 30 04:08:50.938273 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:50.938249    2576 scope.go:117] "RemoveContainer" containerID="5e1dd70d87dc29ae6ca47d1f984d65181a5b9662fb10b42b31506ee3a985ee23"
Jul 30 04:08:50.939268 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:50.939240    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:08:50.939337 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:50.939312    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:08:50.939373 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:50.939347    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:08:51.950786 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:51.949758    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9" event={"ID":"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004","Type":"ContainerStarted","Data":"917e1af1df54800130182f994ff832cbde7cbb353fdcb96dba6fe8f3ce602ba8"}
Jul 30 04:08:52.132245 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:52.132209    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:08:52.132464 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:52.132440    2576 patch_prober.go:28] interesting pod/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 30 04:08:52.132515 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:52.132494    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9" podUID="85a0ec97-0b7e-42f3-8ef0-bf97a54aa004" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 30 04:08:53.132196 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:53.132165    2576 patch_prober.go:28] interesting pod/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 30 04:08:53.132579 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:53.132229    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9" podUID="85a0ec97-0b7e-42f3-8ef0-bf97a54aa004" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 30 04:08:54.132588 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:54.132555    2576 patch_prober.go:28] interesting pod/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 30 04:08:54.133011 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:54.132607    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9" podUID="85a0ec97-0b7e-42f3-8ef0-bf97a54aa004" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 30 04:08:55.132813 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:55.132788    2576 patch_prober.go:28] interesting pod/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 30 04:08:55.133193 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:55.132840    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9" podUID="85a0ec97-0b7e-42f3-8ef0-bf97a54aa004" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 30 04:08:56.136146 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:56.136117    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:08:56.136527 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:56.136505    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:08:56.137239 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:08:56.137220    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:09:00.294280 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:00.294252    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e1a4694c-nfrdd"]
Jul 30 04:09:00.294652 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:00.294622    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94f98593-6e2c-4605-8f12-cc0ec0b05546" containerName="istio-proxy"
Jul 30 04:09:00.294652 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:00.294636    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="94f98593-6e2c-4605-8f12-cc0ec0b05546" containerName="istio-proxy"
Jul 30 04:09:00.294652 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:00.294654    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94f98593-6e2c-4605-8f12-cc0ec0b05546" containerName="istio-proxy"
Jul 30 04:09:00.294750 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:00.294660    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="94f98593-6e2c-4605-8f12-cc0ec0b05546" containerName="istio-proxy"
Jul 30 04:09:00.294750 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:00.294734    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="94f98593-6e2c-4605-8f12-cc0ec0b05546" containerName="istio-proxy"
Jul 30 04:09:00.294750 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:00.294742    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="94f98593-6e2c-4605-8f12-cc0ec0b05546" containerName="istio-proxy"
Jul 30 04:09:00.297837 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:00.297820    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e1a4694c-nfrdd"
Jul 30 04:09:00.300528 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:00.300509    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5cg4s\""
Jul 30 04:09:00.305222 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:00.305125    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e1a4694c-nfrdd"]
Jul 30 04:09:00.449434 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:00.449398    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbe95d7f-b45d-4f33-8a92-62e777ca4d03-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-e1a4694c-nfrdd\" (UID: \"bbe95d7f-b45d-4f33-8a92-62e777ca4d03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e1a4694c-nfrdd"
Jul 30 04:09:00.550757 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:00.550646    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbe95d7f-b45d-4f33-8a92-62e777ca4d03-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-e1a4694c-nfrdd\" (UID: \"bbe95d7f-b45d-4f33-8a92-62e777ca4d03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e1a4694c-nfrdd"
Jul 30 04:09:00.551377 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:00.551357    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbe95d7f-b45d-4f33-8a92-62e777ca4d03-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-e1a4694c-nfrdd\" (UID: \"bbe95d7f-b45d-4f33-8a92-62e777ca4d03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e1a4694c-nfrdd"
Jul 30 04:09:00.609197 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:00.609152    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e1a4694c-nfrdd"
Jul 30 04:09:00.738512 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:00.738480    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e1a4694c-nfrdd"]
Jul 30 04:09:00.739743 ip-10-0-143-117 kubenswrapper[2576]: W0730 04:09:00.739714    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbe95d7f_b45d_4f33_8a92_62e777ca4d03.slice/crio-203f174e56285b0671ee959bdc9360148bbbd418ce30afb12d86232be86b29d0 WatchSource:0}: Error finding container 203f174e56285b0671ee959bdc9360148bbbd418ce30afb12d86232be86b29d0: Status 404 returned error can't find the container with id 203f174e56285b0671ee959bdc9360148bbbd418ce30afb12d86232be86b29d0
Jul 30 04:09:00.741641 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:00.741623    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 04:09:01.007099 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:01.007014    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e1a4694c-nfrdd" event={"ID":"bbe95d7f-b45d-4f33-8a92-62e777ca4d03","Type":"ContainerStarted","Data":"203f174e56285b0671ee959bdc9360148bbbd418ce30afb12d86232be86b29d0"}
Jul 30 04:09:04.025380 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:04.025341    2576 generic.go:356] "Generic (PLEG): container finished" podID="bbe95d7f-b45d-4f33-8a92-62e777ca4d03" containerID="2b16d66c13b7926d5f427f92dc5ea5a3300e7d61f5667a85124b044b6d9a10ec" exitCode=0
Jul 30 04:09:04.025778 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:04.025383    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e1a4694c-nfrdd" event={"ID":"bbe95d7f-b45d-4f33-8a92-62e777ca4d03","Type":"ContainerDied","Data":"2b16d66c13b7926d5f427f92dc5ea5a3300e7d61f5667a85124b044b6d9a10ec"}
Jul 30 04:09:05.165784 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:05.165759    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e1a4694c-nfrdd"
Jul 30 04:09:05.296224 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:05.296142    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbe95d7f-b45d-4f33-8a92-62e777ca4d03-maas-api-service-ca\") pod \"bbe95d7f-b45d-4f33-8a92-62e777ca4d03\" (UID: \"bbe95d7f-b45d-4f33-8a92-62e777ca4d03\") "
Jul 30 04:09:05.296491 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:05.296471    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbe95d7f-b45d-4f33-8a92-62e777ca4d03-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bbe95d7f-b45d-4f33-8a92-62e777ca4d03" (UID: "bbe95d7f-b45d-4f33-8a92-62e777ca4d03"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:09:05.396973 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:05.396946    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbe95d7f-b45d-4f33-8a92-62e777ca4d03-maas-api-service-ca\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:09:06.038001 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:06.037974    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e1a4694c-nfrdd"
Jul 30 04:09:06.038173 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:06.037971    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e1a4694c-nfrdd" event={"ID":"bbe95d7f-b45d-4f33-8a92-62e777ca4d03","Type":"ContainerDied","Data":"203f174e56285b0671ee959bdc9360148bbbd418ce30afb12d86232be86b29d0"}
Jul 30 04:09:06.038173 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:06.038096    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="203f174e56285b0671ee959bdc9360148bbbd418ce30afb12d86232be86b29d0"
Jul 30 04:09:15.937885 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:15.937847    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:09:15.996799 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:15.996767    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:09:16.054777 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:16.054744    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:09:16.101875 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:16.101845    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e1a4694c-nfrdd"]
Jul 30 04:09:16.104300 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:16.104278    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e1a4694c-nfrdd"]
Jul 30 04:09:16.114533 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:16.114501    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:09:16.243433 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:16.243337    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"]
Jul 30 04:09:16.243611 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:16.243582    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9" podUID="85a0ec97-0b7e-42f3-8ef0-bf97a54aa004" containerName="istio-proxy" containerID="cri-o://917e1af1df54800130182f994ff832cbde7cbb353fdcb96dba6fe8f3ce602ba8" gracePeriod=30
Jul 30 04:09:16.340469 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:16.340417    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:09:16.522404 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:16.522366    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:09:16.767007 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:16.766972    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:09:17.052262 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.052228    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbe95d7f-b45d-4f33-8a92-62e777ca4d03" path="/var/lib/kubelet/pods/bbe95d7f-b45d-4f33-8a92-62e777ca4d03/volumes"
Jul 30 04:09:17.054300 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.054280    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"]
Jul 30 04:09:17.054643 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.054632    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbe95d7f-b45d-4f33-8a92-62e777ca4d03" containerName="revoke-keys"
Jul 30 04:09:17.054687 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.054645    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbe95d7f-b45d-4f33-8a92-62e777ca4d03" containerName="revoke-keys"
Jul 30 04:09:17.054722 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.054710    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbe95d7f-b45d-4f33-8a92-62e777ca4d03" containerName="revoke-keys"
Jul 30 04:09:17.062478 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.062459    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:17.065033 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.065017    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-1d4857d1-openshift-default-dockercfg-54p6h\""
Jul 30 04:09:17.071643 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.071623    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"]
Jul 30 04:09:17.075255 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.075225    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:09:17.093694 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.093665    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9712893e-8e6d-4654-b16c-1190a2b2489f-istio-token\") pod \"e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc\" (UID: \"9712893e-8e6d-4654-b16c-1190a2b2489f\") " pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:17.093845 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.093715    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9712893e-8e6d-4654-b16c-1190a2b2489f-istio-envoy\") pod \"e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc\" (UID: \"9712893e-8e6d-4654-b16c-1190a2b2489f\") " pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:17.093845 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.093746    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9712893e-8e6d-4654-b16c-1190a2b2489f-workload-socket\") pod \"e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc\" (UID: \"9712893e-8e6d-4654-b16c-1190a2b2489f\") " pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:17.093845 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.093763    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9712893e-8e6d-4654-b16c-1190a2b2489f-istio-data\") pod \"e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc\" (UID: \"9712893e-8e6d-4654-b16c-1190a2b2489f\") " pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:17.093845 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.093780    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9712893e-8e6d-4654-b16c-1190a2b2489f-istiod-ca-cert\") pod \"e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc\" (UID: \"9712893e-8e6d-4654-b16c-1190a2b2489f\") " pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:17.093845 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.093803    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9712893e-8e6d-4654-b16c-1190a2b2489f-credential-socket\") pod \"e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc\" (UID: \"9712893e-8e6d-4654-b16c-1190a2b2489f\") " pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:17.094059 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.093850    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9712893e-8e6d-4654-b16c-1190a2b2489f-istio-podinfo\") pod \"e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc\" (UID: \"9712893e-8e6d-4654-b16c-1190a2b2489f\") " pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:17.094059 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.093896    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9712893e-8e6d-4654-b16c-1190a2b2489f-workload-certs\") pod \"e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc\" (UID: \"9712893e-8e6d-4654-b16c-1190a2b2489f\") " pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:17.094059 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.093967    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ks2bd\" (UniqueName: \"kubernetes.io/projected/9712893e-8e6d-4654-b16c-1190a2b2489f-kube-api-access-ks2bd\") pod \"e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc\" (UID: \"9712893e-8e6d-4654-b16c-1190a2b2489f\") " pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:17.195197 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.195165    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9712893e-8e6d-4654-b16c-1190a2b2489f-workload-certs\") pod \"e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc\" (UID: \"9712893e-8e6d-4654-b16c-1190a2b2489f\") " pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:17.195374 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.195206    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ks2bd\" (UniqueName: \"kubernetes.io/projected/9712893e-8e6d-4654-b16c-1190a2b2489f-kube-api-access-ks2bd\") pod \"e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc\" (UID: \"9712893e-8e6d-4654-b16c-1190a2b2489f\") " pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:17.195374 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.195244    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9712893e-8e6d-4654-b16c-1190a2b2489f-istio-token\") pod \"e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc\" (UID: \"9712893e-8e6d-4654-b16c-1190a2b2489f\") " pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:17.195374 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.195278    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9712893e-8e6d-4654-b16c-1190a2b2489f-istio-envoy\") pod \"e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc\" (UID: \"9712893e-8e6d-4654-b16c-1190a2b2489f\") " pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:17.195374 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.195310    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9712893e-8e6d-4654-b16c-1190a2b2489f-workload-socket\") pod \"e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc\" (UID: \"9712893e-8e6d-4654-b16c-1190a2b2489f\") " pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:17.195374 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.195328    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9712893e-8e6d-4654-b16c-1190a2b2489f-istio-data\") pod \"e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc\" (UID: \"9712893e-8e6d-4654-b16c-1190a2b2489f\") " pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:17.195374 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.195349    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9712893e-8e6d-4654-b16c-1190a2b2489f-istiod-ca-cert\") pod \"e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc\" (UID: \"9712893e-8e6d-4654-b16c-1190a2b2489f\") " pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:17.195374 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.195373    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9712893e-8e6d-4654-b16c-1190a2b2489f-credential-socket\") pod \"e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc\" (UID: \"9712893e-8e6d-4654-b16c-1190a2b2489f\") " pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:17.195702 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.195391    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9712893e-8e6d-4654-b16c-1190a2b2489f-istio-podinfo\") pod \"e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc\" (UID: \"9712893e-8e6d-4654-b16c-1190a2b2489f\") " pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:17.195702 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.195691    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9712893e-8e6d-4654-b16c-1190a2b2489f-workload-certs\") pod \"e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc\" (UID: \"9712893e-8e6d-4654-b16c-1190a2b2489f\") " pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:17.195819 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.195730    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9712893e-8e6d-4654-b16c-1190a2b2489f-workload-socket\") pod \"e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc\" (UID: \"9712893e-8e6d-4654-b16c-1190a2b2489f\") " pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:17.195886 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.195869    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9712893e-8e6d-4654-b16c-1190a2b2489f-istio-data\") pod \"e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc\" (UID: \"9712893e-8e6d-4654-b16c-1190a2b2489f\") " pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:17.195994 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.195971    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9712893e-8e6d-4654-b16c-1190a2b2489f-credential-socket\") pod \"e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc\" (UID: \"9712893e-8e6d-4654-b16c-1190a2b2489f\") " pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:17.196125 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.196107    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9712893e-8e6d-4654-b16c-1190a2b2489f-istiod-ca-cert\") pod \"e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc\" (UID: \"9712893e-8e6d-4654-b16c-1190a2b2489f\") " pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:17.197675 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.197654    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9712893e-8e6d-4654-b16c-1190a2b2489f-istio-envoy\") pod \"e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc\" (UID: \"9712893e-8e6d-4654-b16c-1190a2b2489f\") " pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:17.197862 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.197846    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9712893e-8e6d-4654-b16c-1190a2b2489f-istio-podinfo\") pod \"e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc\" (UID: \"9712893e-8e6d-4654-b16c-1190a2b2489f\") " pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:17.203426 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.203407    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ks2bd\" (UniqueName: \"kubernetes.io/projected/9712893e-8e6d-4654-b16c-1190a2b2489f-kube-api-access-ks2bd\") pod \"e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc\" (UID: \"9712893e-8e6d-4654-b16c-1190a2b2489f\") " pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:17.203566 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.203550    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9712893e-8e6d-4654-b16c-1190a2b2489f-istio-token\") pod \"e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc\" (UID: \"9712893e-8e6d-4654-b16c-1190a2b2489f\") " pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:17.225500 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.225453    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:09:17.373845 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.373751    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:17.505623 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.505561    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"]
Jul 30 04:09:17.507982 ip-10-0-143-117 kubenswrapper[2576]: W0730 04:09:17.507953    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9712893e_8e6d_4654_b16c_1190a2b2489f.slice/crio-d358c07441bac7025470b1c5fb6f470256f18257e8f48fd937a3ad5c1b21cbe5 WatchSource:0}: Error finding container d358c07441bac7025470b1c5fb6f470256f18257e8f48fd937a3ad5c1b21cbe5: Status 404 returned error can't find the container with id d358c07441bac7025470b1c5fb6f470256f18257e8f48fd937a3ad5c1b21cbe5
Jul 30 04:09:17.510200 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.510167    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:09:17.510308 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.510237    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:09:17.510308 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:17.510268    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:09:18.100364 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:18.100328    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc" event={"ID":"9712893e-8e6d-4654-b16c-1190a2b2489f","Type":"ContainerStarted","Data":"ec4c2d64ed2d5f9dfd4851a753b7f5d37272a750e0b7f5e6e9fc638cdaa8557c"}
Jul 30 04:09:18.100743 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:18.100390    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc" event={"ID":"9712893e-8e6d-4654-b16c-1190a2b2489f","Type":"ContainerStarted","Data":"d358c07441bac7025470b1c5fb6f470256f18257e8f48fd937a3ad5c1b21cbe5"}
Jul 30 04:09:18.121513 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:18.121462    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc" podStartSLOduration=1.121437997 podStartE2EDuration="1.121437997s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:09:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:09:18.118982152 +0000 UTC m=+2063.737214394" watchObservedRunningTime="2026-07-30 04:09:18.121437997 +0000 UTC m=+2063.739670239"
Jul 30 04:09:18.374024 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:18.373907    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:18.378714 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:18.378693    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:19.106330 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:19.106296    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:19.107322 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:19.107304    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:19.467886 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:19.467786    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:09:19.558260 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:19.558225    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:09:20.029950 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:20.029891    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:09:20.662733 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:20.662697    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:09:21.111496 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:21.111459    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:09:21.208226 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:21.208196    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:09:21.276535 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:21.276506    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:09:21.499354 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:21.499331    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:09:21.536621 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:21.536596    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-istio-data\") pod \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\" (UID: \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\") "
Jul 30 04:09:21.536764 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:21.536651    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-btcn6\" (UniqueName: \"kubernetes.io/projected/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-kube-api-access-btcn6\") pod \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\" (UID: \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\") "
Jul 30 04:09:21.536764 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:21.536700    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-istio-envoy\") pod \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\" (UID: \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\") "
Jul 30 04:09:21.536764 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:21.536736    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-workload-socket\") pod \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\" (UID: \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\") "
Jul 30 04:09:21.536904 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:21.536774    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-istio-token\") pod \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\" (UID: \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\") "
Jul 30 04:09:21.536904 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:21.536798    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-credential-socket\") pod \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\" (UID: \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\") "
Jul 30 04:09:21.536904 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:21.536821    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-istio-podinfo\") pod \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\" (UID: \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\") "
Jul 30 04:09:21.536904 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:21.536872    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-workload-certs\") pod \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\" (UID: \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\") "
Jul 30 04:09:21.537139 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:21.536914    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-istiod-ca-cert\") pod \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\" (UID: \"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004\") "
Jul 30 04:09:21.537139 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:21.537111    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "85a0ec97-0b7e-42f3-8ef0-bf97a54aa004" (UID: "85a0ec97-0b7e-42f3-8ef0-bf97a54aa004"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:09:21.537486 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:21.537304    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-credential-socket\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:09:21.537486 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:21.537319    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "85a0ec97-0b7e-42f3-8ef0-bf97a54aa004" (UID: "85a0ec97-0b7e-42f3-8ef0-bf97a54aa004"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:09:21.537486 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:21.537420    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "85a0ec97-0b7e-42f3-8ef0-bf97a54aa004" (UID: "85a0ec97-0b7e-42f3-8ef0-bf97a54aa004"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:09:21.537486 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:21.537481    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "85a0ec97-0b7e-42f3-8ef0-bf97a54aa004" (UID: "85a0ec97-0b7e-42f3-8ef0-bf97a54aa004"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:09:21.537842 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:21.537523    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-istio-data" (OuterVolumeSpecName: "istio-data") pod "85a0ec97-0b7e-42f3-8ef0-bf97a54aa004" (UID: "85a0ec97-0b7e-42f3-8ef0-bf97a54aa004"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:09:21.539137 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:21.539109    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-istio-token" (OuterVolumeSpecName: "istio-token") pod "85a0ec97-0b7e-42f3-8ef0-bf97a54aa004" (UID: "85a0ec97-0b7e-42f3-8ef0-bf97a54aa004"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:09:21.539244 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:21.539132    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-kube-api-access-btcn6" (OuterVolumeSpecName: "kube-api-access-btcn6") pod "85a0ec97-0b7e-42f3-8ef0-bf97a54aa004" (UID: "85a0ec97-0b7e-42f3-8ef0-bf97a54aa004"). InnerVolumeSpecName "kube-api-access-btcn6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:09:21.539299 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:21.539278    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "85a0ec97-0b7e-42f3-8ef0-bf97a54aa004" (UID: "85a0ec97-0b7e-42f3-8ef0-bf97a54aa004"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:09:21.539299 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:21.539287    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "85a0ec97-0b7e-42f3-8ef0-bf97a54aa004" (UID: "85a0ec97-0b7e-42f3-8ef0-bf97a54aa004"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 04:09:21.638113 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:21.638028    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-istio-envoy\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:09:21.638113 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:21.638058    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-workload-socket\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:09:21.638113 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:21.638067    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-istio-token\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:09:21.638113 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:21.638077    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-istio-podinfo\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:09:21.638113 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:21.638089    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-workload-certs\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:09:21.638113 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:21.638098    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-istiod-ca-cert\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:09:21.638113 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:21.638106    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-istio-data\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:09:21.638113 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:21.638113    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btcn6\" (UniqueName: \"kubernetes.io/projected/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004-kube-api-access-btcn6\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:09:22.124309 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:22.124277    2576 generic.go:356] "Generic (PLEG): container finished" podID="85a0ec97-0b7e-42f3-8ef0-bf97a54aa004" containerID="917e1af1df54800130182f994ff832cbde7cbb353fdcb96dba6fe8f3ce602ba8" exitCode=0
Jul 30 04:09:22.124791 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:22.124325    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"
Jul 30 04:09:22.124791 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:22.124365    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9" event={"ID":"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004","Type":"ContainerDied","Data":"917e1af1df54800130182f994ff832cbde7cbb353fdcb96dba6fe8f3ce602ba8"}
Jul 30 04:09:22.124791 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:22.124435    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9" event={"ID":"85a0ec97-0b7e-42f3-8ef0-bf97a54aa004","Type":"ContainerDied","Data":"62a512fd76ecf3d2964feaf8a952692fb057abd0f2655e75acda2a4a9d77e8ba"}
Jul 30 04:09:22.124791 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:22.124473    2576 scope.go:117] "RemoveContainer" containerID="917e1af1df54800130182f994ff832cbde7cbb353fdcb96dba6fe8f3ce602ba8"
Jul 30 04:09:22.133832 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:22.133815    2576 scope.go:117] "RemoveContainer" containerID="5e1dd70d87dc29ae6ca47d1f984d65181a5b9662fb10b42b31506ee3a985ee23"
Jul 30 04:09:22.141896 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:22.141877    2576 scope.go:117] "RemoveContainer" containerID="917e1af1df54800130182f994ff832cbde7cbb353fdcb96dba6fe8f3ce602ba8"
Jul 30 04:09:22.142186 ip-10-0-143-117 kubenswrapper[2576]: E0730 04:09:22.142163    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"917e1af1df54800130182f994ff832cbde7cbb353fdcb96dba6fe8f3ce602ba8\": container with ID starting with 917e1af1df54800130182f994ff832cbde7cbb353fdcb96dba6fe8f3ce602ba8 not found: ID does not exist" containerID="917e1af1df54800130182f994ff832cbde7cbb353fdcb96dba6fe8f3ce602ba8"
Jul 30 04:09:22.142325 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:22.142195    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"917e1af1df54800130182f994ff832cbde7cbb353fdcb96dba6fe8f3ce602ba8"} err="failed to get container status \"917e1af1df54800130182f994ff832cbde7cbb353fdcb96dba6fe8f3ce602ba8\": rpc error: code = NotFound desc = could not find container \"917e1af1df54800130182f994ff832cbde7cbb353fdcb96dba6fe8f3ce602ba8\": container with ID starting with 917e1af1df54800130182f994ff832cbde7cbb353fdcb96dba6fe8f3ce602ba8 not found: ID does not exist"
Jul 30 04:09:22.142325 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:22.142216    2576 scope.go:117] "RemoveContainer" containerID="5e1dd70d87dc29ae6ca47d1f984d65181a5b9662fb10b42b31506ee3a985ee23"
Jul 30 04:09:22.142499 ip-10-0-143-117 kubenswrapper[2576]: E0730 04:09:22.142476    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e1dd70d87dc29ae6ca47d1f984d65181a5b9662fb10b42b31506ee3a985ee23\": container with ID starting with 5e1dd70d87dc29ae6ca47d1f984d65181a5b9662fb10b42b31506ee3a985ee23 not found: ID does not exist" containerID="5e1dd70d87dc29ae6ca47d1f984d65181a5b9662fb10b42b31506ee3a985ee23"
Jul 30 04:09:22.142553 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:22.142513    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e1dd70d87dc29ae6ca47d1f984d65181a5b9662fb10b42b31506ee3a985ee23"} err="failed to get container status \"5e1dd70d87dc29ae6ca47d1f984d65181a5b9662fb10b42b31506ee3a985ee23\": rpc error: code = NotFound desc = could not find container \"5e1dd70d87dc29ae6ca47d1f984d65181a5b9662fb10b42b31506ee3a985ee23\": container with ID starting with 5e1dd70d87dc29ae6ca47d1f984d65181a5b9662fb10b42b31506ee3a985ee23 not found: ID does not exist"
Jul 30 04:09:22.145890 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:22.145867    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"]
Jul 30 04:09:22.149546 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:22.149523    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-e1a4694c-gw-openshift-default-797494b576-pctr9"]
Jul 30 04:09:23.052209 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:23.052178    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85a0ec97-0b7e-42f3-8ef0-bf97a54aa004" path="/var/lib/kubelet/pods/85a0ec97-0b7e-42f3-8ef0-bf97a54aa004/volumes"
Jul 30 04:09:30.819841 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:30.819802    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:09:34.677571 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:34.677531    2576 patch_prober.go:28] interesting pod/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": read tcp 10.134.0.2:57652->10.134.0.56:15021: read: connection reset by peer" start-of-body=
Jul 30 04:09:34.677999 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:34.677609    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc" podUID="9712893e-8e6d-4654-b16c-1190a2b2489f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": read tcp 10.134.0.2:57652->10.134.0.56:15021: read: connection reset by peer"
Jul 30 04:09:35.197829 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:35.197799    2576 generic.go:356] "Generic (PLEG): container finished" podID="9712893e-8e6d-4654-b16c-1190a2b2489f" containerID="ec4c2d64ed2d5f9dfd4851a753b7f5d37272a750e0b7f5e6e9fc638cdaa8557c" exitCode=0
Jul 30 04:09:35.198022 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:35.197852    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc" event={"ID":"9712893e-8e6d-4654-b16c-1190a2b2489f","Type":"ContainerDied","Data":"ec4c2d64ed2d5f9dfd4851a753b7f5d37272a750e0b7f5e6e9fc638cdaa8557c"}
Jul 30 04:09:35.198245 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:35.198232    2576 scope.go:117] "RemoveContainer" containerID="ec4c2d64ed2d5f9dfd4851a753b7f5d37272a750e0b7f5e6e9fc638cdaa8557c"
Jul 30 04:09:35.199223 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:35.199185    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:09:35.199524 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:35.199267    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:09:35.199524 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:35.199316    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:09:36.205085 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:36.205057    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc" event={"ID":"9712893e-8e6d-4654-b16c-1190a2b2489f","Type":"ContainerStarted","Data":"4d20b857eede464b3530a688b8c7635618faba4e79519cbbab9fa9fb0462f2ee"}
Jul 30 04:09:36.375003 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:36.374961    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:36.375260 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:36.375236    2576 patch_prober.go:28] interesting pod/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 30 04:09:36.375359 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:36.375306    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc" podUID="9712893e-8e6d-4654-b16c-1190a2b2489f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 30 04:09:37.375021 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:37.374984    2576 patch_prober.go:28] interesting pod/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 30 04:09:37.375516 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:37.375046    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc" podUID="9712893e-8e6d-4654-b16c-1190a2b2489f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 30 04:09:38.375188 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:38.375143    2576 patch_prober.go:28] interesting pod/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 30 04:09:38.375584 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:38.375204    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc" podUID="9712893e-8e6d-4654-b16c-1190a2b2489f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 30 04:09:39.374397 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:39.374354    2576 patch_prober.go:28] interesting pod/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 30 04:09:39.374588 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:39.374415    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc" podUID="9712893e-8e6d-4654-b16c-1190a2b2489f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 30 04:09:40.377575 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:40.377547    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:40.378042 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:40.377802    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:40.378583 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:40.378567    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:45.408633 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:45.408603    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d4857d1-4g5s2"]
Jul 30 04:09:45.409049 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:45.408977    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85a0ec97-0b7e-42f3-8ef0-bf97a54aa004" containerName="istio-proxy"
Jul 30 04:09:45.409049 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:45.408989    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="85a0ec97-0b7e-42f3-8ef0-bf97a54aa004" containerName="istio-proxy"
Jul 30 04:09:45.409049 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:45.409000    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85a0ec97-0b7e-42f3-8ef0-bf97a54aa004" containerName="istio-proxy"
Jul 30 04:09:45.409049 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:45.409005    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="85a0ec97-0b7e-42f3-8ef0-bf97a54aa004" containerName="istio-proxy"
Jul 30 04:09:45.409181 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:45.409057    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="85a0ec97-0b7e-42f3-8ef0-bf97a54aa004" containerName="istio-proxy"
Jul 30 04:09:45.409181 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:45.409067    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="85a0ec97-0b7e-42f3-8ef0-bf97a54aa004" containerName="istio-proxy"
Jul 30 04:09:45.411053 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:45.411037    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d4857d1-4g5s2"
Jul 30 04:09:45.413460 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:45.413431    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5cg4s\""
Jul 30 04:09:45.418549 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:45.418527    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d4857d1-4g5s2"]
Jul 30 04:09:45.565772 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:45.565736    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ed642ef-04ba-4996-ab12-2eaa9eb0dd79-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-1d4857d1-4g5s2\" (UID: \"1ed642ef-04ba-4996-ab12-2eaa9eb0dd79\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d4857d1-4g5s2"
Jul 30 04:09:45.666969 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:45.666853    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ed642ef-04ba-4996-ab12-2eaa9eb0dd79-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-1d4857d1-4g5s2\" (UID: \"1ed642ef-04ba-4996-ab12-2eaa9eb0dd79\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d4857d1-4g5s2"
Jul 30 04:09:45.667504 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:45.667482    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ed642ef-04ba-4996-ab12-2eaa9eb0dd79-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-1d4857d1-4g5s2\" (UID: \"1ed642ef-04ba-4996-ab12-2eaa9eb0dd79\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d4857d1-4g5s2"
Jul 30 04:09:45.722874 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:45.722849    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d4857d1-4g5s2"
Jul 30 04:09:45.873528 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:45.873499    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d4857d1-4g5s2"]
Jul 30 04:09:45.874894 ip-10-0-143-117 kubenswrapper[2576]: W0730 04:09:45.874871    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ed642ef_04ba_4996_ab12_2eaa9eb0dd79.slice/crio-cb816529d06c75eb0f7a98e42c3b813d9e86a7ad24210bfb2c2af30365332922 WatchSource:0}: Error finding container cb816529d06c75eb0f7a98e42c3b813d9e86a7ad24210bfb2c2af30365332922: Status 404 returned error can't find the container with id cb816529d06c75eb0f7a98e42c3b813d9e86a7ad24210bfb2c2af30365332922
Jul 30 04:09:46.259502 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:46.259470    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d4857d1-4g5s2" event={"ID":"1ed642ef-04ba-4996-ab12-2eaa9eb0dd79","Type":"ContainerStarted","Data":"fd275687f868d9972125c8c9badadc6ad62018a469528279376849480da5152a"}
Jul 30 04:09:46.259670 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:46.259519    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d4857d1-4g5s2" event={"ID":"1ed642ef-04ba-4996-ab12-2eaa9eb0dd79","Type":"ContainerStarted","Data":"cb816529d06c75eb0f7a98e42c3b813d9e86a7ad24210bfb2c2af30365332922"}
Jul 30 04:09:46.276560 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:46.276516    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d4857d1-4g5s2" podStartSLOduration=1.27650213 podStartE2EDuration="1.27650213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:09:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:09:46.274383754 +0000 UTC m=+2091.892615993" watchObservedRunningTime="2026-07-30 04:09:46.27650213 +0000 UTC m=+2091.894734371"
Jul 30 04:09:47.266509 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:47.266473    2576 generic.go:356] "Generic (PLEG): container finished" podID="1ed642ef-04ba-4996-ab12-2eaa9eb0dd79" containerID="fd275687f868d9972125c8c9badadc6ad62018a469528279376849480da5152a" exitCode=0
Jul 30 04:09:47.266877 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:47.266556    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d4857d1-4g5s2" event={"ID":"1ed642ef-04ba-4996-ab12-2eaa9eb0dd79","Type":"ContainerDied","Data":"fd275687f868d9972125c8c9badadc6ad62018a469528279376849480da5152a"}
Jul 30 04:09:48.405483 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:48.405456    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d4857d1-4g5s2"
Jul 30 04:09:48.593221 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:48.593126    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ed642ef-04ba-4996-ab12-2eaa9eb0dd79-maas-api-service-ca\") pod \"1ed642ef-04ba-4996-ab12-2eaa9eb0dd79\" (UID: \"1ed642ef-04ba-4996-ab12-2eaa9eb0dd79\") "
Jul 30 04:09:48.593475 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:48.593444    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ed642ef-04ba-4996-ab12-2eaa9eb0dd79-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1ed642ef-04ba-4996-ab12-2eaa9eb0dd79" (UID: "1ed642ef-04ba-4996-ab12-2eaa9eb0dd79"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:09:48.694625 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:48.694579    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ed642ef-04ba-4996-ab12-2eaa9eb0dd79-maas-api-service-ca\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:09:49.277802 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:49.277771    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d4857d1-4g5s2" event={"ID":"1ed642ef-04ba-4996-ab12-2eaa9eb0dd79","Type":"ContainerDied","Data":"cb816529d06c75eb0f7a98e42c3b813d9e86a7ad24210bfb2c2af30365332922"}
Jul 30 04:09:49.277987 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:49.277821    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cb816529d06c75eb0f7a98e42c3b813d9e86a7ad24210bfb2c2af30365332922"
Jul 30 04:09:49.277987 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:49.277777    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d4857d1-4g5s2"
Jul 30 04:09:51.032486 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:51.032450    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:09:51.095886 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:51.095847    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:09:51.191716 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:51.191683    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d4857d1-4g5s2"]
Jul 30 04:09:51.194240 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:51.194214    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d4857d1-4g5s2"]
Jul 30 04:09:51.194494 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:51.194470    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:09:51.252736 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:51.252672    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:09:52.454956 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:52.454914    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"]
Jul 30 04:09:52.455362 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:52.455127    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc" podUID="9712893e-8e6d-4654-b16c-1190a2b2489f" containerName="istio-proxy" containerID="cri-o://4d20b857eede464b3530a688b8c7635618faba4e79519cbbab9fa9fb0462f2ee" gracePeriod=30
Jul 30 04:09:52.469275 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:52.469240    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:09:52.710429 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:52.710321    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:09:52.941550 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:52.941515    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:09:53.051106 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.051079    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ed642ef-04ba-4996-ab12-2eaa9eb0dd79" path="/var/lib/kubelet/pods/1ed642ef-04ba-4996-ab12-2eaa9eb0dd79/volumes"
Jul 30 04:09:53.244747 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.244717    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"]
Jul 30 04:09:53.245124 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.245110    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ed642ef-04ba-4996-ab12-2eaa9eb0dd79" containerName="revoke-keys"
Jul 30 04:09:53.245215 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.245126    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ed642ef-04ba-4996-ab12-2eaa9eb0dd79" containerName="revoke-keys"
Jul 30 04:09:53.245215 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.245217    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ed642ef-04ba-4996-ab12-2eaa9eb0dd79" containerName="revoke-keys"
Jul 30 04:09:53.247585 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.247569    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:09:53.250103 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.250087    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-3e3f5a98-openshift-default-dockercfg-xw7lg\""
Jul 30 04:09:53.260530 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.260502    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"]
Jul 30 04:09:53.268637 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.268616    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:09:53.335005 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.334915    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/525e2d44-3744-40f8-9888-ed20784e0e9a-istio-envoy\") pod \"e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8\" (UID: \"525e2d44-3744-40f8-9888-ed20784e0e9a\") " pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:09:53.335240 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.335210    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/525e2d44-3744-40f8-9888-ed20784e0e9a-workload-socket\") pod \"e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8\" (UID: \"525e2d44-3744-40f8-9888-ed20784e0e9a\") " pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:09:53.335372 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.335254    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/525e2d44-3744-40f8-9888-ed20784e0e9a-istiod-ca-cert\") pod \"e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8\" (UID: \"525e2d44-3744-40f8-9888-ed20784e0e9a\") " pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:09:53.335372 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.335281    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/525e2d44-3744-40f8-9888-ed20784e0e9a-workload-certs\") pod \"e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8\" (UID: \"525e2d44-3744-40f8-9888-ed20784e0e9a\") " pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:09:53.335372 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.335305    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/525e2d44-3744-40f8-9888-ed20784e0e9a-istio-data\") pod \"e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8\" (UID: \"525e2d44-3744-40f8-9888-ed20784e0e9a\") " pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:09:53.335551 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.335413    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/525e2d44-3744-40f8-9888-ed20784e0e9a-istio-token\") pod \"e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8\" (UID: \"525e2d44-3744-40f8-9888-ed20784e0e9a\") " pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:09:53.335551 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.335453    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khtqg\" (UniqueName: \"kubernetes.io/projected/525e2d44-3744-40f8-9888-ed20784e0e9a-kube-api-access-khtqg\") pod \"e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8\" (UID: \"525e2d44-3744-40f8-9888-ed20784e0e9a\") " pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:09:53.335551 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.335515    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/525e2d44-3744-40f8-9888-ed20784e0e9a-credential-socket\") pod \"e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8\" (UID: \"525e2d44-3744-40f8-9888-ed20784e0e9a\") " pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:09:53.335716 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.335550    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/525e2d44-3744-40f8-9888-ed20784e0e9a-istio-podinfo\") pod \"e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8\" (UID: \"525e2d44-3744-40f8-9888-ed20784e0e9a\") " pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:09:53.436886 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.436847    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/525e2d44-3744-40f8-9888-ed20784e0e9a-istio-token\") pod \"e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8\" (UID: \"525e2d44-3744-40f8-9888-ed20784e0e9a\") " pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:09:53.436886 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.436887    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khtqg\" (UniqueName: \"kubernetes.io/projected/525e2d44-3744-40f8-9888-ed20784e0e9a-kube-api-access-khtqg\") pod \"e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8\" (UID: \"525e2d44-3744-40f8-9888-ed20784e0e9a\") " pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:09:53.437170 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.436946    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/525e2d44-3744-40f8-9888-ed20784e0e9a-credential-socket\") pod \"e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8\" (UID: \"525e2d44-3744-40f8-9888-ed20784e0e9a\") " pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:09:53.437170 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.436975    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/525e2d44-3744-40f8-9888-ed20784e0e9a-istio-podinfo\") pod \"e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8\" (UID: \"525e2d44-3744-40f8-9888-ed20784e0e9a\") " pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:09:53.437170 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.437011    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/525e2d44-3744-40f8-9888-ed20784e0e9a-istio-envoy\") pod \"e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8\" (UID: \"525e2d44-3744-40f8-9888-ed20784e0e9a\") " pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:09:53.437170 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.437033    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/525e2d44-3744-40f8-9888-ed20784e0e9a-workload-socket\") pod \"e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8\" (UID: \"525e2d44-3744-40f8-9888-ed20784e0e9a\") " pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:09:53.437432 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.437168    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/525e2d44-3744-40f8-9888-ed20784e0e9a-istiod-ca-cert\") pod \"e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8\" (UID: \"525e2d44-3744-40f8-9888-ed20784e0e9a\") " pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:09:53.437432 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.437215    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/525e2d44-3744-40f8-9888-ed20784e0e9a-workload-certs\") pod \"e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8\" (UID: \"525e2d44-3744-40f8-9888-ed20784e0e9a\") " pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:09:53.437432 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.437241    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/525e2d44-3744-40f8-9888-ed20784e0e9a-istio-data\") pod \"e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8\" (UID: \"525e2d44-3744-40f8-9888-ed20784e0e9a\") " pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:09:53.437617 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.437441    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/525e2d44-3744-40f8-9888-ed20784e0e9a-credential-socket\") pod \"e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8\" (UID: \"525e2d44-3744-40f8-9888-ed20784e0e9a\") " pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:09:53.437617 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.437475    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/525e2d44-3744-40f8-9888-ed20784e0e9a-workload-socket\") pod \"e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8\" (UID: \"525e2d44-3744-40f8-9888-ed20784e0e9a\") " pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:09:53.437617 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.437575    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/525e2d44-3744-40f8-9888-ed20784e0e9a-workload-certs\") pod \"e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8\" (UID: \"525e2d44-3744-40f8-9888-ed20784e0e9a\") " pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:09:53.437771 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.437617    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/525e2d44-3744-40f8-9888-ed20784e0e9a-istio-data\") pod \"e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8\" (UID: \"525e2d44-3744-40f8-9888-ed20784e0e9a\") " pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:09:53.437887 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.437865    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/525e2d44-3744-40f8-9888-ed20784e0e9a-istiod-ca-cert\") pod \"e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8\" (UID: \"525e2d44-3744-40f8-9888-ed20784e0e9a\") " pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:09:53.439374 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.439351    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/525e2d44-3744-40f8-9888-ed20784e0e9a-istio-envoy\") pod \"e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8\" (UID: \"525e2d44-3744-40f8-9888-ed20784e0e9a\") " pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:09:53.439574 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.439557    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/525e2d44-3744-40f8-9888-ed20784e0e9a-istio-podinfo\") pod \"e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8\" (UID: \"525e2d44-3744-40f8-9888-ed20784e0e9a\") " pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:09:53.449050 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.449029    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/525e2d44-3744-40f8-9888-ed20784e0e9a-istio-token\") pod \"e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8\" (UID: \"525e2d44-3744-40f8-9888-ed20784e0e9a\") " pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:09:53.449236 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.449218    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khtqg\" (UniqueName: \"kubernetes.io/projected/525e2d44-3744-40f8-9888-ed20784e0e9a-kube-api-access-khtqg\") pod \"e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8\" (UID: \"525e2d44-3744-40f8-9888-ed20784e0e9a\") " pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:09:53.481442 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.481414    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:09:53.558132 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.558099    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:09:53.695827 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.695799    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"]
Jul 30 04:09:53.697427 ip-10-0-143-117 kubenswrapper[2576]: W0730 04:09:53.697402    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod525e2d44_3744_40f8_9888_ed20784e0e9a.slice/crio-55a152738d14149cd596065571c1fa043f316115a210cee28139efaacf02312e WatchSource:0}: Error finding container 55a152738d14149cd596065571c1fa043f316115a210cee28139efaacf02312e: Status 404 returned error can't find the container with id 55a152738d14149cd596065571c1fa043f316115a210cee28139efaacf02312e
Jul 30 04:09:53.699486 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.699454    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:09:53.699562 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.699520    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:09:53.699562 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:53.699551    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:09:54.305579 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:54.305545    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8" event={"ID":"525e2d44-3744-40f8-9888-ed20784e0e9a","Type":"ContainerStarted","Data":"926c13494e0b710a7b3120cbf3dc6ab7e3ab88ccf127c46e438236cf8a43b072"}
Jul 30 04:09:54.305765 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:54.305608    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8" event={"ID":"525e2d44-3744-40f8-9888-ed20784e0e9a","Type":"ContainerStarted","Data":"55a152738d14149cd596065571c1fa043f316115a210cee28139efaacf02312e"}
Jul 30 04:09:54.326814 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:54.326718    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8" podStartSLOduration=1.326699665 podStartE2EDuration="1.326699665s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:09:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:09:54.324086456 +0000 UTC m=+2099.942318698" watchObservedRunningTime="2026-07-30 04:09:54.326699665 +0000 UTC m=+2099.944931907"
Jul 30 04:09:54.558676 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:54.558588    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:09:54.563206 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:54.563183    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:09:55.225507 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:55.225475    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:09:55.311689 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:55.311660    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:09:55.312688 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:55.312669    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:09:55.379728 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:55.379686    2576 patch_prober.go:28] interesting pod/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 30 04:09:55.379878 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:55.379752    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc" podUID="9712893e-8e6d-4654-b16c-1190a2b2489f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 04:09:55.668342 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:55.668312    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:09:55.765973 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:55.765905    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:09:56.945362 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:56.945326    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:09:57.290388 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:57.290358    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:09:57.510353 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:57.510315    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:09:57.612325 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:57.612288    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:09:57.718381 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:57.718341    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:57.780738 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:57.780707    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9712893e-8e6d-4654-b16c-1190a2b2489f-credential-socket\") pod \"9712893e-8e6d-4654-b16c-1190a2b2489f\" (UID: \"9712893e-8e6d-4654-b16c-1190a2b2489f\") "
Jul 30 04:09:57.780738 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:57.780750    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9712893e-8e6d-4654-b16c-1190a2b2489f-workload-certs\") pod \"9712893e-8e6d-4654-b16c-1190a2b2489f\" (UID: \"9712893e-8e6d-4654-b16c-1190a2b2489f\") "
Jul 30 04:09:57.781021 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:57.780795    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9712893e-8e6d-4654-b16c-1190a2b2489f-istio-token\") pod \"9712893e-8e6d-4654-b16c-1190a2b2489f\" (UID: \"9712893e-8e6d-4654-b16c-1190a2b2489f\") "
Jul 30 04:09:57.781021 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:57.780882    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9712893e-8e6d-4654-b16c-1190a2b2489f-istio-envoy\") pod \"9712893e-8e6d-4654-b16c-1190a2b2489f\" (UID: \"9712893e-8e6d-4654-b16c-1190a2b2489f\") "
Jul 30 04:09:57.781021 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:57.780945    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9712893e-8e6d-4654-b16c-1190a2b2489f-workload-socket\") pod \"9712893e-8e6d-4654-b16c-1190a2b2489f\" (UID: \"9712893e-8e6d-4654-b16c-1190a2b2489f\") "
Jul 30 04:09:57.781021 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:57.780974    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9712893e-8e6d-4654-b16c-1190a2b2489f-istio-data\") pod \"9712893e-8e6d-4654-b16c-1190a2b2489f\" (UID: \"9712893e-8e6d-4654-b16c-1190a2b2489f\") "
Jul 30 04:09:57.781021 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:57.781003    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ks2bd\" (UniqueName: \"kubernetes.io/projected/9712893e-8e6d-4654-b16c-1190a2b2489f-kube-api-access-ks2bd\") pod \"9712893e-8e6d-4654-b16c-1190a2b2489f\" (UID: \"9712893e-8e6d-4654-b16c-1190a2b2489f\") "
Jul 30 04:09:57.781283 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:57.781030    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9712893e-8e6d-4654-b16c-1190a2b2489f-istiod-ca-cert\") pod \"9712893e-8e6d-4654-b16c-1190a2b2489f\" (UID: \"9712893e-8e6d-4654-b16c-1190a2b2489f\") "
Jul 30 04:09:57.781283 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:57.781025    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9712893e-8e6d-4654-b16c-1190a2b2489f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9712893e-8e6d-4654-b16c-1190a2b2489f" (UID: "9712893e-8e6d-4654-b16c-1190a2b2489f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:09:57.781283 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:57.781067    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9712893e-8e6d-4654-b16c-1190a2b2489f-istio-podinfo\") pod \"9712893e-8e6d-4654-b16c-1190a2b2489f\" (UID: \"9712893e-8e6d-4654-b16c-1190a2b2489f\") "
Jul 30 04:09:57.781283 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:57.781130    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9712893e-8e6d-4654-b16c-1190a2b2489f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9712893e-8e6d-4654-b16c-1190a2b2489f" (UID: "9712893e-8e6d-4654-b16c-1190a2b2489f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:09:57.781490 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:57.781406    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9712893e-8e6d-4654-b16c-1190a2b2489f-credential-socket\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:09:57.781490 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:57.781424    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9712893e-8e6d-4654-b16c-1190a2b2489f-workload-certs\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:09:57.781490 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:57.781365    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9712893e-8e6d-4654-b16c-1190a2b2489f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9712893e-8e6d-4654-b16c-1190a2b2489f" (UID: "9712893e-8e6d-4654-b16c-1190a2b2489f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:09:57.781631 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:57.781602    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9712893e-8e6d-4654-b16c-1190a2b2489f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9712893e-8e6d-4654-b16c-1190a2b2489f" (UID: "9712893e-8e6d-4654-b16c-1190a2b2489f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:09:57.782203 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:57.782159    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9712893e-8e6d-4654-b16c-1190a2b2489f-istio-data" (OuterVolumeSpecName: "istio-data") pod "9712893e-8e6d-4654-b16c-1190a2b2489f" (UID: "9712893e-8e6d-4654-b16c-1190a2b2489f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:09:57.783598 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:57.783571    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9712893e-8e6d-4654-b16c-1190a2b2489f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9712893e-8e6d-4654-b16c-1190a2b2489f" (UID: "9712893e-8e6d-4654-b16c-1190a2b2489f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:09:57.783598 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:57.783585    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9712893e-8e6d-4654-b16c-1190a2b2489f-istio-token" (OuterVolumeSpecName: "istio-token") pod "9712893e-8e6d-4654-b16c-1190a2b2489f" (UID: "9712893e-8e6d-4654-b16c-1190a2b2489f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:09:57.783768 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:57.783667    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9712893e-8e6d-4654-b16c-1190a2b2489f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9712893e-8e6d-4654-b16c-1190a2b2489f" (UID: "9712893e-8e6d-4654-b16c-1190a2b2489f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 04:09:57.783834 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:57.783810    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9712893e-8e6d-4654-b16c-1190a2b2489f-kube-api-access-ks2bd" (OuterVolumeSpecName: "kube-api-access-ks2bd") pod "9712893e-8e6d-4654-b16c-1190a2b2489f" (UID: "9712893e-8e6d-4654-b16c-1190a2b2489f"). InnerVolumeSpecName "kube-api-access-ks2bd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:09:57.882471 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:57.882391    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9712893e-8e6d-4654-b16c-1190a2b2489f-istio-token\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:09:57.882471 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:57.882417    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9712893e-8e6d-4654-b16c-1190a2b2489f-istio-envoy\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:09:57.882471 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:57.882426    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9712893e-8e6d-4654-b16c-1190a2b2489f-workload-socket\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:09:57.882471 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:57.882435    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9712893e-8e6d-4654-b16c-1190a2b2489f-istio-data\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:09:57.882471 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:57.882444    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ks2bd\" (UniqueName: \"kubernetes.io/projected/9712893e-8e6d-4654-b16c-1190a2b2489f-kube-api-access-ks2bd\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:09:57.882471 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:57.882454    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9712893e-8e6d-4654-b16c-1190a2b2489f-istiod-ca-cert\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:09:57.882471 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:57.882462    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9712893e-8e6d-4654-b16c-1190a2b2489f-istio-podinfo\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:09:58.328273 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:58.328234    2576 generic.go:356] "Generic (PLEG): container finished" podID="9712893e-8e6d-4654-b16c-1190a2b2489f" containerID="4d20b857eede464b3530a688b8c7635618faba4e79519cbbab9fa9fb0462f2ee" exitCode=0
Jul 30 04:09:58.328273 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:58.328272    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"
Jul 30 04:09:58.328797 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:58.328307    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc" event={"ID":"9712893e-8e6d-4654-b16c-1190a2b2489f","Type":"ContainerDied","Data":"4d20b857eede464b3530a688b8c7635618faba4e79519cbbab9fa9fb0462f2ee"}
Jul 30 04:09:58.328797 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:58.328376    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc" event={"ID":"9712893e-8e6d-4654-b16c-1190a2b2489f","Type":"ContainerDied","Data":"d358c07441bac7025470b1c5fb6f470256f18257e8f48fd937a3ad5c1b21cbe5"}
Jul 30 04:09:58.328797 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:58.328411    2576 scope.go:117] "RemoveContainer" containerID="4d20b857eede464b3530a688b8c7635618faba4e79519cbbab9fa9fb0462f2ee"
Jul 30 04:09:58.339079 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:58.339056    2576 scope.go:117] "RemoveContainer" containerID="ec4c2d64ed2d5f9dfd4851a753b7f5d37272a750e0b7f5e6e9fc638cdaa8557c"
Jul 30 04:09:58.348254 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:58.348236    2576 scope.go:117] "RemoveContainer" containerID="4d20b857eede464b3530a688b8c7635618faba4e79519cbbab9fa9fb0462f2ee"
Jul 30 04:09:58.348499 ip-10-0-143-117 kubenswrapper[2576]: E0730 04:09:58.348482    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d20b857eede464b3530a688b8c7635618faba4e79519cbbab9fa9fb0462f2ee\": container with ID starting with 4d20b857eede464b3530a688b8c7635618faba4e79519cbbab9fa9fb0462f2ee not found: ID does not exist" containerID="4d20b857eede464b3530a688b8c7635618faba4e79519cbbab9fa9fb0462f2ee"
Jul 30 04:09:58.348553 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:58.348508    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d20b857eede464b3530a688b8c7635618faba4e79519cbbab9fa9fb0462f2ee"} err="failed to get container status \"4d20b857eede464b3530a688b8c7635618faba4e79519cbbab9fa9fb0462f2ee\": rpc error: code = NotFound desc = could not find container \"4d20b857eede464b3530a688b8c7635618faba4e79519cbbab9fa9fb0462f2ee\": container with ID starting with 4d20b857eede464b3530a688b8c7635618faba4e79519cbbab9fa9fb0462f2ee not found: ID does not exist"
Jul 30 04:09:58.348553 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:58.348525    2576 scope.go:117] "RemoveContainer" containerID="ec4c2d64ed2d5f9dfd4851a753b7f5d37272a750e0b7f5e6e9fc638cdaa8557c"
Jul 30 04:09:58.348735 ip-10-0-143-117 kubenswrapper[2576]: E0730 04:09:58.348719    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec4c2d64ed2d5f9dfd4851a753b7f5d37272a750e0b7f5e6e9fc638cdaa8557c\": container with ID starting with ec4c2d64ed2d5f9dfd4851a753b7f5d37272a750e0b7f5e6e9fc638cdaa8557c not found: ID does not exist" containerID="ec4c2d64ed2d5f9dfd4851a753b7f5d37272a750e0b7f5e6e9fc638cdaa8557c"
Jul 30 04:09:58.348772 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:58.348739    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec4c2d64ed2d5f9dfd4851a753b7f5d37272a750e0b7f5e6e9fc638cdaa8557c"} err="failed to get container status \"ec4c2d64ed2d5f9dfd4851a753b7f5d37272a750e0b7f5e6e9fc638cdaa8557c\": rpc error: code = NotFound desc = could not find container \"ec4c2d64ed2d5f9dfd4851a753b7f5d37272a750e0b7f5e6e9fc638cdaa8557c\": container with ID starting with ec4c2d64ed2d5f9dfd4851a753b7f5d37272a750e0b7f5e6e9fc638cdaa8557c not found: ID does not exist"
Jul 30 04:09:58.352831 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:58.352805    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"]
Jul 30 04:09:58.357916 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:58.357896    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-1d4857d1-openshift-default-54cd64c95b-c2rmc"]
Jul 30 04:09:59.051703 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:09:59.051671    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9712893e-8e6d-4654-b16c-1190a2b2489f" path="/var/lib/kubelet/pods/9712893e-8e6d-4654-b16c-1190a2b2489f/volumes"
Jul 30 04:10:02.545988 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:02.545948    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:10:06.720852 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:06.720819    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:10:10.313196 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:10.313160    2576 patch_prober.go:28] interesting pod/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Jul 30 04:10:10.313677 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:10.313220    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8" podUID="525e2d44-3744-40f8-9888-ed20784e0e9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Jul 30 04:10:10.397336 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:10.397303    2576 generic.go:356] "Generic (PLEG): container finished" podID="525e2d44-3744-40f8-9888-ed20784e0e9a" containerID="926c13494e0b710a7b3120cbf3dc6ab7e3ab88ccf127c46e438236cf8a43b072" exitCode=0
Jul 30 04:10:10.397504 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:10.397379    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8" event={"ID":"525e2d44-3744-40f8-9888-ed20784e0e9a","Type":"ContainerDied","Data":"926c13494e0b710a7b3120cbf3dc6ab7e3ab88ccf127c46e438236cf8a43b072"}
Jul 30 04:10:10.397779 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:10.397762    2576 scope.go:117] "RemoveContainer" containerID="926c13494e0b710a7b3120cbf3dc6ab7e3ab88ccf127c46e438236cf8a43b072"
Jul 30 04:10:10.398721 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:10.398696    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:10:10.398774 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:10.398760    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:10:10.398812 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:10.398794    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:10:11.404470 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:11.404438    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8" event={"ID":"525e2d44-3744-40f8-9888-ed20784e0e9a","Type":"ContainerStarted","Data":"09d9f03f5a8cbf8c6e40dcd4c63b7b6dc9c6688faf2a45b15a91c3e4945c373f"}
Jul 30 04:10:11.558347 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:11.558312    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:10:11.558558 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:11.558452    2576 patch_prober.go:28] interesting pod/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Jul 30 04:10:11.558558 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:11.558503    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8" podUID="525e2d44-3744-40f8-9888-ed20784e0e9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Jul 30 04:10:12.558848 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:12.558813    2576 patch_prober.go:28] interesting pod/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Jul 30 04:10:12.559246 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:12.558875    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8" podUID="525e2d44-3744-40f8-9888-ed20784e0e9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Jul 30 04:10:13.559131 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:13.559093    2576 patch_prober.go:28] interesting pod/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Jul 30 04:10:13.559598 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:13.559152    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8" podUID="525e2d44-3744-40f8-9888-ed20784e0e9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Jul 30 04:10:14.559012 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:14.558971    2576 patch_prober.go:28] interesting pod/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Jul 30 04:10:14.559185 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:14.559034    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8" podUID="525e2d44-3744-40f8-9888-ed20784e0e9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Jul 30 04:10:15.562229 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:15.562196    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:10:15.562659 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:15.562590    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:10:15.563254 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:15.563234    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:10:20.593911 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:20.593875    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3e3f5a98-tsnrm"]
Jul 30 04:10:20.594344 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:20.594292    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9712893e-8e6d-4654-b16c-1190a2b2489f" containerName="istio-proxy"
Jul 30 04:10:20.594344 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:20.594304    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9712893e-8e6d-4654-b16c-1190a2b2489f" containerName="istio-proxy"
Jul 30 04:10:20.594344 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:20.594314    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9712893e-8e6d-4654-b16c-1190a2b2489f" containerName="istio-proxy"
Jul 30 04:10:20.594344 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:20.594319    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9712893e-8e6d-4654-b16c-1190a2b2489f" containerName="istio-proxy"
Jul 30 04:10:20.594481 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:20.594389    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9712893e-8e6d-4654-b16c-1190a2b2489f" containerName="istio-proxy"
Jul 30 04:10:20.596594 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:20.596577    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3e3f5a98-tsnrm"
Jul 30 04:10:20.599091 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:20.599069    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5cg4s\""
Jul 30 04:10:20.604686 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:20.604666    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3e3f5a98-tsnrm"]
Jul 30 04:10:20.696749 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:20.696720    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01e6bce1-954c-41f4-a9ed-e8c15f75287a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-3e3f5a98-tsnrm\" (UID: \"01e6bce1-954c-41f4-a9ed-e8c15f75287a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3e3f5a98-tsnrm"
Jul 30 04:10:20.797736 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:20.797687    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01e6bce1-954c-41f4-a9ed-e8c15f75287a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-3e3f5a98-tsnrm\" (UID: \"01e6bce1-954c-41f4-a9ed-e8c15f75287a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3e3f5a98-tsnrm"
Jul 30 04:10:20.798340 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:20.798321    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01e6bce1-954c-41f4-a9ed-e8c15f75287a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-3e3f5a98-tsnrm\" (UID: \"01e6bce1-954c-41f4-a9ed-e8c15f75287a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3e3f5a98-tsnrm"
Jul 30 04:10:20.908174 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:20.908097    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3e3f5a98-tsnrm"
Jul 30 04:10:21.034980 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:21.034955    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3e3f5a98-tsnrm"]
Jul 30 04:10:21.037444 ip-10-0-143-117 kubenswrapper[2576]: W0730 04:10:21.037414    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01e6bce1_954c_41f4_a9ed_e8c15f75287a.slice/crio-beb88ef370381e5e4bcbb3c26005b3246ba66f11b22b3144cdf967906a1137f2 WatchSource:0}: Error finding container beb88ef370381e5e4bcbb3c26005b3246ba66f11b22b3144cdf967906a1137f2: Status 404 returned error can't find the container with id beb88ef370381e5e4bcbb3c26005b3246ba66f11b22b3144cdf967906a1137f2
Jul 30 04:10:21.458617 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:21.458580    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3e3f5a98-tsnrm" event={"ID":"01e6bce1-954c-41f4-a9ed-e8c15f75287a","Type":"ContainerStarted","Data":"5da4617778df8d370f63431bac3eacfcd6e6f8df978b2b48dc7cfa369b23abf4"}
Jul 30 04:10:21.458791 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:21.458632    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3e3f5a98-tsnrm" event={"ID":"01e6bce1-954c-41f4-a9ed-e8c15f75287a","Type":"ContainerStarted","Data":"beb88ef370381e5e4bcbb3c26005b3246ba66f11b22b3144cdf967906a1137f2"}
Jul 30 04:10:21.474746 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:21.474698    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3e3f5a98-tsnrm" podStartSLOduration=1.474684608 podStartE2EDuration="1.474684608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:10:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:10:21.47212082 +0000 UTC m=+2127.090353057" watchObservedRunningTime="2026-07-30 04:10:21.474684608 +0000 UTC m=+2127.092916848"
Jul 30 04:10:22.465347 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:22.465314    2576 generic.go:356] "Generic (PLEG): container finished" podID="01e6bce1-954c-41f4-a9ed-e8c15f75287a" containerID="5da4617778df8d370f63431bac3eacfcd6e6f8df978b2b48dc7cfa369b23abf4" exitCode=0
Jul 30 04:10:22.465775 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:22.465391    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3e3f5a98-tsnrm" event={"ID":"01e6bce1-954c-41f4-a9ed-e8c15f75287a","Type":"ContainerDied","Data":"5da4617778df8d370f63431bac3eacfcd6e6f8df978b2b48dc7cfa369b23abf4"}
Jul 30 04:10:23.602600 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:23.602576    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3e3f5a98-tsnrm"
Jul 30 04:10:23.726158 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:23.726069    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01e6bce1-954c-41f4-a9ed-e8c15f75287a-maas-api-service-ca\") pod \"01e6bce1-954c-41f4-a9ed-e8c15f75287a\" (UID: \"01e6bce1-954c-41f4-a9ed-e8c15f75287a\") "
Jul 30 04:10:23.726402 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:23.726379    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01e6bce1-954c-41f4-a9ed-e8c15f75287a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "01e6bce1-954c-41f4-a9ed-e8c15f75287a" (UID: "01e6bce1-954c-41f4-a9ed-e8c15f75287a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:10:23.726484 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:23.726472    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01e6bce1-954c-41f4-a9ed-e8c15f75287a-maas-api-service-ca\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:10:24.477428 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:24.477401    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3e3f5a98-tsnrm"
Jul 30 04:10:24.477609 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:24.477437    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3e3f5a98-tsnrm" event={"ID":"01e6bce1-954c-41f4-a9ed-e8c15f75287a","Type":"ContainerDied","Data":"beb88ef370381e5e4bcbb3c26005b3246ba66f11b22b3144cdf967906a1137f2"}
Jul 30 04:10:24.477609 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:24.477484    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="beb88ef370381e5e4bcbb3c26005b3246ba66f11b22b3144cdf967906a1137f2"
Jul 30 04:10:27.468549 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:27.468512    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:10:27.527382 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:27.527348    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:10:27.583966 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:27.583902    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:10:27.623187 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:27.623155    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3e3f5a98-tsnrm"]
Jul 30 04:10:27.625283 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:27.625261    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3e3f5a98-tsnrm"]
Jul 30 04:10:27.644711 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:27.644679    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:10:27.758572 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:27.758493    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"]
Jul 30 04:10:27.758768 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:27.758745    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8" podUID="525e2d44-3744-40f8-9888-ed20784e0e9a" containerName="istio-proxy" containerID="cri-o://09d9f03f5a8cbf8c6e40dcd4c63b7b6dc9c6688faf2a45b15a91c3e4945c373f" gracePeriod=30
Jul 30 04:10:27.772628 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:27.772594    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:10:27.990461 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:27.990431    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:10:28.437011 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:28.436978    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:10:28.535704 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:28.535672    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:10:28.656028 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:28.655989    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:10:29.051692 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:29.051660    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01e6bce1-954c-41f4-a9ed-e8c15f75287a" path="/var/lib/kubelet/pods/01e6bce1-954c-41f4-a9ed-e8c15f75287a/volumes"
Jul 30 04:10:30.564997 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:30.564963    2576 patch_prober.go:28] interesting pod/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 30 04:10:30.565372 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:30.565018    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8" podUID="525e2d44-3744-40f8-9888-ed20784e0e9a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 04:10:31.523457 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:31.523423    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:10:33.016335 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.016311    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:10:33.110762 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.110729    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/525e2d44-3744-40f8-9888-ed20784e0e9a-istio-podinfo\") pod \"525e2d44-3744-40f8-9888-ed20784e0e9a\" (UID: \"525e2d44-3744-40f8-9888-ed20784e0e9a\") "
Jul 30 04:10:33.110985 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.110780    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/525e2d44-3744-40f8-9888-ed20784e0e9a-istio-envoy\") pod \"525e2d44-3744-40f8-9888-ed20784e0e9a\" (UID: \"525e2d44-3744-40f8-9888-ed20784e0e9a\") "
Jul 30 04:10:33.110985 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.110832    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/525e2d44-3744-40f8-9888-ed20784e0e9a-workload-socket\") pod \"525e2d44-3744-40f8-9888-ed20784e0e9a\" (UID: \"525e2d44-3744-40f8-9888-ed20784e0e9a\") "
Jul 30 04:10:33.110985 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.110854    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/525e2d44-3744-40f8-9888-ed20784e0e9a-istiod-ca-cert\") pod \"525e2d44-3744-40f8-9888-ed20784e0e9a\" (UID: \"525e2d44-3744-40f8-9888-ed20784e0e9a\") "
Jul 30 04:10:33.110985 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.110889    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/525e2d44-3744-40f8-9888-ed20784e0e9a-workload-certs\") pod \"525e2d44-3744-40f8-9888-ed20784e0e9a\" (UID: \"525e2d44-3744-40f8-9888-ed20784e0e9a\") "
Jul 30 04:10:33.110985 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.110918    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-khtqg\" (UniqueName: \"kubernetes.io/projected/525e2d44-3744-40f8-9888-ed20784e0e9a-kube-api-access-khtqg\") pod \"525e2d44-3744-40f8-9888-ed20784e0e9a\" (UID: \"525e2d44-3744-40f8-9888-ed20784e0e9a\") "
Jul 30 04:10:33.110985 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.110968    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/525e2d44-3744-40f8-9888-ed20784e0e9a-credential-socket\") pod \"525e2d44-3744-40f8-9888-ed20784e0e9a\" (UID: \"525e2d44-3744-40f8-9888-ed20784e0e9a\") "
Jul 30 04:10:33.111308 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.110999    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/525e2d44-3744-40f8-9888-ed20784e0e9a-istio-token\") pod \"525e2d44-3744-40f8-9888-ed20784e0e9a\" (UID: \"525e2d44-3744-40f8-9888-ed20784e0e9a\") "
Jul 30 04:10:33.111406 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.111376    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/525e2d44-3744-40f8-9888-ed20784e0e9a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "525e2d44-3744-40f8-9888-ed20784e0e9a" (UID: "525e2d44-3744-40f8-9888-ed20784e0e9a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:10:33.111474 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.111390    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/525e2d44-3744-40f8-9888-ed20784e0e9a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "525e2d44-3744-40f8-9888-ed20784e0e9a" (UID: "525e2d44-3744-40f8-9888-ed20784e0e9a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:10:33.111568 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.111545    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/525e2d44-3744-40f8-9888-ed20784e0e9a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "525e2d44-3744-40f8-9888-ed20784e0e9a" (UID: "525e2d44-3744-40f8-9888-ed20784e0e9a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:10:33.111656 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.111612    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/525e2d44-3744-40f8-9888-ed20784e0e9a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "525e2d44-3744-40f8-9888-ed20784e0e9a" (UID: "525e2d44-3744-40f8-9888-ed20784e0e9a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:10:33.111761 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.111730    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/525e2d44-3744-40f8-9888-ed20784e0e9a-workload-socket\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:10:33.111761 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.111752    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/525e2d44-3744-40f8-9888-ed20784e0e9a-istiod-ca-cert\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:10:33.111891 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.111766    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/525e2d44-3744-40f8-9888-ed20784e0e9a-workload-certs\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:10:33.111891 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.111777    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/525e2d44-3744-40f8-9888-ed20784e0e9a-credential-socket\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:10:33.113671 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.113642    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/525e2d44-3744-40f8-9888-ed20784e0e9a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "525e2d44-3744-40f8-9888-ed20784e0e9a" (UID: "525e2d44-3744-40f8-9888-ed20784e0e9a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:10:33.113840 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.113810    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/525e2d44-3744-40f8-9888-ed20784e0e9a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "525e2d44-3744-40f8-9888-ed20784e0e9a" (UID: "525e2d44-3744-40f8-9888-ed20784e0e9a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 04:10:33.113840 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.113816    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/525e2d44-3744-40f8-9888-ed20784e0e9a-istio-token" (OuterVolumeSpecName: "istio-token") pod "525e2d44-3744-40f8-9888-ed20784e0e9a" (UID: "525e2d44-3744-40f8-9888-ed20784e0e9a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:10:33.113981 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.113842    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/525e2d44-3744-40f8-9888-ed20784e0e9a-kube-api-access-khtqg" (OuterVolumeSpecName: "kube-api-access-khtqg") pod "525e2d44-3744-40f8-9888-ed20784e0e9a" (UID: "525e2d44-3744-40f8-9888-ed20784e0e9a"). InnerVolumeSpecName "kube-api-access-khtqg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:10:33.212479 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.212448    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/525e2d44-3744-40f8-9888-ed20784e0e9a-istio-data\") pod \"525e2d44-3744-40f8-9888-ed20784e0e9a\" (UID: \"525e2d44-3744-40f8-9888-ed20784e0e9a\") "
Jul 30 04:10:33.212682 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.212671    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/525e2d44-3744-40f8-9888-ed20784e0e9a-istio-token\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:10:33.212735 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.212684    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/525e2d44-3744-40f8-9888-ed20784e0e9a-istio-podinfo\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:10:33.212735 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.212692    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/525e2d44-3744-40f8-9888-ed20784e0e9a-istio-envoy\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:10:33.212735 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.212701    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khtqg\" (UniqueName: \"kubernetes.io/projected/525e2d44-3744-40f8-9888-ed20784e0e9a-kube-api-access-khtqg\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:10:33.213068 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.213047    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/525e2d44-3744-40f8-9888-ed20784e0e9a-istio-data" (OuterVolumeSpecName: "istio-data") pod "525e2d44-3744-40f8-9888-ed20784e0e9a" (UID: "525e2d44-3744-40f8-9888-ed20784e0e9a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:10:33.314281 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.314245    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/525e2d44-3744-40f8-9888-ed20784e0e9a-istio-data\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:10:33.531206 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.531168    2576 generic.go:356] "Generic (PLEG): container finished" podID="525e2d44-3744-40f8-9888-ed20784e0e9a" containerID="09d9f03f5a8cbf8c6e40dcd4c63b7b6dc9c6688faf2a45b15a91c3e4945c373f" exitCode=0
Jul 30 04:10:33.531379 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.531219    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"
Jul 30 04:10:33.531379 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.531219    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8" event={"ID":"525e2d44-3744-40f8-9888-ed20784e0e9a","Type":"ContainerDied","Data":"09d9f03f5a8cbf8c6e40dcd4c63b7b6dc9c6688faf2a45b15a91c3e4945c373f"}
Jul 30 04:10:33.531379 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.531293    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8" event={"ID":"525e2d44-3744-40f8-9888-ed20784e0e9a","Type":"ContainerDied","Data":"55a152738d14149cd596065571c1fa043f316115a210cee28139efaacf02312e"}
Jul 30 04:10:33.531379 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.531327    2576 scope.go:117] "RemoveContainer" containerID="09d9f03f5a8cbf8c6e40dcd4c63b7b6dc9c6688faf2a45b15a91c3e4945c373f"
Jul 30 04:10:33.541035 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.541019    2576 scope.go:117] "RemoveContainer" containerID="926c13494e0b710a7b3120cbf3dc6ab7e3ab88ccf127c46e438236cf8a43b072"
Jul 30 04:10:33.548797 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.548778    2576 scope.go:117] "RemoveContainer" containerID="09d9f03f5a8cbf8c6e40dcd4c63b7b6dc9c6688faf2a45b15a91c3e4945c373f"
Jul 30 04:10:33.549043 ip-10-0-143-117 kubenswrapper[2576]: E0730 04:10:33.549028    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09d9f03f5a8cbf8c6e40dcd4c63b7b6dc9c6688faf2a45b15a91c3e4945c373f\": container with ID starting with 09d9f03f5a8cbf8c6e40dcd4c63b7b6dc9c6688faf2a45b15a91c3e4945c373f not found: ID does not exist" containerID="09d9f03f5a8cbf8c6e40dcd4c63b7b6dc9c6688faf2a45b15a91c3e4945c373f"
Jul 30 04:10:33.549087 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.549052    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09d9f03f5a8cbf8c6e40dcd4c63b7b6dc9c6688faf2a45b15a91c3e4945c373f"} err="failed to get container status \"09d9f03f5a8cbf8c6e40dcd4c63b7b6dc9c6688faf2a45b15a91c3e4945c373f\": rpc error: code = NotFound desc = could not find container \"09d9f03f5a8cbf8c6e40dcd4c63b7b6dc9c6688faf2a45b15a91c3e4945c373f\": container with ID starting with 09d9f03f5a8cbf8c6e40dcd4c63b7b6dc9c6688faf2a45b15a91c3e4945c373f not found: ID does not exist"
Jul 30 04:10:33.549087 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.549068    2576 scope.go:117] "RemoveContainer" containerID="926c13494e0b710a7b3120cbf3dc6ab7e3ab88ccf127c46e438236cf8a43b072"
Jul 30 04:10:33.549262 ip-10-0-143-117 kubenswrapper[2576]: E0730 04:10:33.549247    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"926c13494e0b710a7b3120cbf3dc6ab7e3ab88ccf127c46e438236cf8a43b072\": container with ID starting with 926c13494e0b710a7b3120cbf3dc6ab7e3ab88ccf127c46e438236cf8a43b072 not found: ID does not exist" containerID="926c13494e0b710a7b3120cbf3dc6ab7e3ab88ccf127c46e438236cf8a43b072"
Jul 30 04:10:33.549300 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.549265    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"926c13494e0b710a7b3120cbf3dc6ab7e3ab88ccf127c46e438236cf8a43b072"} err="failed to get container status \"926c13494e0b710a7b3120cbf3dc6ab7e3ab88ccf127c46e438236cf8a43b072\": rpc error: code = NotFound desc = could not find container \"926c13494e0b710a7b3120cbf3dc6ab7e3ab88ccf127c46e438236cf8a43b072\": container with ID starting with 926c13494e0b710a7b3120cbf3dc6ab7e3ab88ccf127c46e438236cf8a43b072 not found: ID does not exist"
Jul 30 04:10:33.554818 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.554796    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"]
Jul 30 04:10:33.557523 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:33.557504    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-3e3f5a98-openshift-default-6787584964-b8wt8"]
Jul 30 04:10:35.051583 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:35.051546    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="525e2d44-3744-40f8-9888-ed20784e0e9a" path="/var/lib/kubelet/pods/525e2d44-3744-40f8-9888-ed20784e0e9a/volumes"
Jul 30 04:10:50.553690 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:50.553594    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:10:54.324254 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:54.324217    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:10:54.813373 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:54.813338    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:10:54.913471 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:54.913427    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:10:56.426777 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:56.426741    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:10:56.527205 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:56.527170    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:10:56.630769 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:56.630737    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:10:56.728468 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:56.728364    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:10:56.828357 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:56.828319    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:10:57.228783 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:10:57.228741    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:11:23.127639 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:11:23.127601    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:11:29.393608 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:11:29.393562    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:11:29.501144 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:11:29.501098    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:11:29.608723 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:11:29.608684    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:11:45.449036 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:11:45.448997    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:11:51.301272 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:11:51.301240    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:11:51.402335 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:11:51.402299    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:11:51.501633 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:11:51.501596    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:12:02.025272 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:02.025236    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:12:13.005613 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:13.005570    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:12:19.137259 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:19.137169    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:12:20.550949 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:20.550890    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:12:21.804447 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:21.804414    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:12:22.521608 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:22.521572    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:12:27.324660 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:27.324618    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:12:27.380255 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:27.380221    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:12:27.963018 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:27.962975    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:12:32.723716 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:32.723677    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:12:34.200979 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.200945    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:12:34.300942 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.300884    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:12:34.414569 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.414527    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:12:34.696714 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.696686    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"]
Jul 30 04:12:34.697118 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.697102    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="525e2d44-3744-40f8-9888-ed20784e0e9a" containerName="istio-proxy"
Jul 30 04:12:34.697219 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.697121    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="525e2d44-3744-40f8-9888-ed20784e0e9a" containerName="istio-proxy"
Jul 30 04:12:34.697219 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.697148    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="525e2d44-3744-40f8-9888-ed20784e0e9a" containerName="istio-proxy"
Jul 30 04:12:34.697219 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.697156    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="525e2d44-3744-40f8-9888-ed20784e0e9a" containerName="istio-proxy"
Jul 30 04:12:34.697219 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.697167    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01e6bce1-954c-41f4-a9ed-e8c15f75287a" containerName="revoke-keys"
Jul 30 04:12:34.697219 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.697176    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="01e6bce1-954c-41f4-a9ed-e8c15f75287a" containerName="revoke-keys"
Jul 30 04:12:34.697477 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.697283    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9712893e-8e6d-4654-b16c-1190a2b2489f" containerName="istio-proxy"
Jul 30 04:12:34.697477 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.697296    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="525e2d44-3744-40f8-9888-ed20784e0e9a" containerName="istio-proxy"
Jul 30 04:12:34.697477 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.697307    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="01e6bce1-954c-41f4-a9ed-e8c15f75287a" containerName="revoke-keys"
Jul 30 04:12:34.699971 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.699952    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:12:34.702521 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.702496    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-246dcb85-openshift-default-dockercfg-zpwvs\""
Jul 30 04:12:34.724192 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.724109    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"]
Jul 30 04:12:34.725075 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.725038    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:12:34.805279 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.805246    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a20d00c-ae7e-42eb-8098-f0ef094975f5-istio-data\") pod \"e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7\" (UID: \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\") " pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:12:34.805279 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.805289    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a20d00c-ae7e-42eb-8098-f0ef094975f5-istio-envoy\") pod \"e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7\" (UID: \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\") " pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:12:34.805504 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.805332    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a20d00c-ae7e-42eb-8098-f0ef094975f5-istio-podinfo\") pod \"e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7\" (UID: \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\") " pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:12:34.805504 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.805465    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a20d00c-ae7e-42eb-8098-f0ef094975f5-workload-socket\") pod \"e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7\" (UID: \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\") " pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:12:34.805580 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.805515    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a20d00c-ae7e-42eb-8098-f0ef094975f5-istio-token\") pod \"e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7\" (UID: \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\") " pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:12:34.805580 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.805555    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a20d00c-ae7e-42eb-8098-f0ef094975f5-credential-socket\") pod \"e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7\" (UID: \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\") " pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:12:34.805648 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.805588    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a20d00c-ae7e-42eb-8098-f0ef094975f5-istiod-ca-cert\") pod \"e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7\" (UID: \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\") " pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:12:34.805648 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.805621    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fzj9\" (UniqueName: \"kubernetes.io/projected/7a20d00c-ae7e-42eb-8098-f0ef094975f5-kube-api-access-8fzj9\") pod \"e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7\" (UID: \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\") " pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:12:34.805750 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.805649    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a20d00c-ae7e-42eb-8098-f0ef094975f5-workload-certs\") pod \"e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7\" (UID: \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\") " pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:12:34.873495 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.873441    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:12:34.906597 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.906550    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a20d00c-ae7e-42eb-8098-f0ef094975f5-workload-socket\") pod \"e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7\" (UID: \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\") " pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:12:34.906809 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.906610    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a20d00c-ae7e-42eb-8098-f0ef094975f5-istio-token\") pod \"e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7\" (UID: \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\") " pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:12:34.906809 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.906642    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a20d00c-ae7e-42eb-8098-f0ef094975f5-credential-socket\") pod \"e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7\" (UID: \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\") " pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:12:34.906809 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.906671    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a20d00c-ae7e-42eb-8098-f0ef094975f5-istiod-ca-cert\") pod \"e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7\" (UID: \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\") " pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:12:34.906809 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.906699    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fzj9\" (UniqueName: \"kubernetes.io/projected/7a20d00c-ae7e-42eb-8098-f0ef094975f5-kube-api-access-8fzj9\") pod \"e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7\" (UID: \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\") " pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:12:34.906809 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.906728    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a20d00c-ae7e-42eb-8098-f0ef094975f5-workload-certs\") pod \"e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7\" (UID: \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\") " pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:12:34.906809 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.906803    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a20d00c-ae7e-42eb-8098-f0ef094975f5-istio-data\") pod \"e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7\" (UID: \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\") " pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:12:34.907144 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.906848    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a20d00c-ae7e-42eb-8098-f0ef094975f5-istio-envoy\") pod \"e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7\" (UID: \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\") " pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:12:34.907144 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.906910    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a20d00c-ae7e-42eb-8098-f0ef094975f5-istio-podinfo\") pod \"e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7\" (UID: \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\") " pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:12:34.907832 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.907797    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a20d00c-ae7e-42eb-8098-f0ef094975f5-workload-socket\") pod \"e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7\" (UID: \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\") " pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:12:34.908333 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.908288    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a20d00c-ae7e-42eb-8098-f0ef094975f5-credential-socket\") pod \"e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7\" (UID: \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\") " pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:12:34.908333 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.908318    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a20d00c-ae7e-42eb-8098-f0ef094975f5-istio-data\") pod \"e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7\" (UID: \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\") " pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:12:34.908790 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.908735    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a20d00c-ae7e-42eb-8098-f0ef094975f5-istiod-ca-cert\") pod \"e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7\" (UID: \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\") " pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:12:34.908899 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.908849    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a20d00c-ae7e-42eb-8098-f0ef094975f5-workload-certs\") pod \"e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7\" (UID: \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\") " pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:12:34.910522 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.910454    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a20d00c-ae7e-42eb-8098-f0ef094975f5-istio-podinfo\") pod \"e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7\" (UID: \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\") " pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:12:34.910743 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.910722    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a20d00c-ae7e-42eb-8098-f0ef094975f5-istio-envoy\") pod \"e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7\" (UID: \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\") " pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:12:34.918332 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.917959    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fzj9\" (UniqueName: \"kubernetes.io/projected/7a20d00c-ae7e-42eb-8098-f0ef094975f5-kube-api-access-8fzj9\") pod \"e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7\" (UID: \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\") " pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:12:34.923187 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:34.923146    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a20d00c-ae7e-42eb-8098-f0ef094975f5-istio-token\") pod \"e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7\" (UID: \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\") " pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:12:35.011063 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:35.010969    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:12:35.153599 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:35.153573    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"]
Jul 30 04:12:35.155348 ip-10-0-143-117 kubenswrapper[2576]: W0730 04:12:35.155311    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a20d00c_ae7e_42eb_8098_f0ef094975f5.slice/crio-0069e2ea0873fe4627cf05b03c800843da56f952cdb31213b3807d2225d8e16d WatchSource:0}: Error finding container 0069e2ea0873fe4627cf05b03c800843da56f952cdb31213b3807d2225d8e16d: Status 404 returned error can't find the container with id 0069e2ea0873fe4627cf05b03c800843da56f952cdb31213b3807d2225d8e16d
Jul 30 04:12:35.157490 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:35.157442    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:12:35.157576 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:35.157513    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:12:35.157576 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:35.157543    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:12:35.181766 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:35.181740    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7" event={"ID":"7a20d00c-ae7e-42eb-8098-f0ef094975f5","Type":"ContainerStarted","Data":"0069e2ea0873fe4627cf05b03c800843da56f952cdb31213b3807d2225d8e16d"}
Jul 30 04:12:36.190181 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:36.190142    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7" event={"ID":"7a20d00c-ae7e-42eb-8098-f0ef094975f5","Type":"ContainerStarted","Data":"b10bf98a9949f135995aebe7ee7843be12031cdb1eaac2d1ec071fda1081df28"}
Jul 30 04:12:36.211458 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:36.211407    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7" podStartSLOduration=2.211391445 podStartE2EDuration="2.211391445s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:12:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:12:36.209682686 +0000 UTC m=+2261.827914928" watchObservedRunningTime="2026-07-30 04:12:36.211391445 +0000 UTC m=+2261.829623686"
Jul 30 04:12:37.011180 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:37.011134    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:12:37.015813 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:37.015785    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:12:37.195580 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:37.195545    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:12:37.196582 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:37.196563    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:12:38.108918 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:38.108883    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:12:38.210528 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:38.210494    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:12:39.237591 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:39.237555    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:12:39.400993 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:39.400959    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:12:39.787122 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:39.787067    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:12:39.894066 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:39.894026    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:12:39.904209 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:39.904181    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-246dcb85-78d5fdcfbd-4mqpc"]
Jul 30 04:12:39.904638 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:39.904625    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="525e2d44-3744-40f8-9888-ed20784e0e9a" containerName="istio-proxy"
Jul 30 04:12:39.907570 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:39.907553    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-246dcb85-78d5fdcfbd-4mqpc"
Jul 30 04:12:39.910320 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:39.910300    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-246dcb85-dockercfg-64drx\""
Jul 30 04:12:39.910441 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:39.910343    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-246dcb85\""
Jul 30 04:12:39.917548 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:39.917526    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-246dcb85-78d5fdcfbd-4mqpc"]
Jul 30 04:12:40.055978 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:40.055868    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qx4k2\" (UniqueName: \"kubernetes.io/projected/8fa93881-df00-47d1-b06c-c6cd21047334-kube-api-access-qx4k2\") pod \"payload-pre-processing-e2e-mt-246dcb85-78d5fdcfbd-4mqpc\" (UID: \"8fa93881-df00-47d1-b06c-c6cd21047334\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-246dcb85-78d5fdcfbd-4mqpc"
Jul 30 04:12:40.055978 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:40.055968    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fa93881-df00-47d1-b06c-c6cd21047334-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-246dcb85-78d5fdcfbd-4mqpc\" (UID: \"8fa93881-df00-47d1-b06c-c6cd21047334\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-246dcb85-78d5fdcfbd-4mqpc"
Jul 30 04:12:40.157496 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:40.157465    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qx4k2\" (UniqueName: \"kubernetes.io/projected/8fa93881-df00-47d1-b06c-c6cd21047334-kube-api-access-qx4k2\") pod \"payload-pre-processing-e2e-mt-246dcb85-78d5fdcfbd-4mqpc\" (UID: \"8fa93881-df00-47d1-b06c-c6cd21047334\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-246dcb85-78d5fdcfbd-4mqpc"
Jul 30 04:12:40.157645 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:40.157525    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fa93881-df00-47d1-b06c-c6cd21047334-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-246dcb85-78d5fdcfbd-4mqpc\" (UID: \"8fa93881-df00-47d1-b06c-c6cd21047334\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-246dcb85-78d5fdcfbd-4mqpc"
Jul 30 04:12:40.158239 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:40.158222    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fa93881-df00-47d1-b06c-c6cd21047334-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-246dcb85-78d5fdcfbd-4mqpc\" (UID: \"8fa93881-df00-47d1-b06c-c6cd21047334\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-246dcb85-78d5fdcfbd-4mqpc"
Jul 30 04:12:40.166494 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:40.166474    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qx4k2\" (UniqueName: \"kubernetes.io/projected/8fa93881-df00-47d1-b06c-c6cd21047334-kube-api-access-qx4k2\") pod \"payload-pre-processing-e2e-mt-246dcb85-78d5fdcfbd-4mqpc\" (UID: \"8fa93881-df00-47d1-b06c-c6cd21047334\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-246dcb85-78d5fdcfbd-4mqpc"
Jul 30 04:12:40.219441 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:40.219415    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-246dcb85-78d5fdcfbd-4mqpc"
Jul 30 04:12:40.347658 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:40.347631    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-246dcb85-78d5fdcfbd-4mqpc"]
Jul 30 04:12:40.349109 ip-10-0-143-117 kubenswrapper[2576]: W0730 04:12:40.349078    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fa93881_df00_47d1_b06c_c6cd21047334.slice/crio-7408bca9bb9e94c2a7a43b5ac07e7566afb7188639383df7df800e169fc9e9f8 WatchSource:0}: Error finding container 7408bca9bb9e94c2a7a43b5ac07e7566afb7188639383df7df800e169fc9e9f8: Status 404 returned error can't find the container with id 7408bca9bb9e94c2a7a43b5ac07e7566afb7188639383df7df800e169fc9e9f8
Jul 30 04:12:41.222507 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:41.222466    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-246dcb85-78d5fdcfbd-4mqpc" event={"ID":"8fa93881-df00-47d1-b06c-c6cd21047334","Type":"ContainerStarted","Data":"39ebdaa8e137eb942ef77496851bd20f0d9f9ee49351889e33f7bc79668095d8"}
Jul 30 04:12:41.222716 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:41.222526    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-246dcb85-78d5fdcfbd-4mqpc" event={"ID":"8fa93881-df00-47d1-b06c-c6cd21047334","Type":"ContainerStarted","Data":"7408bca9bb9e94c2a7a43b5ac07e7566afb7188639383df7df800e169fc9e9f8"}
Jul 30 04:12:41.222716 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:41.222553    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-246dcb85-78d5fdcfbd-4mqpc"
Jul 30 04:12:41.245263 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:41.239758    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-246dcb85-78d5fdcfbd-4mqpc" podStartSLOduration=2.239732841 podStartE2EDuration="2.239732841s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:12:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:12:41.236284576 +0000 UTC m=+2266.854516819" watchObservedRunningTime="2026-07-30 04:12:41.239732841 +0000 UTC m=+2266.857965085"
Jul 30 04:12:45.935723 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:45.935682    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:12:52.230029 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:52.229995    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-246dcb85-78d5fdcfbd-4mqpc"
Jul 30 04:12:53.237138 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:53.237087    2576 patch_prober.go:28] interesting pod/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 30 04:12:53.237571 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:53.237168    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7" podUID="7a20d00c-ae7e-42eb-8098-f0ef094975f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 30 04:12:54.297893 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:54.297856    2576 generic.go:356] "Generic (PLEG): container finished" podID="7a20d00c-ae7e-42eb-8098-f0ef094975f5" containerID="b10bf98a9949f135995aebe7ee7843be12031cdb1eaac2d1ec071fda1081df28" exitCode=0
Jul 30 04:12:54.298317 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:54.297949    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7" event={"ID":"7a20d00c-ae7e-42eb-8098-f0ef094975f5","Type":"ContainerDied","Data":"b10bf98a9949f135995aebe7ee7843be12031cdb1eaac2d1ec071fda1081df28"}
Jul 30 04:12:54.298373 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:54.298361    2576 scope.go:117] "RemoveContainer" containerID="b10bf98a9949f135995aebe7ee7843be12031cdb1eaac2d1ec071fda1081df28"
Jul 30 04:12:54.299421 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:54.299397    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:12:54.299478 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:54.299463    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:12:54.299514 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:54.299505    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:12:55.305129 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:55.305093    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7" event={"ID":"7a20d00c-ae7e-42eb-8098-f0ef094975f5","Type":"ContainerStarted","Data":"88ac6e13d027d2c5aa0b53df40684cf7c1d4980f4e3cdec806572d7baf6ba021"}
Jul 30 04:12:56.011319 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:56.011277    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:12:56.011589 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:56.011565    2576 patch_prober.go:28] interesting pod/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 30 04:12:56.011680 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:56.011624    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7" podUID="7a20d00c-ae7e-42eb-8098-f0ef094975f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 30 04:12:57.012078 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:57.012042    2576 patch_prober.go:28] interesting pod/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 30 04:12:57.012470 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:57.012099    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7" podUID="7a20d00c-ae7e-42eb-8098-f0ef094975f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 30 04:12:58.011734 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:58.011697    2576 patch_prober.go:28] interesting pod/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 30 04:12:58.011912 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:58.011752    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7" podUID="7a20d00c-ae7e-42eb-8098-f0ef094975f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 30 04:12:59.012176 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:59.012139    2576 patch_prober.go:28] interesting pod/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 30 04:12:59.012593 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:12:59.012198    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7" podUID="7a20d00c-ae7e-42eb-8098-f0ef094975f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 30 04:13:00.014846 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:00.014813    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:13:00.015239 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:00.015161    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:13:00.015820 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:00.015803    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:13:03.041017 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:03.040979    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:13:03.683190 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:03.683154    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:13:03.781840 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:03.781792    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:13:04.092906 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:04.092856    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:13:09.805000 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:09.804962    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:13:09.865977 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:09.865944    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:13:10.150565 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:10.150483    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-246dcb85-sbdv4"]
Jul 30 04:13:10.153932 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:10.153905    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-246dcb85-sbdv4"
Jul 30 04:13:10.156683 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:10.156662    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5cg4s\""
Jul 30 04:13:10.161911 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:10.161438    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-246dcb85-sbdv4"]
Jul 30 04:13:10.231416 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:10.231380    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1709862-6b43-4f9b-9277-6d0c0dcc725d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-246dcb85-sbdv4\" (UID: \"e1709862-6b43-4f9b-9277-6d0c0dcc725d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-246dcb85-sbdv4"
Jul 30 04:13:10.332094 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:10.332061    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1709862-6b43-4f9b-9277-6d0c0dcc725d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-246dcb85-sbdv4\" (UID: \"e1709862-6b43-4f9b-9277-6d0c0dcc725d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-246dcb85-sbdv4"
Jul 30 04:13:10.332671 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:10.332650    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1709862-6b43-4f9b-9277-6d0c0dcc725d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-246dcb85-sbdv4\" (UID: \"e1709862-6b43-4f9b-9277-6d0c0dcc725d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-246dcb85-sbdv4"
Jul 30 04:13:10.467214 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:10.467134    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-246dcb85-sbdv4"
Jul 30 04:13:10.598034 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:10.598006    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-246dcb85-sbdv4"]
Jul 30 04:13:10.600038 ip-10-0-143-117 kubenswrapper[2576]: W0730 04:13:10.600015    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1709862_6b43_4f9b_9277_6d0c0dcc725d.slice/crio-4d495dc8a75f8e28185865cfcf5e9a2fce574bb4f1d0f9a07fc9f5c4f3330efe WatchSource:0}: Error finding container 4d495dc8a75f8e28185865cfcf5e9a2fce574bb4f1d0f9a07fc9f5c4f3330efe: Status 404 returned error can't find the container with id 4d495dc8a75f8e28185865cfcf5e9a2fce574bb4f1d0f9a07fc9f5c4f3330efe
Jul 30 04:13:11.394848 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:11.394811    2576 generic.go:356] "Generic (PLEG): container finished" podID="e1709862-6b43-4f9b-9277-6d0c0dcc725d" containerID="1acda411bc06270b757a875ef7bd808893dbd817b96d05adfb0c14bff8cef9fa" exitCode=0
Jul 30 04:13:11.395304 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:11.394912    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-246dcb85-sbdv4" event={"ID":"e1709862-6b43-4f9b-9277-6d0c0dcc725d","Type":"ContainerDied","Data":"1acda411bc06270b757a875ef7bd808893dbd817b96d05adfb0c14bff8cef9fa"}
Jul 30 04:13:11.395304 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:11.394991    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-246dcb85-sbdv4" event={"ID":"e1709862-6b43-4f9b-9277-6d0c0dcc725d","Type":"ContainerStarted","Data":"4d495dc8a75f8e28185865cfcf5e9a2fce574bb4f1d0f9a07fc9f5c4f3330efe"}
Jul 30 04:13:12.529324 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:12.529299    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-246dcb85-sbdv4"
Jul 30 04:13:12.651508 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:12.651415    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1709862-6b43-4f9b-9277-6d0c0dcc725d-maas-api-service-ca\") pod \"e1709862-6b43-4f9b-9277-6d0c0dcc725d\" (UID: \"e1709862-6b43-4f9b-9277-6d0c0dcc725d\") "
Jul 30 04:13:12.651781 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:12.651757    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1709862-6b43-4f9b-9277-6d0c0dcc725d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e1709862-6b43-4f9b-9277-6d0c0dcc725d" (UID: "e1709862-6b43-4f9b-9277-6d0c0dcc725d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:13:12.752066 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:12.752040    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1709862-6b43-4f9b-9277-6d0c0dcc725d-maas-api-service-ca\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:13:13.406796 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:13.406713    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-246dcb85-sbdv4"
Jul 30 04:13:13.406796 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:13.406752    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-246dcb85-sbdv4" event={"ID":"e1709862-6b43-4f9b-9277-6d0c0dcc725d","Type":"ContainerDied","Data":"4d495dc8a75f8e28185865cfcf5e9a2fce574bb4f1d0f9a07fc9f5c4f3330efe"}
Jul 30 04:13:13.406996 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:13.406802    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4d495dc8a75f8e28185865cfcf5e9a2fce574bb4f1d0f9a07fc9f5c4f3330efe"
Jul 30 04:13:20.669499 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:20.669457    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:13:20.699892 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:20.699861    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-246dcb85-78d5fdcfbd-4mqpc"]
Jul 30 04:13:20.700144 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:20.700115    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-246dcb85-78d5fdcfbd-4mqpc" podUID="8fa93881-df00-47d1-b06c-c6cd21047334" containerName="payload-pre-processing" containerID="cri-o://39ebdaa8e137eb942ef77496851bd20f0d9f9ee49351889e33f7bc79668095d8" gracePeriod=30
Jul 30 04:13:20.768981 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:20.768918    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:13:20.824963 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:20.824934    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-246dcb85-sbdv4"]
Jul 30 04:13:20.828914 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:20.828891    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-246dcb85-sbdv4"]
Jul 30 04:13:20.830402 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:20.830360    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:13:20.912047 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:20.912013    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:13:20.961624 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:20.961597    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-246dcb85-78d5fdcfbd-4mqpc"
Jul 30 04:13:21.023616 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:21.023583    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fa93881-df00-47d1-b06c-c6cd21047334-plugins-config-volume\") pod \"8fa93881-df00-47d1-b06c-c6cd21047334\" (UID: \"8fa93881-df00-47d1-b06c-c6cd21047334\") "
Jul 30 04:13:21.023779 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:21.023679    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qx4k2\" (UniqueName: \"kubernetes.io/projected/8fa93881-df00-47d1-b06c-c6cd21047334-kube-api-access-qx4k2\") pod \"8fa93881-df00-47d1-b06c-c6cd21047334\" (UID: \"8fa93881-df00-47d1-b06c-c6cd21047334\") "
Jul 30 04:13:21.024087 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:21.024053    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fa93881-df00-47d1-b06c-c6cd21047334-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8fa93881-df00-47d1-b06c-c6cd21047334" (UID: "8fa93881-df00-47d1-b06c-c6cd21047334"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:13:21.025716 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:21.025694    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fa93881-df00-47d1-b06c-c6cd21047334-kube-api-access-qx4k2" (OuterVolumeSpecName: "kube-api-access-qx4k2") pod "8fa93881-df00-47d1-b06c-c6cd21047334" (UID: "8fa93881-df00-47d1-b06c-c6cd21047334"). InnerVolumeSpecName "kube-api-access-qx4k2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:13:21.051394 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:21.051361    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1709862-6b43-4f9b-9277-6d0c0dcc725d" path="/var/lib/kubelet/pods/e1709862-6b43-4f9b-9277-6d0c0dcc725d/volumes"
Jul 30 04:13:21.125122 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:21.125092    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qx4k2\" (UniqueName: \"kubernetes.io/projected/8fa93881-df00-47d1-b06c-c6cd21047334-kube-api-access-qx4k2\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:13:21.125122 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:21.125118    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fa93881-df00-47d1-b06c-c6cd21047334-plugins-config-volume\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:13:21.159949 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:21.159886    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:13:21.450329 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:21.450292    2576 generic.go:356] "Generic (PLEG): container finished" podID="8fa93881-df00-47d1-b06c-c6cd21047334" containerID="39ebdaa8e137eb942ef77496851bd20f0d9f9ee49351889e33f7bc79668095d8" exitCode=0
Jul 30 04:13:21.450512 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:21.450337    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-246dcb85-78d5fdcfbd-4mqpc"
Jul 30 04:13:21.450512 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:21.450377    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-246dcb85-78d5fdcfbd-4mqpc" event={"ID":"8fa93881-df00-47d1-b06c-c6cd21047334","Type":"ContainerDied","Data":"39ebdaa8e137eb942ef77496851bd20f0d9f9ee49351889e33f7bc79668095d8"}
Jul 30 04:13:21.450512 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:21.450426    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-246dcb85-78d5fdcfbd-4mqpc" event={"ID":"8fa93881-df00-47d1-b06c-c6cd21047334","Type":"ContainerDied","Data":"7408bca9bb9e94c2a7a43b5ac07e7566afb7188639383df7df800e169fc9e9f8"}
Jul 30 04:13:21.450512 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:21.450449    2576 scope.go:117] "RemoveContainer" containerID="39ebdaa8e137eb942ef77496851bd20f0d9f9ee49351889e33f7bc79668095d8"
Jul 30 04:13:21.459589 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:21.459560    2576 scope.go:117] "RemoveContainer" containerID="39ebdaa8e137eb942ef77496851bd20f0d9f9ee49351889e33f7bc79668095d8"
Jul 30 04:13:21.459835 ip-10-0-143-117 kubenswrapper[2576]: E0730 04:13:21.459817    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39ebdaa8e137eb942ef77496851bd20f0d9f9ee49351889e33f7bc79668095d8\": container with ID starting with 39ebdaa8e137eb942ef77496851bd20f0d9f9ee49351889e33f7bc79668095d8 not found: ID does not exist" containerID="39ebdaa8e137eb942ef77496851bd20f0d9f9ee49351889e33f7bc79668095d8"
Jul 30 04:13:21.459882 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:21.459844    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39ebdaa8e137eb942ef77496851bd20f0d9f9ee49351889e33f7bc79668095d8"} err="failed to get container status \"39ebdaa8e137eb942ef77496851bd20f0d9f9ee49351889e33f7bc79668095d8\": rpc error: code = NotFound desc = could not find container \"39ebdaa8e137eb942ef77496851bd20f0d9f9ee49351889e33f7bc79668095d8\": container with ID starting with 39ebdaa8e137eb942ef77496851bd20f0d9f9ee49351889e33f7bc79668095d8 not found: ID does not exist"
Jul 30 04:13:21.467482 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:21.467451    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-246dcb85-78d5fdcfbd-4mqpc"]
Jul 30 04:13:21.469551 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:21.469534    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-246dcb85-78d5fdcfbd-4mqpc"]
Jul 30 04:13:23.051352 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:23.051319    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fa93881-df00-47d1-b06c-c6cd21047334" path="/var/lib/kubelet/pods/8fa93881-df00-47d1-b06c-c6cd21047334/volumes"
Jul 30 04:13:23.721740 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:23.721705    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:13:27.036078 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:27.036041    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:13:27.106959 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:27.106904    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:13:27.291746 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:27.291662    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"]
Jul 30 04:13:27.291917 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:27.291885    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7" podUID="7a20d00c-ae7e-42eb-8098-f0ef094975f5" containerName="istio-proxy" containerID="cri-o://88ac6e13d027d2c5aa0b53df40684cf7c1d4980f4e3cdec806572d7baf6ba021" gracePeriod=30
Jul 30 04:13:27.315908 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:27.315879    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:13:27.537969 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:27.537916    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:13:27.693393 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:27.693310    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:13:27.782217 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:27.782186    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:13:27.885326 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:27.885294    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:13:30.017563 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:30.017524    2576 patch_prober.go:28] interesting pod/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 30 04:13:30.017967 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:30.017592    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7" podUID="7a20d00c-ae7e-42eb-8098-f0ef094975f5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 04:13:32.517149 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:32.517106    2576 generic.go:356] "Generic (PLEG): container finished" podID="7a20d00c-ae7e-42eb-8098-f0ef094975f5" containerID="88ac6e13d027d2c5aa0b53df40684cf7c1d4980f4e3cdec806572d7baf6ba021" exitCode=0
Jul 30 04:13:32.517628 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:32.517178    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7" event={"ID":"7a20d00c-ae7e-42eb-8098-f0ef094975f5","Type":"ContainerDied","Data":"88ac6e13d027d2c5aa0b53df40684cf7c1d4980f4e3cdec806572d7baf6ba021"}
Jul 30 04:13:32.517628 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:32.517241    2576 scope.go:117] "RemoveContainer" containerID="b10bf98a9949f135995aebe7ee7843be12031cdb1eaac2d1ec071fda1081df28"
Jul 30 04:13:32.550500 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:32.550477    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:13:32.637213 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:32.637120    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a20d00c-ae7e-42eb-8098-f0ef094975f5-istio-envoy\") pod \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\" (UID: \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\") "
Jul 30 04:13:32.637213 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:32.637157    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8fzj9\" (UniqueName: \"kubernetes.io/projected/7a20d00c-ae7e-42eb-8098-f0ef094975f5-kube-api-access-8fzj9\") pod \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\" (UID: \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\") "
Jul 30 04:13:32.637463 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:32.637237    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a20d00c-ae7e-42eb-8098-f0ef094975f5-istio-data\") pod \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\" (UID: \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\") "
Jul 30 04:13:32.637463 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:32.637276    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a20d00c-ae7e-42eb-8098-f0ef094975f5-istio-podinfo\") pod \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\" (UID: \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\") "
Jul 30 04:13:32.637463 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:32.637304    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a20d00c-ae7e-42eb-8098-f0ef094975f5-istiod-ca-cert\") pod \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\" (UID: \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\") "
Jul 30 04:13:32.637463 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:32.637385    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a20d00c-ae7e-42eb-8098-f0ef094975f5-workload-socket\") pod \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\" (UID: \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\") "
Jul 30 04:13:32.637463 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:32.637410    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a20d00c-ae7e-42eb-8098-f0ef094975f5-credential-socket\") pod \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\" (UID: \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\") "
Jul 30 04:13:32.637463 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:32.637433    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a20d00c-ae7e-42eb-8098-f0ef094975f5-workload-certs\") pod \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\" (UID: \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\") "
Jul 30 04:13:32.637786 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:32.637471    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a20d00c-ae7e-42eb-8098-f0ef094975f5-istio-token\") pod \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\" (UID: \"7a20d00c-ae7e-42eb-8098-f0ef094975f5\") "
Jul 30 04:13:32.637786 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:32.637725    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a20d00c-ae7e-42eb-8098-f0ef094975f5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7a20d00c-ae7e-42eb-8098-f0ef094975f5" (UID: "7a20d00c-ae7e-42eb-8098-f0ef094975f5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:13:32.637786 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:32.637736    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a20d00c-ae7e-42eb-8098-f0ef094975f5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7a20d00c-ae7e-42eb-8098-f0ef094975f5" (UID: "7a20d00c-ae7e-42eb-8098-f0ef094975f5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:13:32.637957 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:32.637826    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a20d00c-ae7e-42eb-8098-f0ef094975f5-credential-socket\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:13:32.637957 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:32.637847    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a20d00c-ae7e-42eb-8098-f0ef094975f5-workload-certs\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:13:32.638081 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:32.638058    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a20d00c-ae7e-42eb-8098-f0ef094975f5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7a20d00c-ae7e-42eb-8098-f0ef094975f5" (UID: "7a20d00c-ae7e-42eb-8098-f0ef094975f5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:13:32.638202 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:32.638180    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a20d00c-ae7e-42eb-8098-f0ef094975f5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7a20d00c-ae7e-42eb-8098-f0ef094975f5" (UID: "7a20d00c-ae7e-42eb-8098-f0ef094975f5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:13:32.638316 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:32.638190    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a20d00c-ae7e-42eb-8098-f0ef094975f5-istio-data" (OuterVolumeSpecName: "istio-data") pod "7a20d00c-ae7e-42eb-8098-f0ef094975f5" (UID: "7a20d00c-ae7e-42eb-8098-f0ef094975f5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:13:32.639891 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:32.639859    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a20d00c-ae7e-42eb-8098-f0ef094975f5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7a20d00c-ae7e-42eb-8098-f0ef094975f5" (UID: "7a20d00c-ae7e-42eb-8098-f0ef094975f5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:13:32.639997 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:32.639889    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7a20d00c-ae7e-42eb-8098-f0ef094975f5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7a20d00c-ae7e-42eb-8098-f0ef094975f5" (UID: "7a20d00c-ae7e-42eb-8098-f0ef094975f5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 04:13:32.639997 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:32.639904    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a20d00c-ae7e-42eb-8098-f0ef094975f5-kube-api-access-8fzj9" (OuterVolumeSpecName: "kube-api-access-8fzj9") pod "7a20d00c-ae7e-42eb-8098-f0ef094975f5" (UID: "7a20d00c-ae7e-42eb-8098-f0ef094975f5"). InnerVolumeSpecName "kube-api-access-8fzj9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:13:32.640078 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:32.640020    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a20d00c-ae7e-42eb-8098-f0ef094975f5-istio-token" (OuterVolumeSpecName: "istio-token") pod "7a20d00c-ae7e-42eb-8098-f0ef094975f5" (UID: "7a20d00c-ae7e-42eb-8098-f0ef094975f5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:13:32.739180 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:32.739147    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a20d00c-ae7e-42eb-8098-f0ef094975f5-workload-socket\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:13:32.739180 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:32.739174    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a20d00c-ae7e-42eb-8098-f0ef094975f5-istio-token\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:13:32.739180 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:32.739183    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a20d00c-ae7e-42eb-8098-f0ef094975f5-istio-envoy\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:13:32.739403 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:32.739190    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fzj9\" (UniqueName: \"kubernetes.io/projected/7a20d00c-ae7e-42eb-8098-f0ef094975f5-kube-api-access-8fzj9\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:13:32.739403 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:32.739199    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a20d00c-ae7e-42eb-8098-f0ef094975f5-istio-data\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:13:32.739403 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:32.739208    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a20d00c-ae7e-42eb-8098-f0ef094975f5-istio-podinfo\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:13:32.739403 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:32.739222    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a20d00c-ae7e-42eb-8098-f0ef094975f5-istiod-ca-cert\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:13:33.523859 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:33.523830    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"
Jul 30 04:13:33.524310 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:33.523862    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7" event={"ID":"7a20d00c-ae7e-42eb-8098-f0ef094975f5","Type":"ContainerDied","Data":"0069e2ea0873fe4627cf05b03c800843da56f952cdb31213b3807d2225d8e16d"}
Jul 30 04:13:33.524310 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:33.523938    2576 scope.go:117] "RemoveContainer" containerID="88ac6e13d027d2c5aa0b53df40684cf7c1d4980f4e3cdec806572d7baf6ba021"
Jul 30 04:13:33.547697 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:33.547671    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"]
Jul 30 04:13:33.550990 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:33.550967    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-246dcb85-openshift-default-84b7db85cb-clsx7"]
Jul 30 04:13:33.736737 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:33.736707    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:13:33.805655 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:33.805572    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:13:33.905426 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:33.905397    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:13:34.002059 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:34.002026    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:13:34.116971 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:34.116832    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:13:35.051612 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:35.051580    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a20d00c-ae7e-42eb-8098-f0ef094975f5" path="/var/lib/kubelet/pods/7a20d00c-ae7e-42eb-8098-f0ef094975f5/volumes"
Jul 30 04:13:40.150876 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:40.150822    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:13:40.247163 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:13:40.247122    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:14:00.230202 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:14:00.230160    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:14:59.218536 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:14:59.218498    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:14:59.357225 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:14:59.357185    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:14:59.835695 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:14:59.835656    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:28.410068 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:28.409980    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:42.663457 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:42.663420    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:42.940264 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:42.940185    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"]
Jul 30 04:15:42.940537 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:42.940524    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a20d00c-ae7e-42eb-8098-f0ef094975f5" containerName="istio-proxy"
Jul 30 04:15:42.940537 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:42.940538    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a20d00c-ae7e-42eb-8098-f0ef094975f5" containerName="istio-proxy"
Jul 30 04:15:42.940621 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:42.940552    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fa93881-df00-47d1-b06c-c6cd21047334" containerName="payload-pre-processing"
Jul 30 04:15:42.940621 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:42.940558    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fa93881-df00-47d1-b06c-c6cd21047334" containerName="payload-pre-processing"
Jul 30 04:15:42.940621 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:42.940565    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1709862-6b43-4f9b-9277-6d0c0dcc725d" containerName="revoke-keys"
Jul 30 04:15:42.940621 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:42.940571    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1709862-6b43-4f9b-9277-6d0c0dcc725d" containerName="revoke-keys"
Jul 30 04:15:42.940757 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:42.940624    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1709862-6b43-4f9b-9277-6d0c0dcc725d" containerName="revoke-keys"
Jul 30 04:15:42.940757 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:42.940634    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fa93881-df00-47d1-b06c-c6cd21047334" containerName="payload-pre-processing"
Jul 30 04:15:42.940757 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:42.940640    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a20d00c-ae7e-42eb-8098-f0ef094975f5" containerName="istio-proxy"
Jul 30 04:15:42.940757 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:42.940646    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a20d00c-ae7e-42eb-8098-f0ef094975f5" containerName="istio-proxy"
Jul 30 04:15:42.943468 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:42.943452    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:15:42.947244 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:42.947223    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-7278e753-openshift-default-dockercfg-j9tbv\""
Jul 30 04:15:42.965236 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:42.965207    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"]
Jul 30 04:15:42.978997 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:42.978967    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:43.009704 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:43.009665    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37194cdf-be1a-4b6b-98d8-472e223fd49e-credential-socket\") pod \"e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw\" (UID: \"37194cdf-be1a-4b6b-98d8-472e223fd49e\") " pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:15:43.009704 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:43.009707    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37194cdf-be1a-4b6b-98d8-472e223fd49e-istiod-ca-cert\") pod \"e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw\" (UID: \"37194cdf-be1a-4b6b-98d8-472e223fd49e\") " pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:15:43.009960 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:43.009733    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql9x6\" (UniqueName: \"kubernetes.io/projected/37194cdf-be1a-4b6b-98d8-472e223fd49e-kube-api-access-ql9x6\") pod \"e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw\" (UID: \"37194cdf-be1a-4b6b-98d8-472e223fd49e\") " pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:15:43.009960 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:43.009807    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37194cdf-be1a-4b6b-98d8-472e223fd49e-istio-data\") pod \"e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw\" (UID: \"37194cdf-be1a-4b6b-98d8-472e223fd49e\") " pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:15:43.009960 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:43.009887    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37194cdf-be1a-4b6b-98d8-472e223fd49e-istio-envoy\") pod \"e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw\" (UID: \"37194cdf-be1a-4b6b-98d8-472e223fd49e\") " pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:15:43.009960 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:43.009910    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37194cdf-be1a-4b6b-98d8-472e223fd49e-workload-socket\") pod \"e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw\" (UID: \"37194cdf-be1a-4b6b-98d8-472e223fd49e\") " pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:15:43.009960 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:43.009958    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37194cdf-be1a-4b6b-98d8-472e223fd49e-istio-podinfo\") pod \"e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw\" (UID: \"37194cdf-be1a-4b6b-98d8-472e223fd49e\") " pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:15:43.010146 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:43.009980    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37194cdf-be1a-4b6b-98d8-472e223fd49e-workload-certs\") pod \"e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw\" (UID: \"37194cdf-be1a-4b6b-98d8-472e223fd49e\") " pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:15:43.010146 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:43.009994    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37194cdf-be1a-4b6b-98d8-472e223fd49e-istio-token\") pod \"e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw\" (UID: \"37194cdf-be1a-4b6b-98d8-472e223fd49e\") " pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:15:43.083306 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:43.083273    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:43.110603 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:43.110572    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37194cdf-be1a-4b6b-98d8-472e223fd49e-istio-envoy\") pod \"e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw\" (UID: \"37194cdf-be1a-4b6b-98d8-472e223fd49e\") " pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:15:43.110772 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:43.110608    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37194cdf-be1a-4b6b-98d8-472e223fd49e-workload-socket\") pod \"e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw\" (UID: \"37194cdf-be1a-4b6b-98d8-472e223fd49e\") " pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:15:43.110772 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:43.110637    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37194cdf-be1a-4b6b-98d8-472e223fd49e-istio-podinfo\") pod \"e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw\" (UID: \"37194cdf-be1a-4b6b-98d8-472e223fd49e\") " pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:15:43.110772 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:43.110681    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37194cdf-be1a-4b6b-98d8-472e223fd49e-workload-certs\") pod \"e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw\" (UID: \"37194cdf-be1a-4b6b-98d8-472e223fd49e\") " pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:15:43.110772 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:43.110704    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37194cdf-be1a-4b6b-98d8-472e223fd49e-istio-token\") pod \"e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw\" (UID: \"37194cdf-be1a-4b6b-98d8-472e223fd49e\") " pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:15:43.110772 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:43.110758    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37194cdf-be1a-4b6b-98d8-472e223fd49e-credential-socket\") pod \"e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw\" (UID: \"37194cdf-be1a-4b6b-98d8-472e223fd49e\") " pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:15:43.111062 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:43.110789    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37194cdf-be1a-4b6b-98d8-472e223fd49e-istiod-ca-cert\") pod \"e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw\" (UID: \"37194cdf-be1a-4b6b-98d8-472e223fd49e\") " pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:15:43.111062 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:43.110821    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql9x6\" (UniqueName: \"kubernetes.io/projected/37194cdf-be1a-4b6b-98d8-472e223fd49e-kube-api-access-ql9x6\") pod \"e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw\" (UID: \"37194cdf-be1a-4b6b-98d8-472e223fd49e\") " pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:15:43.111062 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:43.110864    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37194cdf-be1a-4b6b-98d8-472e223fd49e-istio-data\") pod \"e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw\" (UID: \"37194cdf-be1a-4b6b-98d8-472e223fd49e\") " pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:15:43.111221 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:43.111121    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37194cdf-be1a-4b6b-98d8-472e223fd49e-workload-socket\") pod \"e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw\" (UID: \"37194cdf-be1a-4b6b-98d8-472e223fd49e\") " pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:15:43.111404 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:43.111381    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37194cdf-be1a-4b6b-98d8-472e223fd49e-workload-certs\") pod \"e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw\" (UID: \"37194cdf-be1a-4b6b-98d8-472e223fd49e\") " pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:15:43.111496 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:43.111413    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37194cdf-be1a-4b6b-98d8-472e223fd49e-istio-data\") pod \"e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw\" (UID: \"37194cdf-be1a-4b6b-98d8-472e223fd49e\") " pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:15:43.111496 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:43.111436    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37194cdf-be1a-4b6b-98d8-472e223fd49e-credential-socket\") pod \"e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw\" (UID: \"37194cdf-be1a-4b6b-98d8-472e223fd49e\") " pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:15:43.111653 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:43.111632    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37194cdf-be1a-4b6b-98d8-472e223fd49e-istiod-ca-cert\") pod \"e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw\" (UID: \"37194cdf-be1a-4b6b-98d8-472e223fd49e\") " pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:15:43.113529 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:43.113505    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37194cdf-be1a-4b6b-98d8-472e223fd49e-istio-envoy\") pod \"e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw\" (UID: \"37194cdf-be1a-4b6b-98d8-472e223fd49e\") " pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:15:43.113734 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:43.113718    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37194cdf-be1a-4b6b-98d8-472e223fd49e-istio-podinfo\") pod \"e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw\" (UID: \"37194cdf-be1a-4b6b-98d8-472e223fd49e\") " pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:15:43.122379 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:43.122351    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql9x6\" (UniqueName: \"kubernetes.io/projected/37194cdf-be1a-4b6b-98d8-472e223fd49e-kube-api-access-ql9x6\") pod \"e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw\" (UID: \"37194cdf-be1a-4b6b-98d8-472e223fd49e\") " pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:15:43.122523 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:43.122508    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37194cdf-be1a-4b6b-98d8-472e223fd49e-istio-token\") pod \"e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw\" (UID: \"37194cdf-be1a-4b6b-98d8-472e223fd49e\") " pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:15:43.253818 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:43.253720    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:15:43.393108 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:43.393084    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"]
Jul 30 04:15:43.394424 ip-10-0-143-117 kubenswrapper[2576]: W0730 04:15:43.394396    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37194cdf_be1a_4b6b_98d8_472e223fd49e.slice/crio-5de227af29b5effc1b3a3b475adb8f85038b0b7af3bd5a7f0e93b844db031bc2 WatchSource:0}: Error finding container 5de227af29b5effc1b3a3b475adb8f85038b0b7af3bd5a7f0e93b844db031bc2: Status 404 returned error can't find the container with id 5de227af29b5effc1b3a3b475adb8f85038b0b7af3bd5a7f0e93b844db031bc2
Jul 30 04:15:43.396553 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:43.396525    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:15:43.396631 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:43.396589    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:15:43.396631 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:43.396618    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:15:44.216019 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:44.215980    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw" event={"ID":"37194cdf-be1a-4b6b-98d8-472e223fd49e","Type":"ContainerStarted","Data":"b89b81e4d23abe46d33d690e48950b855bb6703318c8cf4214fedf7efd0bebe1"}
Jul 30 04:15:44.216414 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:44.216036    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw" event={"ID":"37194cdf-be1a-4b6b-98d8-472e223fd49e","Type":"ContainerStarted","Data":"5de227af29b5effc1b3a3b475adb8f85038b0b7af3bd5a7f0e93b844db031bc2"}
Jul 30 04:15:44.254305 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:44.254274    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:15:44.259627 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:44.259597    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:15:44.259775 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:44.259718    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw" podStartSLOduration=2.259700178 podStartE2EDuration="2.259700178s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:15:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:15:44.257702707 +0000 UTC m=+2449.875934947" watchObservedRunningTime="2026-07-30 04:15:44.259700178 +0000 UTC m=+2449.877932420"
Jul 30 04:15:45.222221 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:45.222187    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:15:45.223325 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:45.223304    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:15:46.224772 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:46.224740    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:46.325260 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:46.325226    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:46.827767 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:46.827733    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:15:47.370967 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:47.370906    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:47.573839 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:47.573786    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:47.975049 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:47.975006    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:48.082613 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:48.082579    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:15:48.185944 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:48.185766    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7278e753-77f8bf65b4-6q56v"]
Jul 30 04:15:48.195961 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:48.195307    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a20d00c-ae7e-42eb-8098-f0ef094975f5" containerName="istio-proxy"
Jul 30 04:15:48.195961 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:48.195338    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a20d00c-ae7e-42eb-8098-f0ef094975f5" containerName="istio-proxy"
Jul 30 04:15:48.198861 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:48.198831    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-7278e753-77f8bf65b4-6q56v"
Jul 30 04:15:48.205832 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:48.205803    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7278e753-77f8bf65b4-6q56v"]
Jul 30 04:15:48.206354 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:48.206334    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-7278e753\""
Jul 30 04:15:48.207036 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:48.207018    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-7278e753-dockercfg-bspf7\""
Jul 30 04:15:48.356990 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:48.356947    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8eff8ae0-353e-453e-a76d-beab47cc5b6b-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-7278e753-77f8bf65b4-6q56v\" (UID: \"8eff8ae0-353e-453e-a76d-beab47cc5b6b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7278e753-77f8bf65b4-6q56v"
Jul 30 04:15:48.357182 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:48.357040    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tzcv\" (UniqueName: \"kubernetes.io/projected/8eff8ae0-353e-453e-a76d-beab47cc5b6b-kube-api-access-7tzcv\") pod \"payload-pre-processing-e2e-mt-7278e753-77f8bf65b4-6q56v\" (UID: \"8eff8ae0-353e-453e-a76d-beab47cc5b6b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7278e753-77f8bf65b4-6q56v"
Jul 30 04:15:48.458236 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:48.458192    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tzcv\" (UniqueName: \"kubernetes.io/projected/8eff8ae0-353e-453e-a76d-beab47cc5b6b-kube-api-access-7tzcv\") pod \"payload-pre-processing-e2e-mt-7278e753-77f8bf65b4-6q56v\" (UID: \"8eff8ae0-353e-453e-a76d-beab47cc5b6b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7278e753-77f8bf65b4-6q56v"
Jul 30 04:15:48.458621 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:48.458334    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8eff8ae0-353e-453e-a76d-beab47cc5b6b-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-7278e753-77f8bf65b4-6q56v\" (UID: \"8eff8ae0-353e-453e-a76d-beab47cc5b6b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7278e753-77f8bf65b4-6q56v"
Jul 30 04:15:48.459064 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:48.459043    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8eff8ae0-353e-453e-a76d-beab47cc5b6b-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-7278e753-77f8bf65b4-6q56v\" (UID: \"8eff8ae0-353e-453e-a76d-beab47cc5b6b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7278e753-77f8bf65b4-6q56v"
Jul 30 04:15:48.466452 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:48.466429    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tzcv\" (UniqueName: \"kubernetes.io/projected/8eff8ae0-353e-453e-a76d-beab47cc5b6b-kube-api-access-7tzcv\") pod \"payload-pre-processing-e2e-mt-7278e753-77f8bf65b4-6q56v\" (UID: \"8eff8ae0-353e-453e-a76d-beab47cc5b6b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7278e753-77f8bf65b4-6q56v"
Jul 30 04:15:48.528103 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:48.528066    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-7278e753-77f8bf65b4-6q56v"
Jul 30 04:15:48.674676 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:48.674647    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7278e753-77f8bf65b4-6q56v"]
Jul 30 04:15:48.675780 ip-10-0-143-117 kubenswrapper[2576]: W0730 04:15:48.675749    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8eff8ae0_353e_453e_a76d_beab47cc5b6b.slice/crio-420738048e496ca996f3735b325c8c9d1b2411fa484da75f00ad42a937c4d19d WatchSource:0}: Error finding container 420738048e496ca996f3735b325c8c9d1b2411fa484da75f00ad42a937c4d19d: Status 404 returned error can't find the container with id 420738048e496ca996f3735b325c8c9d1b2411fa484da75f00ad42a937c4d19d
Jul 30 04:15:49.251094 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:49.251058    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-7278e753-77f8bf65b4-6q56v" event={"ID":"8eff8ae0-353e-453e-a76d-beab47cc5b6b","Type":"ContainerStarted","Data":"cf3ba8355e647bb808332dbb1d23e982a6bf7ac8d107b492571a4d952d6593ef"}
Jul 30 04:15:49.251308 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:49.251109    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-7278e753-77f8bf65b4-6q56v" event={"ID":"8eff8ae0-353e-453e-a76d-beab47cc5b6b","Type":"ContainerStarted","Data":"420738048e496ca996f3735b325c8c9d1b2411fa484da75f00ad42a937c4d19d"}
Jul 30 04:15:49.251308 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:49.251146    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-7278e753-77f8bf65b4-6q56v"
Jul 30 04:15:49.266700 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:49.266654    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-7278e753-77f8bf65b4-6q56v" podStartSLOduration=1.26663993 podStartE2EDuration="1.26663993s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:15:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:15:49.266058986 +0000 UTC m=+2454.884291238" watchObservedRunningTime="2026-07-30 04:15:49.26663993 +0000 UTC m=+2454.884872171"
Jul 30 04:15:55.202277 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:55.202247    2576 scope.go:117] "RemoveContainer" containerID="fd275687f868d9972125c8c9badadc6ad62018a469528279376849480da5152a"
Jul 30 04:15:55.210881 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:55.210861    2576 scope.go:117] "RemoveContainer" containerID="2b16d66c13b7926d5f427f92dc5ea5a3300e7d61f5667a85124b044b6d9a10ec"
Jul 30 04:15:59.126918 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:15:59.126885    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:16:00.258124 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:00.258083    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-7278e753-77f8bf65b4-6q56v"
Jul 30 04:16:04.341668 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:04.341634    2576 generic.go:356] "Generic (PLEG): container finished" podID="37194cdf-be1a-4b6b-98d8-472e223fd49e" containerID="b89b81e4d23abe46d33d690e48950b855bb6703318c8cf4214fedf7efd0bebe1" exitCode=0
Jul 30 04:16:04.342133 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:04.341676    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw" event={"ID":"37194cdf-be1a-4b6b-98d8-472e223fd49e","Type":"ContainerDied","Data":"b89b81e4d23abe46d33d690e48950b855bb6703318c8cf4214fedf7efd0bebe1"}
Jul 30 04:16:04.342133 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:04.342069    2576 scope.go:117] "RemoveContainer" containerID="b89b81e4d23abe46d33d690e48950b855bb6703318c8cf4214fedf7efd0bebe1"
Jul 30 04:16:04.343018 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:04.342992    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:16:04.343101 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:04.343058    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:16:04.343101 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:04.343089    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:16:05.358573 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:05.358534    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw" event={"ID":"37194cdf-be1a-4b6b-98d8-472e223fd49e","Type":"ContainerStarted","Data":"06b194c901751b499f056045681815defb60ec64e0b36d6069bd917a93823d50"}
Jul 30 04:16:06.254744 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:06.254706    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:16:06.255238 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:06.255212    2576 patch_prober.go:28] interesting pod/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Jul 30 04:16:06.255414 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:06.255390    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw" podUID="37194cdf-be1a-4b6b-98d8-472e223fd49e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Jul 30 04:16:07.254475 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:07.254438    2576 patch_prober.go:28] interesting pod/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Jul 30 04:16:07.255061 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:07.255030    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw" podUID="37194cdf-be1a-4b6b-98d8-472e223fd49e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Jul 30 04:16:08.254653 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:08.254611    2576 patch_prober.go:28] interesting pod/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Jul 30 04:16:08.255176 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:08.254688    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw" podUID="37194cdf-be1a-4b6b-98d8-472e223fd49e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Jul 30 04:16:09.258050 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:09.258016    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:16:09.258476 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:09.258295    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:16:09.258995 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:09.258979    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:16:11.158731 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:11.158687    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:16:15.923356 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:15.923322    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-7278e753/e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw"]
Jul 30 04:16:15.925902 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:15.925880    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7278e753/e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw"
Jul 30 04:16:15.928566 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:15.928541    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-7278e753\"/\"openshift-service-ca.crt\""
Jul 30 04:16:15.929696 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:15.929675    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-7278e753\"/\"default-dockercfg-7f4p8\""
Jul 30 04:16:15.929788 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:15.929709    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-7278e753\"/\"e2e-lifecycle-model-7278e753-kserve-self-signed-certs\""
Jul 30 04:16:15.929788 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:15.929748    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-7278e753\"/\"kube-root-ca.crt\""
Jul 30 04:16:15.936741 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:15.936719    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-7278e753/e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw"]
Jul 30 04:16:15.957304 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:15.957280    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:16:16.009000 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:16.008969    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41356a26-e7e5-49b4-b820-66cb3488caaf-home\") pod \"e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw\" (UID: \"41356a26-e7e5-49b4-b820-66cb3488caaf\") " pod="ai-tenant-e2e-mt-7278e753/e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw"
Jul 30 04:16:16.009180 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:16.009024    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41356a26-e7e5-49b4-b820-66cb3488caaf-tmp-dir\") pod \"e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw\" (UID: \"41356a26-e7e5-49b4-b820-66cb3488caaf\") " pod="ai-tenant-e2e-mt-7278e753/e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw"
Jul 30 04:16:16.009180 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:16.009091    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41356a26-e7e5-49b4-b820-66cb3488caaf-tls-certs\") pod \"e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw\" (UID: \"41356a26-e7e5-49b4-b820-66cb3488caaf\") " pod="ai-tenant-e2e-mt-7278e753/e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw"
Jul 30 04:16:16.009180 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:16.009125    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wncfn\" (UniqueName: \"kubernetes.io/projected/41356a26-e7e5-49b4-b820-66cb3488caaf-kube-api-access-wncfn\") pod \"e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw\" (UID: \"41356a26-e7e5-49b4-b820-66cb3488caaf\") " pod="ai-tenant-e2e-mt-7278e753/e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw"
Jul 30 04:16:16.009180 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:16.009160    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41356a26-e7e5-49b4-b820-66cb3488caaf-dshm\") pod \"e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw\" (UID: \"41356a26-e7e5-49b4-b820-66cb3488caaf\") " pod="ai-tenant-e2e-mt-7278e753/e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw"
Jul 30 04:16:16.009344 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:16.009220    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41356a26-e7e5-49b4-b820-66cb3488caaf-model-cache\") pod \"e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw\" (UID: \"41356a26-e7e5-49b4-b820-66cb3488caaf\") " pod="ai-tenant-e2e-mt-7278e753/e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw"
Jul 30 04:16:16.110171 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:16.110110    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41356a26-e7e5-49b4-b820-66cb3488caaf-dshm\") pod \"e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw\" (UID: \"41356a26-e7e5-49b4-b820-66cb3488caaf\") " pod="ai-tenant-e2e-mt-7278e753/e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw"
Jul 30 04:16:16.110381 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:16.110225    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41356a26-e7e5-49b4-b820-66cb3488caaf-model-cache\") pod \"e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw\" (UID: \"41356a26-e7e5-49b4-b820-66cb3488caaf\") " pod="ai-tenant-e2e-mt-7278e753/e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw"
Jul 30 04:16:16.110381 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:16.110289    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41356a26-e7e5-49b4-b820-66cb3488caaf-home\") pod \"e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw\" (UID: \"41356a26-e7e5-49b4-b820-66cb3488caaf\") " pod="ai-tenant-e2e-mt-7278e753/e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw"
Jul 30 04:16:16.110524 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:16.110388    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41356a26-e7e5-49b4-b820-66cb3488caaf-tmp-dir\") pod \"e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw\" (UID: \"41356a26-e7e5-49b4-b820-66cb3488caaf\") " pod="ai-tenant-e2e-mt-7278e753/e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw"
Jul 30 04:16:16.110524 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:16.110433    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41356a26-e7e5-49b4-b820-66cb3488caaf-tls-certs\") pod \"e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw\" (UID: \"41356a26-e7e5-49b4-b820-66cb3488caaf\") " pod="ai-tenant-e2e-mt-7278e753/e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw"
Jul 30 04:16:16.110524 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:16.110463    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wncfn\" (UniqueName: \"kubernetes.io/projected/41356a26-e7e5-49b4-b820-66cb3488caaf-kube-api-access-wncfn\") pod \"e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw\" (UID: \"41356a26-e7e5-49b4-b820-66cb3488caaf\") " pod="ai-tenant-e2e-mt-7278e753/e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw"
Jul 30 04:16:16.111007 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:16.110963    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41356a26-e7e5-49b4-b820-66cb3488caaf-model-cache\") pod \"e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw\" (UID: \"41356a26-e7e5-49b4-b820-66cb3488caaf\") " pod="ai-tenant-e2e-mt-7278e753/e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw"
Jul 30 04:16:16.111188 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:16.111154    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41356a26-e7e5-49b4-b820-66cb3488caaf-home\") pod \"e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw\" (UID: \"41356a26-e7e5-49b4-b820-66cb3488caaf\") " pod="ai-tenant-e2e-mt-7278e753/e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw"
Jul 30 04:16:16.111302 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:16.111208    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41356a26-e7e5-49b4-b820-66cb3488caaf-tmp-dir\") pod \"e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw\" (UID: \"41356a26-e7e5-49b4-b820-66cb3488caaf\") " pod="ai-tenant-e2e-mt-7278e753/e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw"
Jul 30 04:16:16.113359 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:16.113319    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41356a26-e7e5-49b4-b820-66cb3488caaf-dshm\") pod \"e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw\" (UID: \"41356a26-e7e5-49b4-b820-66cb3488caaf\") " pod="ai-tenant-e2e-mt-7278e753/e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw"
Jul 30 04:16:16.113619 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:16.113602    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41356a26-e7e5-49b4-b820-66cb3488caaf-tls-certs\") pod \"e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw\" (UID: \"41356a26-e7e5-49b4-b820-66cb3488caaf\") " pod="ai-tenant-e2e-mt-7278e753/e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw"
Jul 30 04:16:16.118373 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:16.118353    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wncfn\" (UniqueName: \"kubernetes.io/projected/41356a26-e7e5-49b4-b820-66cb3488caaf-kube-api-access-wncfn\") pod \"e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw\" (UID: \"41356a26-e7e5-49b4-b820-66cb3488caaf\") " pod="ai-tenant-e2e-mt-7278e753/e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw"
Jul 30 04:16:16.237712 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:16.237624    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7278e753/e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw"
Jul 30 04:16:16.365168 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:16.365137    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-7278e753/e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw"]
Jul 30 04:16:16.367041 ip-10-0-143-117 kubenswrapper[2576]: W0730 04:16:16.367013    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod41356a26_e7e5_49b4_b820_66cb3488caaf.slice/crio-4c5f004919a454c276408d6ed0ec5ab52391d4f53ca9b011f6b0d83e3761696d WatchSource:0}: Error finding container 4c5f004919a454c276408d6ed0ec5ab52391d4f53ca9b011f6b0d83e3761696d: Status 404 returned error can't find the container with id 4c5f004919a454c276408d6ed0ec5ab52391d4f53ca9b011f6b0d83e3761696d
Jul 30 04:16:16.418493 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:16.418468    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7278e753/e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw" event={"ID":"41356a26-e7e5-49b4-b820-66cb3488caaf","Type":"ContainerStarted","Data":"4c5f004919a454c276408d6ed0ec5ab52391d4f53ca9b011f6b0d83e3761696d"}
Jul 30 04:16:17.430916 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:17.430879    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7278e753/e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw" event={"ID":"41356a26-e7e5-49b4-b820-66cb3488caaf","Type":"ContainerStarted","Data":"066c4518706e944bdab47c92c076c2105c168fc57dcf0ae4621e106cc07cecf2"}
Jul 30 04:16:17.450100 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:17.450048    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-7278e753/e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw" podStartSLOduration=2.4500318930000002 podStartE2EDuration="2.450031893s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:16:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:16:17.449319357 +0000 UTC m=+2483.067551610" watchObservedRunningTime="2026-07-30 04:16:17.450031893 +0000 UTC m=+2483.068264165"
Jul 30 04:16:19.631100 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:19.631058    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:16:26.238572 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:26.238532    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-7278e753/e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw"
Jul 30 04:16:26.238572 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:26.238583    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-7278e753/e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw"
Jul 30 04:16:26.250889 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:26.250856    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-7278e753/e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw"
Jul 30 04:16:26.492451 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:26.492358    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-7278e753/e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw"
Jul 30 04:16:43.130053 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:43.130012    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:16:44.271417 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:44.271382    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:16:44.374682 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:44.374644    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:16:55.234341 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:16:55.234261    2576 scope.go:117] "RemoveContainer" containerID="5da4617778df8d370f63431bac3eacfcd6e6f8df978b2b48dc7cfa369b23abf4"
Jul 30 04:17:09.334409 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:09.334370    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:17:10.875146 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:10.875107    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:17:10.904072 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:10.904039    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7278e753-77f8bf65b4-6q56v"]
Jul 30 04:17:10.904326 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:10.904300    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-7278e753-77f8bf65b4-6q56v" podUID="8eff8ae0-353e-453e-a76d-beab47cc5b6b" containerName="payload-pre-processing" containerID="cri-o://cf3ba8355e647bb808332dbb1d23e982a6bf7ac8d107b492571a4d952d6593ef" gracePeriod=30
Jul 30 04:17:10.975226 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:10.975192    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:17:11.036227 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:11.036192    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:17:11.161764 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:11.161738    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-7278e753-77f8bf65b4-6q56v"
Jul 30 04:17:11.226727 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:11.226695    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7tzcv\" (UniqueName: \"kubernetes.io/projected/8eff8ae0-353e-453e-a76d-beab47cc5b6b-kube-api-access-7tzcv\") pod \"8eff8ae0-353e-453e-a76d-beab47cc5b6b\" (UID: \"8eff8ae0-353e-453e-a76d-beab47cc5b6b\") "
Jul 30 04:17:11.226893 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:11.226742    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8eff8ae0-353e-453e-a76d-beab47cc5b6b-plugins-config-volume\") pod \"8eff8ae0-353e-453e-a76d-beab47cc5b6b\" (UID: \"8eff8ae0-353e-453e-a76d-beab47cc5b6b\") "
Jul 30 04:17:11.227159 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:11.227134    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8eff8ae0-353e-453e-a76d-beab47cc5b6b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8eff8ae0-353e-453e-a76d-beab47cc5b6b" (UID: "8eff8ae0-353e-453e-a76d-beab47cc5b6b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:17:11.228765 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:11.228741    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8eff8ae0-353e-453e-a76d-beab47cc5b6b-kube-api-access-7tzcv" (OuterVolumeSpecName: "kube-api-access-7tzcv") pod "8eff8ae0-353e-453e-a76d-beab47cc5b6b" (UID: "8eff8ae0-353e-453e-a76d-beab47cc5b6b"). InnerVolumeSpecName "kube-api-access-7tzcv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:17:11.327679 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:11.327646    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7tzcv\" (UniqueName: \"kubernetes.io/projected/8eff8ae0-353e-453e-a76d-beab47cc5b6b-kube-api-access-7tzcv\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:17:11.327679 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:11.327673    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8eff8ae0-353e-453e-a76d-beab47cc5b6b-plugins-config-volume\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:17:11.732014 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:11.731977    2576 generic.go:356] "Generic (PLEG): container finished" podID="8eff8ae0-353e-453e-a76d-beab47cc5b6b" containerID="cf3ba8355e647bb808332dbb1d23e982a6bf7ac8d107b492571a4d952d6593ef" exitCode=0
Jul 30 04:17:11.732218 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:11.732020    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-7278e753-77f8bf65b4-6q56v"
Jul 30 04:17:11.732218 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:11.732023    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-7278e753-77f8bf65b4-6q56v" event={"ID":"8eff8ae0-353e-453e-a76d-beab47cc5b6b","Type":"ContainerDied","Data":"cf3ba8355e647bb808332dbb1d23e982a6bf7ac8d107b492571a4d952d6593ef"}
Jul 30 04:17:11.732218 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:11.732160    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-7278e753-77f8bf65b4-6q56v" event={"ID":"8eff8ae0-353e-453e-a76d-beab47cc5b6b","Type":"ContainerDied","Data":"420738048e496ca996f3735b325c8c9d1b2411fa484da75f00ad42a937c4d19d"}
Jul 30 04:17:11.732218 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:11.732198    2576 scope.go:117] "RemoveContainer" containerID="cf3ba8355e647bb808332dbb1d23e982a6bf7ac8d107b492571a4d952d6593ef"
Jul 30 04:17:11.740996 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:11.740974    2576 scope.go:117] "RemoveContainer" containerID="cf3ba8355e647bb808332dbb1d23e982a6bf7ac8d107b492571a4d952d6593ef"
Jul 30 04:17:11.741253 ip-10-0-143-117 kubenswrapper[2576]: E0730 04:17:11.741232    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf3ba8355e647bb808332dbb1d23e982a6bf7ac8d107b492571a4d952d6593ef\": container with ID starting with cf3ba8355e647bb808332dbb1d23e982a6bf7ac8d107b492571a4d952d6593ef not found: ID does not exist" containerID="cf3ba8355e647bb808332dbb1d23e982a6bf7ac8d107b492571a4d952d6593ef"
Jul 30 04:17:11.741318 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:11.741266    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf3ba8355e647bb808332dbb1d23e982a6bf7ac8d107b492571a4d952d6593ef"} err="failed to get container status \"cf3ba8355e647bb808332dbb1d23e982a6bf7ac8d107b492571a4d952d6593ef\": rpc error: code = NotFound desc = could not find container \"cf3ba8355e647bb808332dbb1d23e982a6bf7ac8d107b492571a4d952d6593ef\": container with ID starting with cf3ba8355e647bb808332dbb1d23e982a6bf7ac8d107b492571a4d952d6593ef not found: ID does not exist"
Jul 30 04:17:11.754993 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:11.754958    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7278e753-77f8bf65b4-6q56v"]
Jul 30 04:17:11.763115 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:11.758660    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7278e753-77f8bf65b4-6q56v"]
Jul 30 04:17:13.051505 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:13.051475    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8eff8ae0-353e-453e-a76d-beab47cc5b6b" path="/var/lib/kubelet/pods/8eff8ae0-353e-453e-a76d-beab47cc5b6b/volumes"
Jul 30 04:17:14.026276 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:14.026237    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:17:19.005012 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:19.004976    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-7278e753/e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw"]
Jul 30 04:17:19.049890 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:19.049848    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:17:20.051487 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:20.051451    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:17:34.024393 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:34.024351    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-7278e753/e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw" podUID="41356a26-e7e5-49b4-b820-66cb3488caaf" containerName="main" containerID="cri-o://066c4518706e944bdab47c92c076c2105c168fc57dcf0ae4621e106cc07cecf2" gracePeriod=45
Jul 30 04:17:34.274359 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:34.274333    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7278e753/e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw"
Jul 30 04:17:34.343712 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:34.343630    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41356a26-e7e5-49b4-b820-66cb3488caaf-home\") pod \"41356a26-e7e5-49b4-b820-66cb3488caaf\" (UID: \"41356a26-e7e5-49b4-b820-66cb3488caaf\") "
Jul 30 04:17:34.343712 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:34.343678    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41356a26-e7e5-49b4-b820-66cb3488caaf-model-cache\") pod \"41356a26-e7e5-49b4-b820-66cb3488caaf\" (UID: \"41356a26-e7e5-49b4-b820-66cb3488caaf\") "
Jul 30 04:17:34.343712 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:34.343708    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41356a26-e7e5-49b4-b820-66cb3488caaf-dshm\") pod \"41356a26-e7e5-49b4-b820-66cb3488caaf\" (UID: \"41356a26-e7e5-49b4-b820-66cb3488caaf\") "
Jul 30 04:17:34.344015 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:34.343740    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41356a26-e7e5-49b4-b820-66cb3488caaf-tmp-dir\") pod \"41356a26-e7e5-49b4-b820-66cb3488caaf\" (UID: \"41356a26-e7e5-49b4-b820-66cb3488caaf\") "
Jul 30 04:17:34.344015 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:34.343761    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41356a26-e7e5-49b4-b820-66cb3488caaf-tls-certs\") pod \"41356a26-e7e5-49b4-b820-66cb3488caaf\" (UID: \"41356a26-e7e5-49b4-b820-66cb3488caaf\") "
Jul 30 04:17:34.344015 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:34.343825    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wncfn\" (UniqueName: \"kubernetes.io/projected/41356a26-e7e5-49b4-b820-66cb3488caaf-kube-api-access-wncfn\") pod \"41356a26-e7e5-49b4-b820-66cb3488caaf\" (UID: \"41356a26-e7e5-49b4-b820-66cb3488caaf\") "
Jul 30 04:17:34.344015 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:34.343868    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41356a26-e7e5-49b4-b820-66cb3488caaf-home" (OuterVolumeSpecName: "home") pod "41356a26-e7e5-49b4-b820-66cb3488caaf" (UID: "41356a26-e7e5-49b4-b820-66cb3488caaf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:17:34.344015 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:34.344002    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41356a26-e7e5-49b4-b820-66cb3488caaf-model-cache" (OuterVolumeSpecName: "model-cache") pod "41356a26-e7e5-49b4-b820-66cb3488caaf" (UID: "41356a26-e7e5-49b4-b820-66cb3488caaf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:17:34.344291 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:34.344009    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41356a26-e7e5-49b4-b820-66cb3488caaf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "41356a26-e7e5-49b4-b820-66cb3488caaf" (UID: "41356a26-e7e5-49b4-b820-66cb3488caaf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:17:34.344291 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:34.344225    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41356a26-e7e5-49b4-b820-66cb3488caaf-home\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:17:34.344291 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:34.344244    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41356a26-e7e5-49b4-b820-66cb3488caaf-model-cache\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:17:34.344291 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:34.344260    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41356a26-e7e5-49b4-b820-66cb3488caaf-tmp-dir\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:17:34.345915 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:34.345888    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41356a26-e7e5-49b4-b820-66cb3488caaf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "41356a26-e7e5-49b4-b820-66cb3488caaf" (UID: "41356a26-e7e5-49b4-b820-66cb3488caaf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 04:17:34.346049 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:34.345998    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41356a26-e7e5-49b4-b820-66cb3488caaf-kube-api-access-wncfn" (OuterVolumeSpecName: "kube-api-access-wncfn") pod "41356a26-e7e5-49b4-b820-66cb3488caaf" (UID: "41356a26-e7e5-49b4-b820-66cb3488caaf"). InnerVolumeSpecName "kube-api-access-wncfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:17:34.346049 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:34.346005    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41356a26-e7e5-49b4-b820-66cb3488caaf-dshm" (OuterVolumeSpecName: "dshm") pod "41356a26-e7e5-49b4-b820-66cb3488caaf" (UID: "41356a26-e7e5-49b4-b820-66cb3488caaf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:17:34.445670 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:34.445632    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41356a26-e7e5-49b4-b820-66cb3488caaf-tls-certs\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:17:34.445670 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:34.445663    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wncfn\" (UniqueName: \"kubernetes.io/projected/41356a26-e7e5-49b4-b820-66cb3488caaf-kube-api-access-wncfn\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:17:34.445670 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:34.445673    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41356a26-e7e5-49b4-b820-66cb3488caaf-dshm\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:17:34.860659 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:34.860626    2576 generic.go:356] "Generic (PLEG): container finished" podID="41356a26-e7e5-49b4-b820-66cb3488caaf" containerID="066c4518706e944bdab47c92c076c2105c168fc57dcf0ae4621e106cc07cecf2" exitCode=0
Jul 30 04:17:34.860832 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:34.860668    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7278e753/e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw"
Jul 30 04:17:34.860832 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:34.860713    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7278e753/e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw" event={"ID":"41356a26-e7e5-49b4-b820-66cb3488caaf","Type":"ContainerDied","Data":"066c4518706e944bdab47c92c076c2105c168fc57dcf0ae4621e106cc07cecf2"}
Jul 30 04:17:34.860832 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:34.860761    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7278e753/e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw" event={"ID":"41356a26-e7e5-49b4-b820-66cb3488caaf","Type":"ContainerDied","Data":"4c5f004919a454c276408d6ed0ec5ab52391d4f53ca9b011f6b0d83e3761696d"}
Jul 30 04:17:34.860832 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:34.860784    2576 scope.go:117] "RemoveContainer" containerID="066c4518706e944bdab47c92c076c2105c168fc57dcf0ae4621e106cc07cecf2"
Jul 30 04:17:34.869898 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:34.869883    2576 scope.go:117] "RemoveContainer" containerID="066c4518706e944bdab47c92c076c2105c168fc57dcf0ae4621e106cc07cecf2"
Jul 30 04:17:34.870156 ip-10-0-143-117 kubenswrapper[2576]: E0730 04:17:34.870139    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"066c4518706e944bdab47c92c076c2105c168fc57dcf0ae4621e106cc07cecf2\": container with ID starting with 066c4518706e944bdab47c92c076c2105c168fc57dcf0ae4621e106cc07cecf2 not found: ID does not exist" containerID="066c4518706e944bdab47c92c076c2105c168fc57dcf0ae4621e106cc07cecf2"
Jul 30 04:17:34.870212 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:34.870168    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"066c4518706e944bdab47c92c076c2105c168fc57dcf0ae4621e106cc07cecf2"} err="failed to get container status \"066c4518706e944bdab47c92c076c2105c168fc57dcf0ae4621e106cc07cecf2\": rpc error: code = NotFound desc = could not find container \"066c4518706e944bdab47c92c076c2105c168fc57dcf0ae4621e106cc07cecf2\": container with ID starting with 066c4518706e944bdab47c92c076c2105c168fc57dcf0ae4621e106cc07cecf2 not found: ID does not exist"
Jul 30 04:17:34.881639 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:34.881611    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-7278e753/e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw"]
Jul 30 04:17:34.885265 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:34.885245    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-7278e753/e2e-lifecycle-model-7278e753-kserve-74f5b5f75f-v8vmw"]
Jul 30 04:17:35.052527 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:35.052494    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41356a26-e7e5-49b4-b820-66cb3488caaf" path="/var/lib/kubelet/pods/41356a26-e7e5-49b4-b820-66cb3488caaf/volumes"
Jul 30 04:17:55.639128 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:55.639092    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:17:56.818302 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:17:56.818255    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:18:01.660637 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:01.660598    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:18:01.721721 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:01.721690    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:18:01.919365 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:01.919263    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"]
Jul 30 04:18:01.919959 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:01.919888    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw" podUID="37194cdf-be1a-4b6b-98d8-472e223fd49e" containerName="istio-proxy" containerID="cri-o://06b194c901751b499f056045681815defb60ec64e0b36d6069bd917a93823d50" gracePeriod=30
Jul 30 04:18:01.937427 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:01.937389    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:18:02.162808 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:02.162770    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:18:04.532382 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:04.532343    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:18:07.181024 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:07.180997    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:18:07.355677 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:07.355627    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37194cdf-be1a-4b6b-98d8-472e223fd49e-istio-podinfo\") pod \"37194cdf-be1a-4b6b-98d8-472e223fd49e\" (UID: \"37194cdf-be1a-4b6b-98d8-472e223fd49e\") "
Jul 30 04:18:07.355677 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:07.355687    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ql9x6\" (UniqueName: \"kubernetes.io/projected/37194cdf-be1a-4b6b-98d8-472e223fd49e-kube-api-access-ql9x6\") pod \"37194cdf-be1a-4b6b-98d8-472e223fd49e\" (UID: \"37194cdf-be1a-4b6b-98d8-472e223fd49e\") "
Jul 30 04:18:07.355976 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:07.355715    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37194cdf-be1a-4b6b-98d8-472e223fd49e-workload-socket\") pod \"37194cdf-be1a-4b6b-98d8-472e223fd49e\" (UID: \"37194cdf-be1a-4b6b-98d8-472e223fd49e\") "
Jul 30 04:18:07.355976 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:07.355739    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37194cdf-be1a-4b6b-98d8-472e223fd49e-workload-certs\") pod \"37194cdf-be1a-4b6b-98d8-472e223fd49e\" (UID: \"37194cdf-be1a-4b6b-98d8-472e223fd49e\") "
Jul 30 04:18:07.355976 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:07.355771    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37194cdf-be1a-4b6b-98d8-472e223fd49e-istiod-ca-cert\") pod \"37194cdf-be1a-4b6b-98d8-472e223fd49e\" (UID: \"37194cdf-be1a-4b6b-98d8-472e223fd49e\") "
Jul 30 04:18:07.355976 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:07.355795    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37194cdf-be1a-4b6b-98d8-472e223fd49e-istio-token\") pod \"37194cdf-be1a-4b6b-98d8-472e223fd49e\" (UID: \"37194cdf-be1a-4b6b-98d8-472e223fd49e\") "
Jul 30 04:18:07.355976 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:07.355838    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37194cdf-be1a-4b6b-98d8-472e223fd49e-istio-data\") pod \"37194cdf-be1a-4b6b-98d8-472e223fd49e\" (UID: \"37194cdf-be1a-4b6b-98d8-472e223fd49e\") "
Jul 30 04:18:07.355976 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:07.355866    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37194cdf-be1a-4b6b-98d8-472e223fd49e-istio-envoy\") pod \"37194cdf-be1a-4b6b-98d8-472e223fd49e\" (UID: \"37194cdf-be1a-4b6b-98d8-472e223fd49e\") "
Jul 30 04:18:07.356307 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:07.356016    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37194cdf-be1a-4b6b-98d8-472e223fd49e-credential-socket\") pod \"37194cdf-be1a-4b6b-98d8-472e223fd49e\" (UID: \"37194cdf-be1a-4b6b-98d8-472e223fd49e\") "
Jul 30 04:18:07.356307 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:07.356147    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37194cdf-be1a-4b6b-98d8-472e223fd49e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "37194cdf-be1a-4b6b-98d8-472e223fd49e" (UID: "37194cdf-be1a-4b6b-98d8-472e223fd49e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:18:07.356307 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:07.356233    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37194cdf-be1a-4b6b-98d8-472e223fd49e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "37194cdf-be1a-4b6b-98d8-472e223fd49e" (UID: "37194cdf-be1a-4b6b-98d8-472e223fd49e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:18:07.356456 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:07.356368    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37194cdf-be1a-4b6b-98d8-472e223fd49e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "37194cdf-be1a-4b6b-98d8-472e223fd49e" (UID: "37194cdf-be1a-4b6b-98d8-472e223fd49e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:18:07.356511 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:07.356495    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37194cdf-be1a-4b6b-98d8-472e223fd49e-workload-certs\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:18:07.356564 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:07.356518    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37194cdf-be1a-4b6b-98d8-472e223fd49e-istiod-ca-cert\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:18:07.356564 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:07.356532    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37194cdf-be1a-4b6b-98d8-472e223fd49e-credential-socket\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:18:07.356564 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:07.356543    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37194cdf-be1a-4b6b-98d8-472e223fd49e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "37194cdf-be1a-4b6b-98d8-472e223fd49e" (UID: "37194cdf-be1a-4b6b-98d8-472e223fd49e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:18:07.356913 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:07.356880    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37194cdf-be1a-4b6b-98d8-472e223fd49e-istio-data" (OuterVolumeSpecName: "istio-data") pod "37194cdf-be1a-4b6b-98d8-472e223fd49e" (UID: "37194cdf-be1a-4b6b-98d8-472e223fd49e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:18:07.358063 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:07.358038    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37194cdf-be1a-4b6b-98d8-472e223fd49e-istio-token" (OuterVolumeSpecName: "istio-token") pod "37194cdf-be1a-4b6b-98d8-472e223fd49e" (UID: "37194cdf-be1a-4b6b-98d8-472e223fd49e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:18:07.358180 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:07.358145    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37194cdf-be1a-4b6b-98d8-472e223fd49e-kube-api-access-ql9x6" (OuterVolumeSpecName: "kube-api-access-ql9x6") pod "37194cdf-be1a-4b6b-98d8-472e223fd49e" (UID: "37194cdf-be1a-4b6b-98d8-472e223fd49e"). InnerVolumeSpecName "kube-api-access-ql9x6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:18:07.358271 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:07.358253    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/37194cdf-be1a-4b6b-98d8-472e223fd49e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "37194cdf-be1a-4b6b-98d8-472e223fd49e" (UID: "37194cdf-be1a-4b6b-98d8-472e223fd49e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 04:18:07.358470 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:07.358450    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37194cdf-be1a-4b6b-98d8-472e223fd49e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "37194cdf-be1a-4b6b-98d8-472e223fd49e" (UID: "37194cdf-be1a-4b6b-98d8-472e223fd49e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:18:07.457317 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:07.457277    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37194cdf-be1a-4b6b-98d8-472e223fd49e-istio-podinfo\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:18:07.457317 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:07.457312    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ql9x6\" (UniqueName: \"kubernetes.io/projected/37194cdf-be1a-4b6b-98d8-472e223fd49e-kube-api-access-ql9x6\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:18:07.457317 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:07.457321    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37194cdf-be1a-4b6b-98d8-472e223fd49e-workload-socket\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:18:07.457564 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:07.457332    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37194cdf-be1a-4b6b-98d8-472e223fd49e-istio-token\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:18:07.457564 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:07.457340    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37194cdf-be1a-4b6b-98d8-472e223fd49e-istio-data\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:18:07.457564 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:07.457347    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37194cdf-be1a-4b6b-98d8-472e223fd49e-istio-envoy\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:18:08.044226 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:08.044186    2576 generic.go:356] "Generic (PLEG): container finished" podID="37194cdf-be1a-4b6b-98d8-472e223fd49e" containerID="06b194c901751b499f056045681815defb60ec64e0b36d6069bd917a93823d50" exitCode=0
Jul 30 04:18:08.044435 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:08.044252    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"
Jul 30 04:18:08.044435 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:08.044275    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw" event={"ID":"37194cdf-be1a-4b6b-98d8-472e223fd49e","Type":"ContainerDied","Data":"06b194c901751b499f056045681815defb60ec64e0b36d6069bd917a93823d50"}
Jul 30 04:18:08.044435 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:08.044364    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw" event={"ID":"37194cdf-be1a-4b6b-98d8-472e223fd49e","Type":"ContainerDied","Data":"5de227af29b5effc1b3a3b475adb8f85038b0b7af3bd5a7f0e93b844db031bc2"}
Jul 30 04:18:08.044435 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:08.044395    2576 scope.go:117] "RemoveContainer" containerID="06b194c901751b499f056045681815defb60ec64e0b36d6069bd917a93823d50"
Jul 30 04:18:08.053801 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:08.053782    2576 scope.go:117] "RemoveContainer" containerID="b89b81e4d23abe46d33d690e48950b855bb6703318c8cf4214fedf7efd0bebe1"
Jul 30 04:18:08.061977 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:08.061955    2576 scope.go:117] "RemoveContainer" containerID="06b194c901751b499f056045681815defb60ec64e0b36d6069bd917a93823d50"
Jul 30 04:18:08.062285 ip-10-0-143-117 kubenswrapper[2576]: E0730 04:18:08.062256    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06b194c901751b499f056045681815defb60ec64e0b36d6069bd917a93823d50\": container with ID starting with 06b194c901751b499f056045681815defb60ec64e0b36d6069bd917a93823d50 not found: ID does not exist" containerID="06b194c901751b499f056045681815defb60ec64e0b36d6069bd917a93823d50"
Jul 30 04:18:08.062346 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:08.062295    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06b194c901751b499f056045681815defb60ec64e0b36d6069bd917a93823d50"} err="failed to get container status \"06b194c901751b499f056045681815defb60ec64e0b36d6069bd917a93823d50\": rpc error: code = NotFound desc = could not find container \"06b194c901751b499f056045681815defb60ec64e0b36d6069bd917a93823d50\": container with ID starting with 06b194c901751b499f056045681815defb60ec64e0b36d6069bd917a93823d50 not found: ID does not exist"
Jul 30 04:18:08.062346 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:08.062313    2576 scope.go:117] "RemoveContainer" containerID="b89b81e4d23abe46d33d690e48950b855bb6703318c8cf4214fedf7efd0bebe1"
Jul 30 04:18:08.062645 ip-10-0-143-117 kubenswrapper[2576]: E0730 04:18:08.062618    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b89b81e4d23abe46d33d690e48950b855bb6703318c8cf4214fedf7efd0bebe1\": container with ID starting with b89b81e4d23abe46d33d690e48950b855bb6703318c8cf4214fedf7efd0bebe1 not found: ID does not exist" containerID="b89b81e4d23abe46d33d690e48950b855bb6703318c8cf4214fedf7efd0bebe1"
Jul 30 04:18:08.062720 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:08.062652    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b89b81e4d23abe46d33d690e48950b855bb6703318c8cf4214fedf7efd0bebe1"} err="failed to get container status \"b89b81e4d23abe46d33d690e48950b855bb6703318c8cf4214fedf7efd0bebe1\": rpc error: code = NotFound desc = could not find container \"b89b81e4d23abe46d33d690e48950b855bb6703318c8cf4214fedf7efd0bebe1\": container with ID starting with b89b81e4d23abe46d33d690e48950b855bb6703318c8cf4214fedf7efd0bebe1 not found: ID does not exist"
Jul 30 04:18:08.065631 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:08.065611    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"]
Jul 30 04:18:08.069671 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:08.069652    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-7278e753-openshift-default-bd95d768d-hlgsw"]
Jul 30 04:18:09.051263 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:09.051230    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37194cdf-be1a-4b6b-98d8-472e223fd49e" path="/var/lib/kubelet/pods/37194cdf-be1a-4b6b-98d8-472e223fd49e/volumes"
Jul 30 04:18:23.702026 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:23.701939    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:18:23.806384 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:23.806352    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:18:23.898087 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:23.898050    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:18:24.521593 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:24.521554    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:18:24.618615 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:24.618577    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:18:24.718067 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:24.718038    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:18:25.924146 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:25.924112    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:18:28.328434 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:28.328401    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:18:31.503202 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:31.503167    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:18:31.603305 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:31.603270    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:18:32.162911 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:32.162864    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:18:37.533893 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:37.533860    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:18:39.090726 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:39.090688    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:18:39.193513 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:39.193458    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:18:39.292685 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:39.292651    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:18:49.757610 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:49.757580    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:18:49.974237 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:18:49.974204    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:10.903793 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:10.903752    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:16.734601 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:16.734560    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:17.511705 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:17.511670    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:17.610556 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:17.610513    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:17.835331 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:17.835285    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:18.120203 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.120126    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"]
Jul 30 04:19:18.120480 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.120468    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37194cdf-be1a-4b6b-98d8-472e223fd49e" containerName="istio-proxy"
Jul 30 04:19:18.120480 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.120481    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="37194cdf-be1a-4b6b-98d8-472e223fd49e" containerName="istio-proxy"
Jul 30 04:19:18.120572 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.120493    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41356a26-e7e5-49b4-b820-66cb3488caaf" containerName="main"
Jul 30 04:19:18.120572 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.120498    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="41356a26-e7e5-49b4-b820-66cb3488caaf" containerName="main"
Jul 30 04:19:18.120572 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.120515    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37194cdf-be1a-4b6b-98d8-472e223fd49e" containerName="istio-proxy"
Jul 30 04:19:18.120572 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.120522    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="37194cdf-be1a-4b6b-98d8-472e223fd49e" containerName="istio-proxy"
Jul 30 04:19:18.120572 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.120530    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8eff8ae0-353e-453e-a76d-beab47cc5b6b" containerName="payload-pre-processing"
Jul 30 04:19:18.120572 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.120535    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8eff8ae0-353e-453e-a76d-beab47cc5b6b" containerName="payload-pre-processing"
Jul 30 04:19:18.120775 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.120597    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="41356a26-e7e5-49b4-b820-66cb3488caaf" containerName="main"
Jul 30 04:19:18.120775 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.120605    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8eff8ae0-353e-453e-a76d-beab47cc5b6b" containerName="payload-pre-processing"
Jul 30 04:19:18.120775 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.120610    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="37194cdf-be1a-4b6b-98d8-472e223fd49e" containerName="istio-proxy"
Jul 30 04:19:18.120775 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.120617    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="37194cdf-be1a-4b6b-98d8-472e223fd49e" containerName="istio-proxy"
Jul 30 04:19:18.123173 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.123158    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:19:18.129686 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.129241    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-8d492b-openshift-default-dockercfg-qn9km\""
Jul 30 04:19:18.143958 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.143911    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"]
Jul 30 04:19:18.152319 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.152283    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:18.189765 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.189725    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5732b28f-b1be-4de2-8708-a535a8d559e7-istiod-ca-cert\") pod \"e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn\" (UID: \"5732b28f-b1be-4de2-8708-a535a8d559e7\") " pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:19:18.189971 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.189772    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5732b28f-b1be-4de2-8708-a535a8d559e7-workload-certs\") pod \"e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn\" (UID: \"5732b28f-b1be-4de2-8708-a535a8d559e7\") " pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:19:18.189971 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.189840    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5732b28f-b1be-4de2-8708-a535a8d559e7-workload-socket\") pod \"e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn\" (UID: \"5732b28f-b1be-4de2-8708-a535a8d559e7\") " pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:19:18.189971 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.189885    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5732b28f-b1be-4de2-8708-a535a8d559e7-istio-data\") pod \"e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn\" (UID: \"5732b28f-b1be-4de2-8708-a535a8d559e7\") " pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:19:18.189971 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.189947    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5732b28f-b1be-4de2-8708-a535a8d559e7-istio-podinfo\") pod \"e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn\" (UID: \"5732b28f-b1be-4de2-8708-a535a8d559e7\") " pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:19:18.190113 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.190001    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5732b28f-b1be-4de2-8708-a535a8d559e7-istio-token\") pod \"e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn\" (UID: \"5732b28f-b1be-4de2-8708-a535a8d559e7\") " pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:19:18.190113 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.190050    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5732b28f-b1be-4de2-8708-a535a8d559e7-istio-envoy\") pod \"e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn\" (UID: \"5732b28f-b1be-4de2-8708-a535a8d559e7\") " pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:19:18.190113 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.190093    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jg2rv\" (UniqueName: \"kubernetes.io/projected/5732b28f-b1be-4de2-8708-a535a8d559e7-kube-api-access-jg2rv\") pod \"e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn\" (UID: \"5732b28f-b1be-4de2-8708-a535a8d559e7\") " pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:19:18.190217 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.190125    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5732b28f-b1be-4de2-8708-a535a8d559e7-credential-socket\") pod \"e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn\" (UID: \"5732b28f-b1be-4de2-8708-a535a8d559e7\") " pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:19:18.219427 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.219391    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:18.291297 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.291265    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5732b28f-b1be-4de2-8708-a535a8d559e7-istiod-ca-cert\") pod \"e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn\" (UID: \"5732b28f-b1be-4de2-8708-a535a8d559e7\") " pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:19:18.291521 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.291305    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5732b28f-b1be-4de2-8708-a535a8d559e7-workload-certs\") pod \"e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn\" (UID: \"5732b28f-b1be-4de2-8708-a535a8d559e7\") " pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:19:18.291521 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.291339    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5732b28f-b1be-4de2-8708-a535a8d559e7-workload-socket\") pod \"e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn\" (UID: \"5732b28f-b1be-4de2-8708-a535a8d559e7\") " pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:19:18.291521 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.291374    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5732b28f-b1be-4de2-8708-a535a8d559e7-istio-data\") pod \"e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn\" (UID: \"5732b28f-b1be-4de2-8708-a535a8d559e7\") " pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:19:18.291521 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.291397    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5732b28f-b1be-4de2-8708-a535a8d559e7-istio-podinfo\") pod \"e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn\" (UID: \"5732b28f-b1be-4de2-8708-a535a8d559e7\") " pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:19:18.291521 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.291417    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5732b28f-b1be-4de2-8708-a535a8d559e7-istio-token\") pod \"e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn\" (UID: \"5732b28f-b1be-4de2-8708-a535a8d559e7\") " pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:19:18.291521 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.291464    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5732b28f-b1be-4de2-8708-a535a8d559e7-istio-envoy\") pod \"e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn\" (UID: \"5732b28f-b1be-4de2-8708-a535a8d559e7\") " pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:19:18.291521 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.291497    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jg2rv\" (UniqueName: \"kubernetes.io/projected/5732b28f-b1be-4de2-8708-a535a8d559e7-kube-api-access-jg2rv\") pod \"e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn\" (UID: \"5732b28f-b1be-4de2-8708-a535a8d559e7\") " pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:19:18.291939 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.291814    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5732b28f-b1be-4de2-8708-a535a8d559e7-workload-socket\") pod \"e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn\" (UID: \"5732b28f-b1be-4de2-8708-a535a8d559e7\") " pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:19:18.291939 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.291866    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5732b28f-b1be-4de2-8708-a535a8d559e7-credential-socket\") pod \"e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn\" (UID: \"5732b28f-b1be-4de2-8708-a535a8d559e7\") " pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:19:18.291939 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.291867    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5732b28f-b1be-4de2-8708-a535a8d559e7-workload-certs\") pod \"e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn\" (UID: \"5732b28f-b1be-4de2-8708-a535a8d559e7\") " pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:19:18.291939 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.291895    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5732b28f-b1be-4de2-8708-a535a8d559e7-istio-data\") pod \"e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn\" (UID: \"5732b28f-b1be-4de2-8708-a535a8d559e7\") " pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:19:18.292169 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.292121    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5732b28f-b1be-4de2-8708-a535a8d559e7-credential-socket\") pod \"e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn\" (UID: \"5732b28f-b1be-4de2-8708-a535a8d559e7\") " pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:19:18.292224 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.292169    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5732b28f-b1be-4de2-8708-a535a8d559e7-istiod-ca-cert\") pod \"e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn\" (UID: \"5732b28f-b1be-4de2-8708-a535a8d559e7\") " pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:19:18.293707 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.293687    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5732b28f-b1be-4de2-8708-a535a8d559e7-istio-envoy\") pod \"e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn\" (UID: \"5732b28f-b1be-4de2-8708-a535a8d559e7\") " pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:19:18.294035 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.294018    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5732b28f-b1be-4de2-8708-a535a8d559e7-istio-podinfo\") pod \"e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn\" (UID: \"5732b28f-b1be-4de2-8708-a535a8d559e7\") " pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:19:18.299267 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.299245    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5732b28f-b1be-4de2-8708-a535a8d559e7-istio-token\") pod \"e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn\" (UID: \"5732b28f-b1be-4de2-8708-a535a8d559e7\") " pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:19:18.299466 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.299448    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jg2rv\" (UniqueName: \"kubernetes.io/projected/5732b28f-b1be-4de2-8708-a535a8d559e7-kube-api-access-jg2rv\") pod \"e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn\" (UID: \"5732b28f-b1be-4de2-8708-a535a8d559e7\") " pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:19:18.433493 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.433389    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:19:18.567628 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.567590    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"]
Jul 30 04:19:18.568764 ip-10-0-143-117 kubenswrapper[2576]: W0730 04:19:18.568736    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5732b28f_b1be_4de2_8708_a535a8d559e7.slice/crio-229edc00113ab561291d9544353db4f5e5b1deb0f81853e600d78a957963aec1 WatchSource:0}: Error finding container 229edc00113ab561291d9544353db4f5e5b1deb0f81853e600d78a957963aec1: Status 404 returned error can't find the container with id 229edc00113ab561291d9544353db4f5e5b1deb0f81853e600d78a957963aec1
Jul 30 04:19:18.570939 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.570896    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:19:18.571022 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.570972    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:19:18.571022 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:18.571006    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:19:19.432118 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:19.432073    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn" event={"ID":"5732b28f-b1be-4de2-8708-a535a8d559e7","Type":"ContainerStarted","Data":"8f11082eaf7df52abe847e02052a0717c8e3792a1d3ce9e646c52448b8f16546"}
Jul 30 04:19:19.432513 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:19.432171    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn" event={"ID":"5732b28f-b1be-4de2-8708-a535a8d559e7","Type":"ContainerStarted","Data":"229edc00113ab561291d9544353db4f5e5b1deb0f81853e600d78a957963aec1"}
Jul 30 04:19:19.434414 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:19.434399    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:19:19.439419 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:19.439397    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:19:19.455121 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:19.455080    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn" podStartSLOduration=1.455066663 podStartE2EDuration="1.455066663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:19:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:19:19.452180597 +0000 UTC m=+2665.070412839" watchObservedRunningTime="2026-07-30 04:19:19.455066663 +0000 UTC m=+2665.073298903"
Jul 30 04:19:20.438051 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:20.438014    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:19:20.439023 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:20.438989    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:19:21.326335 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:21.326299    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:21.426285 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:21.426246    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:22.609871 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:22.609828    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:22.718901 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:22.718857    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:23.099306 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:23.099273    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:23.199493 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:23.199448    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:23.296444 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:23.296409    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-8d492b-7464844d44-dgdqf"]
Jul 30 04:19:23.300012 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:23.299997    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8d492b-7464844d44-dgdqf"
Jul 30 04:19:23.303310 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:23.303289    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-8d492b\""
Jul 30 04:19:23.303310 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:23.303303    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-8d492b-dockercfg-lgsjk\""
Jul 30 04:19:23.314351 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:23.314325    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-8d492b-7464844d44-dgdqf"]
Jul 30 04:19:23.445863 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:23.445782    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqjp6\" (UniqueName: \"kubernetes.io/projected/7b6bc68a-c396-44b8-b562-704153888052-kube-api-access-zqjp6\") pod \"payload-pre-processing-e2e-inf-a-8d492b-7464844d44-dgdqf\" (UID: \"7b6bc68a-c396-44b8-b562-704153888052\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8d492b-7464844d44-dgdqf"
Jul 30 04:19:23.446036 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:23.445899    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b6bc68a-c396-44b8-b562-704153888052-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-8d492b-7464844d44-dgdqf\" (UID: \"7b6bc68a-c396-44b8-b562-704153888052\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8d492b-7464844d44-dgdqf"
Jul 30 04:19:23.546915 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:23.546876    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b6bc68a-c396-44b8-b562-704153888052-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-8d492b-7464844d44-dgdqf\" (UID: \"7b6bc68a-c396-44b8-b562-704153888052\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8d492b-7464844d44-dgdqf"
Jul 30 04:19:23.547176 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:23.546980    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqjp6\" (UniqueName: \"kubernetes.io/projected/7b6bc68a-c396-44b8-b562-704153888052-kube-api-access-zqjp6\") pod \"payload-pre-processing-e2e-inf-a-8d492b-7464844d44-dgdqf\" (UID: \"7b6bc68a-c396-44b8-b562-704153888052\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8d492b-7464844d44-dgdqf"
Jul 30 04:19:23.547807 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:23.547775    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b6bc68a-c396-44b8-b562-704153888052-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-8d492b-7464844d44-dgdqf\" (UID: \"7b6bc68a-c396-44b8-b562-704153888052\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8d492b-7464844d44-dgdqf"
Jul 30 04:19:23.555249 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:23.555224    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqjp6\" (UniqueName: \"kubernetes.io/projected/7b6bc68a-c396-44b8-b562-704153888052-kube-api-access-zqjp6\") pod \"payload-pre-processing-e2e-inf-a-8d492b-7464844d44-dgdqf\" (UID: \"7b6bc68a-c396-44b8-b562-704153888052\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8d492b-7464844d44-dgdqf"
Jul 30 04:19:23.610667 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:23.610635    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8d492b-7464844d44-dgdqf"
Jul 30 04:19:23.741322 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:23.741292    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-8d492b-7464844d44-dgdqf"]
Jul 30 04:19:23.742602 ip-10-0-143-117 kubenswrapper[2576]: W0730 04:19:23.742567    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b6bc68a_c396_44b8_b562_704153888052.slice/crio-e0def3739b1bf116dd25562a604b361a0938ec3afa6962f594b53377e893bd16 WatchSource:0}: Error finding container e0def3739b1bf116dd25562a604b361a0938ec3afa6962f594b53377e893bd16: Status 404 returned error can't find the container with id e0def3739b1bf116dd25562a604b361a0938ec3afa6962f594b53377e893bd16
Jul 30 04:19:24.460312 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:24.460272    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8d492b-7464844d44-dgdqf" event={"ID":"7b6bc68a-c396-44b8-b562-704153888052","Type":"ContainerStarted","Data":"4505b199fa8c5befdaaa908453ea7db38f2f4a6369b1c1b8c51a0fda44d1baf3"}
Jul 30 04:19:24.460520 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:24.460326    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8d492b-7464844d44-dgdqf" event={"ID":"7b6bc68a-c396-44b8-b562-704153888052","Type":"ContainerStarted","Data":"e0def3739b1bf116dd25562a604b361a0938ec3afa6962f594b53377e893bd16"}
Jul 30 04:19:24.460520 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:24.460372    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8d492b-7464844d44-dgdqf"
Jul 30 04:19:24.480471 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:24.480425    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8d492b-7464844d44-dgdqf" podStartSLOduration=1.480412135 podStartE2EDuration="1.480412135s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:19:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:19:24.478635085 +0000 UTC m=+2670.096867327" watchObservedRunningTime="2026-07-30 04:19:24.480412135 +0000 UTC m=+2670.098644376"
Jul 30 04:19:27.641339 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:27.641305    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:19:35.439168 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:35.439128    2576 patch_prober.go:28] interesting pod/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Jul 30 04:19:35.439496 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:35.439206    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn" podUID="5732b28f-b1be-4de2-8708-a535a8d559e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Jul 30 04:19:35.467416 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:35.467394    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8d492b-7464844d44-dgdqf"
Jul 30 04:19:35.520868 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:35.520835    2576 generic.go:356] "Generic (PLEG): container finished" podID="5732b28f-b1be-4de2-8708-a535a8d559e7" containerID="8f11082eaf7df52abe847e02052a0717c8e3792a1d3ce9e646c52448b8f16546" exitCode=0
Jul 30 04:19:35.521045 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:35.520891    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn" event={"ID":"5732b28f-b1be-4de2-8708-a535a8d559e7","Type":"ContainerDied","Data":"8f11082eaf7df52abe847e02052a0717c8e3792a1d3ce9e646c52448b8f16546"}
Jul 30 04:19:35.521440 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:35.521419    2576 scope.go:117] "RemoveContainer" containerID="8f11082eaf7df52abe847e02052a0717c8e3792a1d3ce9e646c52448b8f16546"
Jul 30 04:19:35.522640 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:35.522612    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:19:35.522726 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:35.522709    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:19:35.522770 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:35.522749    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:19:36.528446 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:36.528406    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn" event={"ID":"5732b28f-b1be-4de2-8708-a535a8d559e7","Type":"ContainerStarted","Data":"fdd52ad1fcf92153830f9c974886786e7b440f8e628937b4a487f8bb1160544b"}
Jul 30 04:19:37.434091 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:37.434047    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:19:37.434407 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:37.434364    2576 patch_prober.go:28] interesting pod/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Jul 30 04:19:37.434499 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:37.434457    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn" podUID="5732b28f-b1be-4de2-8708-a535a8d559e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Jul 30 04:19:38.434496 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:38.434453    2576 patch_prober.go:28] interesting pod/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Jul 30 04:19:38.434998 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:38.434554    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn" podUID="5732b28f-b1be-4de2-8708-a535a8d559e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Jul 30 04:19:39.434884 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:39.434844    2576 patch_prober.go:28] interesting pod/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Jul 30 04:19:39.435409 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:39.434907    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn" podUID="5732b28f-b1be-4de2-8708-a535a8d559e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Jul 30 04:19:40.434643 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:40.434606    2576 patch_prober.go:28] interesting pod/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Jul 30 04:19:40.434847 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:40.434663    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn" podUID="5732b28f-b1be-4de2-8708-a535a8d559e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Jul 30 04:19:41.438278 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:41.438246    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:19:41.438670 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:41.438577    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:19:41.439148 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:41.439132    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:19:46.265675 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:46.265581    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:46.595470 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:46.595426    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:46.886529 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:46.886456    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"]
Jul 30 04:19:46.888999 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:46.888976    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:19:46.891986 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:46.891964    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-5f61e5-openshift-default-dockercfg-6pfq8\""
Jul 30 04:19:46.904105 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:46.904078    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"]
Jul 30 04:19:46.925565 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:46.925531    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:46.973103 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:46.973070    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dc89457-4084-4353-9043-3bf110a777a5-istio-podinfo\") pod \"e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5\" (UID: \"6dc89457-4084-4353-9043-3bf110a777a5\") " pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:19:46.973279 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:46.973109    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dc89457-4084-4353-9043-3bf110a777a5-credential-socket\") pod \"e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5\" (UID: \"6dc89457-4084-4353-9043-3bf110a777a5\") " pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:19:46.973279 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:46.973178    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dc89457-4084-4353-9043-3bf110a777a5-workload-socket\") pod \"e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5\" (UID: \"6dc89457-4084-4353-9043-3bf110a777a5\") " pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:19:46.973279 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:46.973223    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dc89457-4084-4353-9043-3bf110a777a5-workload-certs\") pod \"e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5\" (UID: \"6dc89457-4084-4353-9043-3bf110a777a5\") " pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:19:46.973279 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:46.973249    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dc89457-4084-4353-9043-3bf110a777a5-istiod-ca-cert\") pod \"e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5\" (UID: \"6dc89457-4084-4353-9043-3bf110a777a5\") " pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:19:46.973279 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:46.973269    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9tjr\" (UniqueName: \"kubernetes.io/projected/6dc89457-4084-4353-9043-3bf110a777a5-kube-api-access-p9tjr\") pod \"e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5\" (UID: \"6dc89457-4084-4353-9043-3bf110a777a5\") " pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:19:46.973462 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:46.973329    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dc89457-4084-4353-9043-3bf110a777a5-istio-envoy\") pod \"e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5\" (UID: \"6dc89457-4084-4353-9043-3bf110a777a5\") " pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:19:46.973462 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:46.973348    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dc89457-4084-4353-9043-3bf110a777a5-istio-token\") pod \"e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5\" (UID: \"6dc89457-4084-4353-9043-3bf110a777a5\") " pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:19:46.973462 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:46.973421    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dc89457-4084-4353-9043-3bf110a777a5-istio-data\") pod \"e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5\" (UID: \"6dc89457-4084-4353-9043-3bf110a777a5\") " pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:19:47.074194 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:47.074158    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dc89457-4084-4353-9043-3bf110a777a5-workload-socket\") pod \"e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5\" (UID: \"6dc89457-4084-4353-9043-3bf110a777a5\") " pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:19:47.074393 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:47.074203    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dc89457-4084-4353-9043-3bf110a777a5-workload-certs\") pod \"e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5\" (UID: \"6dc89457-4084-4353-9043-3bf110a777a5\") " pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:19:47.074393 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:47.074221    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dc89457-4084-4353-9043-3bf110a777a5-istiod-ca-cert\") pod \"e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5\" (UID: \"6dc89457-4084-4353-9043-3bf110a777a5\") " pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:19:47.074393 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:47.074334    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9tjr\" (UniqueName: \"kubernetes.io/projected/6dc89457-4084-4353-9043-3bf110a777a5-kube-api-access-p9tjr\") pod \"e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5\" (UID: \"6dc89457-4084-4353-9043-3bf110a777a5\") " pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:19:47.074393 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:47.074385    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dc89457-4084-4353-9043-3bf110a777a5-istio-envoy\") pod \"e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5\" (UID: \"6dc89457-4084-4353-9043-3bf110a777a5\") " pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:19:47.074633 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:47.074412    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dc89457-4084-4353-9043-3bf110a777a5-istio-token\") pod \"e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5\" (UID: \"6dc89457-4084-4353-9043-3bf110a777a5\") " pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:19:47.074633 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:47.074492    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dc89457-4084-4353-9043-3bf110a777a5-istio-data\") pod \"e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5\" (UID: \"6dc89457-4084-4353-9043-3bf110a777a5\") " pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:19:47.074633 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:47.074544    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dc89457-4084-4353-9043-3bf110a777a5-workload-socket\") pod \"e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5\" (UID: \"6dc89457-4084-4353-9043-3bf110a777a5\") " pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:19:47.074633 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:47.074556    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dc89457-4084-4353-9043-3bf110a777a5-istio-podinfo\") pod \"e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5\" (UID: \"6dc89457-4084-4353-9043-3bf110a777a5\") " pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:19:47.074633 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:47.074596    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dc89457-4084-4353-9043-3bf110a777a5-workload-certs\") pod \"e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5\" (UID: \"6dc89457-4084-4353-9043-3bf110a777a5\") " pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:19:47.074633 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:47.074603    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dc89457-4084-4353-9043-3bf110a777a5-credential-socket\") pod \"e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5\" (UID: \"6dc89457-4084-4353-9043-3bf110a777a5\") " pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:19:47.074995 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:47.074889    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dc89457-4084-4353-9043-3bf110a777a5-istio-data\") pod \"e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5\" (UID: \"6dc89457-4084-4353-9043-3bf110a777a5\") " pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:19:47.074995 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:47.074946    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dc89457-4084-4353-9043-3bf110a777a5-credential-socket\") pod \"e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5\" (UID: \"6dc89457-4084-4353-9043-3bf110a777a5\") " pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:19:47.075168 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:47.075149    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dc89457-4084-4353-9043-3bf110a777a5-istiod-ca-cert\") pod \"e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5\" (UID: \"6dc89457-4084-4353-9043-3bf110a777a5\") " pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:19:47.076783 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:47.076761    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dc89457-4084-4353-9043-3bf110a777a5-istio-envoy\") pod \"e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5\" (UID: \"6dc89457-4084-4353-9043-3bf110a777a5\") " pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:19:47.077034 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:47.077016    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dc89457-4084-4353-9043-3bf110a777a5-istio-podinfo\") pod \"e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5\" (UID: \"6dc89457-4084-4353-9043-3bf110a777a5\") " pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:19:47.083402 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:47.083380    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dc89457-4084-4353-9043-3bf110a777a5-istio-token\") pod \"e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5\" (UID: \"6dc89457-4084-4353-9043-3bf110a777a5\") " pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:19:47.083617 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:47.083596    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9tjr\" (UniqueName: \"kubernetes.io/projected/6dc89457-4084-4353-9043-3bf110a777a5-kube-api-access-p9tjr\") pod \"e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5\" (UID: \"6dc89457-4084-4353-9043-3bf110a777a5\") " pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:19:47.159285 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:47.159197    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:47.199259 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:47.199235    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:19:47.331427 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:47.331396    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"]
Jul 30 04:19:47.333367 ip-10-0-143-117 kubenswrapper[2576]: W0730 04:19:47.333340    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dc89457_4084_4353_9043_3bf110a777a5.slice/crio-61bfa04c3e965e0a02b80f68373747d87d8dacf147119cb28e063561f8675825 WatchSource:0}: Error finding container 61bfa04c3e965e0a02b80f68373747d87d8dacf147119cb28e063561f8675825: Status 404 returned error can't find the container with id 61bfa04c3e965e0a02b80f68373747d87d8dacf147119cb28e063561f8675825
Jul 30 04:19:47.336062 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:47.336032    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:19:47.336139 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:47.336097    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:19:47.336139 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:47.336125    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:19:47.595154 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:47.595115    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5" event={"ID":"6dc89457-4084-4353-9043-3bf110a777a5","Type":"ContainerStarted","Data":"d0719ddbf22f362c2b789cc6184bfe3f2418f2efad751bd6144c6ca79ad2c78a"}
Jul 30 04:19:47.595403 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:47.595180    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5" event={"ID":"6dc89457-4084-4353-9043-3bf110a777a5","Type":"ContainerStarted","Data":"61bfa04c3e965e0a02b80f68373747d87d8dacf147119cb28e063561f8675825"}
Jul 30 04:19:47.618050 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:47.617999    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5" podStartSLOduration=1.61798216 podStartE2EDuration="1.61798216s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:19:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:19:47.615597252 +0000 UTC m=+2693.233829492" watchObservedRunningTime="2026-07-30 04:19:47.61798216 +0000 UTC m=+2693.236214658"
Jul 30 04:19:48.199769 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:48.199732    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:19:48.204596 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:48.204573    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:19:48.600606 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:48.600582    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:19:48.601613 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:48.601592    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:19:49.511416 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:49.511380    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:49.614876 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:49.614841    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:50.786477 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:50.786444    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:51.131603 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:51.131519    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:51.234171 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:51.234136    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:51.366501 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:51.366468    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:51.456740 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:51.456653    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:19:55.282608 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:55.282576    2576 scope.go:117] "RemoveContainer" containerID="1acda411bc06270b757a875ef7bd808893dbd817b96d05adfb0c14bff8cef9fa"
Jul 30 04:19:57.328428 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:19:57.328387    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:20:04.642096 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:04.642047    2576 patch_prober.go:28] interesting pod/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 30 04:20:04.642532 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:04.642116    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5" podUID="6dc89457-4084-4353-9043-3bf110a777a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 30 04:20:05.694737 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:05.694698    2576 generic.go:356] "Generic (PLEG): container finished" podID="6dc89457-4084-4353-9043-3bf110a777a5" containerID="d0719ddbf22f362c2b789cc6184bfe3f2418f2efad751bd6144c6ca79ad2c78a" exitCode=0
Jul 30 04:20:05.695213 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:05.694769    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5" event={"ID":"6dc89457-4084-4353-9043-3bf110a777a5","Type":"ContainerDied","Data":"d0719ddbf22f362c2b789cc6184bfe3f2418f2efad751bd6144c6ca79ad2c78a"}
Jul 30 04:20:05.695213 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:05.695200    2576 scope.go:117] "RemoveContainer" containerID="d0719ddbf22f362c2b789cc6184bfe3f2418f2efad751bd6144c6ca79ad2c78a"
Jul 30 04:20:05.696275 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:05.696248    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:20:05.696351 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:05.696316    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:20:05.696391 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:05.696349    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:20:06.702136 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:06.702099    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5" event={"ID":"6dc89457-4084-4353-9043-3bf110a777a5","Type":"ContainerStarted","Data":"4aa058c21255aa3841a0d5f5ee3f7c307c7e108eaa071697a7b1fb11f3b57a91"}
Jul 30 04:20:07.199972 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:07.199916    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:20:07.200164 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:07.200139    2576 patch_prober.go:28] interesting pod/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Jul 30 04:20:07.200212 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:07.200193    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5" podUID="6dc89457-4084-4353-9043-3bf110a777a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Jul 30 04:20:08.200399 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:08.200363    2576 patch_prober.go:28] interesting pod/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Jul 30 04:20:08.200813 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:08.200418    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5" podUID="6dc89457-4084-4353-9043-3bf110a777a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Jul 30 04:20:09.199746 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:09.199707    2576 patch_prober.go:28] interesting pod/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Jul 30 04:20:09.199955 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:09.199766    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5" podUID="6dc89457-4084-4353-9043-3bf110a777a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Jul 30 04:20:10.200227 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:10.200187    2576 patch_prober.go:28] interesting pod/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Jul 30 04:20:10.200717 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:10.200249    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5" podUID="6dc89457-4084-4353-9043-3bf110a777a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Jul 30 04:20:11.203488 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:11.203453    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:20:11.203863 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:11.203690    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:20:11.204304 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:11.204289    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:20:14.504043 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:14.504004    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:20:47.244472 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:47.244435    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-8d492b/test-model-8d492b-kserve-5c4bdf86d7-mkmpv"]
Jul 30 04:20:47.246918 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:47.246902    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8d492b/test-model-8d492b-kserve-5c4bdf86d7-mkmpv"
Jul 30 04:20:47.250713 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:47.250689    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-8d492b\"/\"test-model-8d492b-kserve-self-signed-certs\""
Jul 30 04:20:47.250871 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:47.250698    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-8d492b\"/\"openshift-service-ca.crt\""
Jul 30 04:20:47.250871 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:47.250729    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-8d492b\"/\"default-dockercfg-lglz2\""
Jul 30 04:20:47.250871 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:47.250777    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-8d492b\"/\"kube-root-ca.crt\""
Jul 30 04:20:47.255665 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:47.255642    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-8d492b/test-model-8d492b-kserve-5c4bdf86d7-mkmpv"]
Jul 30 04:20:47.261179 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:47.261153    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:20:47.329291 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:47.329251    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g964r\" (UniqueName: \"kubernetes.io/projected/8d515889-4686-4a4f-906d-82ed471b9a62-kube-api-access-g964r\") pod \"test-model-8d492b-kserve-5c4bdf86d7-mkmpv\" (UID: \"8d515889-4686-4a4f-906d-82ed471b9a62\") " pod="ai-tenant-e2e-inf-a-8d492b/test-model-8d492b-kserve-5c4bdf86d7-mkmpv"
Jul 30 04:20:47.329465 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:47.329304    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d515889-4686-4a4f-906d-82ed471b9a62-dshm\") pod \"test-model-8d492b-kserve-5c4bdf86d7-mkmpv\" (UID: \"8d515889-4686-4a4f-906d-82ed471b9a62\") " pod="ai-tenant-e2e-inf-a-8d492b/test-model-8d492b-kserve-5c4bdf86d7-mkmpv"
Jul 30 04:20:47.329465 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:47.329327    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d515889-4686-4a4f-906d-82ed471b9a62-model-cache\") pod \"test-model-8d492b-kserve-5c4bdf86d7-mkmpv\" (UID: \"8d515889-4686-4a4f-906d-82ed471b9a62\") " pod="ai-tenant-e2e-inf-a-8d492b/test-model-8d492b-kserve-5c4bdf86d7-mkmpv"
Jul 30 04:20:47.329465 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:47.329365    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d515889-4686-4a4f-906d-82ed471b9a62-home\") pod \"test-model-8d492b-kserve-5c4bdf86d7-mkmpv\" (UID: \"8d515889-4686-4a4f-906d-82ed471b9a62\") " pod="ai-tenant-e2e-inf-a-8d492b/test-model-8d492b-kserve-5c4bdf86d7-mkmpv"
Jul 30 04:20:47.329465 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:47.329384    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d515889-4686-4a4f-906d-82ed471b9a62-tls-certs\") pod \"test-model-8d492b-kserve-5c4bdf86d7-mkmpv\" (UID: \"8d515889-4686-4a4f-906d-82ed471b9a62\") " pod="ai-tenant-e2e-inf-a-8d492b/test-model-8d492b-kserve-5c4bdf86d7-mkmpv"
Jul 30 04:20:47.329465 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:47.329400    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d515889-4686-4a4f-906d-82ed471b9a62-tmp-dir\") pod \"test-model-8d492b-kserve-5c4bdf86d7-mkmpv\" (UID: \"8d515889-4686-4a4f-906d-82ed471b9a62\") " pod="ai-tenant-e2e-inf-a-8d492b/test-model-8d492b-kserve-5c4bdf86d7-mkmpv"
Jul 30 04:20:47.430259 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:47.430227    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g964r\" (UniqueName: \"kubernetes.io/projected/8d515889-4686-4a4f-906d-82ed471b9a62-kube-api-access-g964r\") pod \"test-model-8d492b-kserve-5c4bdf86d7-mkmpv\" (UID: \"8d515889-4686-4a4f-906d-82ed471b9a62\") " pod="ai-tenant-e2e-inf-a-8d492b/test-model-8d492b-kserve-5c4bdf86d7-mkmpv"
Jul 30 04:20:47.430505 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:47.430281    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d515889-4686-4a4f-906d-82ed471b9a62-dshm\") pod \"test-model-8d492b-kserve-5c4bdf86d7-mkmpv\" (UID: \"8d515889-4686-4a4f-906d-82ed471b9a62\") " pod="ai-tenant-e2e-inf-a-8d492b/test-model-8d492b-kserve-5c4bdf86d7-mkmpv"
Jul 30 04:20:47.430505 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:47.430307    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d515889-4686-4a4f-906d-82ed471b9a62-model-cache\") pod \"test-model-8d492b-kserve-5c4bdf86d7-mkmpv\" (UID: \"8d515889-4686-4a4f-906d-82ed471b9a62\") " pod="ai-tenant-e2e-inf-a-8d492b/test-model-8d492b-kserve-5c4bdf86d7-mkmpv"
Jul 30 04:20:47.430505 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:47.430329    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d515889-4686-4a4f-906d-82ed471b9a62-home\") pod \"test-model-8d492b-kserve-5c4bdf86d7-mkmpv\" (UID: \"8d515889-4686-4a4f-906d-82ed471b9a62\") " pod="ai-tenant-e2e-inf-a-8d492b/test-model-8d492b-kserve-5c4bdf86d7-mkmpv"
Jul 30 04:20:47.430505 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:47.430347    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d515889-4686-4a4f-906d-82ed471b9a62-tls-certs\") pod \"test-model-8d492b-kserve-5c4bdf86d7-mkmpv\" (UID: \"8d515889-4686-4a4f-906d-82ed471b9a62\") " pod="ai-tenant-e2e-inf-a-8d492b/test-model-8d492b-kserve-5c4bdf86d7-mkmpv"
Jul 30 04:20:47.430505 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:47.430365    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d515889-4686-4a4f-906d-82ed471b9a62-tmp-dir\") pod \"test-model-8d492b-kserve-5c4bdf86d7-mkmpv\" (UID: \"8d515889-4686-4a4f-906d-82ed471b9a62\") " pod="ai-tenant-e2e-inf-a-8d492b/test-model-8d492b-kserve-5c4bdf86d7-mkmpv"
Jul 30 04:20:47.430871 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:47.430791    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d515889-4686-4a4f-906d-82ed471b9a62-model-cache\") pod \"test-model-8d492b-kserve-5c4bdf86d7-mkmpv\" (UID: \"8d515889-4686-4a4f-906d-82ed471b9a62\") " pod="ai-tenant-e2e-inf-a-8d492b/test-model-8d492b-kserve-5c4bdf86d7-mkmpv"
Jul 30 04:20:47.430871 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:47.430796    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d515889-4686-4a4f-906d-82ed471b9a62-home\") pod \"test-model-8d492b-kserve-5c4bdf86d7-mkmpv\" (UID: \"8d515889-4686-4a4f-906d-82ed471b9a62\") " pod="ai-tenant-e2e-inf-a-8d492b/test-model-8d492b-kserve-5c4bdf86d7-mkmpv"
Jul 30 04:20:47.430871 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:47.430852    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d515889-4686-4a4f-906d-82ed471b9a62-tmp-dir\") pod \"test-model-8d492b-kserve-5c4bdf86d7-mkmpv\" (UID: \"8d515889-4686-4a4f-906d-82ed471b9a62\") " pod="ai-tenant-e2e-inf-a-8d492b/test-model-8d492b-kserve-5c4bdf86d7-mkmpv"
Jul 30 04:20:47.432609 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:47.432580    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d515889-4686-4a4f-906d-82ed471b9a62-dshm\") pod \"test-model-8d492b-kserve-5c4bdf86d7-mkmpv\" (UID: \"8d515889-4686-4a4f-906d-82ed471b9a62\") " pod="ai-tenant-e2e-inf-a-8d492b/test-model-8d492b-kserve-5c4bdf86d7-mkmpv"
Jul 30 04:20:47.432759 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:47.432739    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d515889-4686-4a4f-906d-82ed471b9a62-tls-certs\") pod \"test-model-8d492b-kserve-5c4bdf86d7-mkmpv\" (UID: \"8d515889-4686-4a4f-906d-82ed471b9a62\") " pod="ai-tenant-e2e-inf-a-8d492b/test-model-8d492b-kserve-5c4bdf86d7-mkmpv"
Jul 30 04:20:47.438239 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:47.438218    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g964r\" (UniqueName: \"kubernetes.io/projected/8d515889-4686-4a4f-906d-82ed471b9a62-kube-api-access-g964r\") pod \"test-model-8d492b-kserve-5c4bdf86d7-mkmpv\" (UID: \"8d515889-4686-4a4f-906d-82ed471b9a62\") " pod="ai-tenant-e2e-inf-a-8d492b/test-model-8d492b-kserve-5c4bdf86d7-mkmpv"
Jul 30 04:20:47.557004 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:47.556976    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8d492b/test-model-8d492b-kserve-5c4bdf86d7-mkmpv"
Jul 30 04:20:47.693100 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:47.693071    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-8d492b/test-model-8d492b-kserve-5c4bdf86d7-mkmpv"]
Jul 30 04:20:47.694646 ip-10-0-143-117 kubenswrapper[2576]: W0730 04:20:47.694620    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8d515889_4686_4a4f_906d_82ed471b9a62.slice/crio-126922f68a91ab305607737265f05b02a234214ac40101ceb66b167b1cf3a3fb WatchSource:0}: Error finding container 126922f68a91ab305607737265f05b02a234214ac40101ceb66b167b1cf3a3fb: Status 404 returned error can't find the container with id 126922f68a91ab305607737265f05b02a234214ac40101ceb66b167b1cf3a3fb
Jul 30 04:20:47.928632 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:47.928539    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8d492b/test-model-8d492b-kserve-5c4bdf86d7-mkmpv" event={"ID":"8d515889-4686-4a4f-906d-82ed471b9a62","Type":"ContainerStarted","Data":"394c3fa1ec764043d41e29f8938680598634e6af937ed851573cb3fc74f37d5e"}
Jul 30 04:20:47.928632 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:47.928586    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8d492b/test-model-8d492b-kserve-5c4bdf86d7-mkmpv" event={"ID":"8d515889-4686-4a4f-906d-82ed471b9a62","Type":"ContainerStarted","Data":"126922f68a91ab305607737265f05b02a234214ac40101ceb66b167b1cf3a3fb"}
Jul 30 04:20:47.947186 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:47.947141    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-8d492b/test-model-8d492b-kserve-5c4bdf86d7-mkmpv" podStartSLOduration=0.947125524 podStartE2EDuration="947.125524ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:20:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:20:47.945121113 +0000 UTC m=+2753.563353355" watchObservedRunningTime="2026-07-30 04:20:47.947125524 +0000 UTC m=+2753.565357766"
Jul 30 04:20:49.531721 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:49.531689    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:20:57.558150 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:57.558099    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-8d492b/test-model-8d492b-kserve-5c4bdf86d7-mkmpv"
Jul 30 04:20:57.558562 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:57.558163    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-8d492b/test-model-8d492b-kserve-5c4bdf86d7-mkmpv"
Jul 30 04:20:57.571145 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:57.571117    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-8d492b/test-model-8d492b-kserve-5c4bdf86d7-mkmpv"
Jul 30 04:20:57.994100 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:20:57.994014    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-8d492b/test-model-8d492b-kserve-5c4bdf86d7-mkmpv"
Jul 30 04:21:03.232896 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:21:03.232857    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:21:18.545442 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:21:18.545356    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:21:22.837601 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:21:22.837557    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:21:37.734631 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:21:37.734589    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:22:14.879106 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:14.879067    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-8d492b/test-model-8d492b-kserve-5c4bdf86d7-mkmpv"]
Jul 30 04:22:14.902660 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:14.902628    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:16.019218 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:16.019170    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:22.534875 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:22.534839    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:22:26.820910 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:26.820878    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:26.843231 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:26.843196    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-8d492b-7464844d44-dgdqf"]
Jul 30 04:22:26.843511 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:26.843478    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8d492b-7464844d44-dgdqf" podUID="7b6bc68a-c396-44b8-b562-704153888052" containerName="payload-pre-processing" containerID="cri-o://4505b199fa8c5befdaaa908453ea7db38f2f4a6369b1c1b8c51a0fda44d1baf3" gracePeriod=30
Jul 30 04:22:26.882675 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:26.882633    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:26.984128 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:26.984020    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:27.083847 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:27.083774    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:27.104066 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:27.104041    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8d492b-7464844d44-dgdqf"
Jul 30 04:22:27.182096 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:27.182063    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zqjp6\" (UniqueName: \"kubernetes.io/projected/7b6bc68a-c396-44b8-b562-704153888052-kube-api-access-zqjp6\") pod \"7b6bc68a-c396-44b8-b562-704153888052\" (UID: \"7b6bc68a-c396-44b8-b562-704153888052\") "
Jul 30 04:22:27.182274 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:27.182121    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b6bc68a-c396-44b8-b562-704153888052-plugins-config-volume\") pod \"7b6bc68a-c396-44b8-b562-704153888052\" (UID: \"7b6bc68a-c396-44b8-b562-704153888052\") "
Jul 30 04:22:27.182493 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:27.182473    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b6bc68a-c396-44b8-b562-704153888052-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7b6bc68a-c396-44b8-b562-704153888052" (UID: "7b6bc68a-c396-44b8-b562-704153888052"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:22:27.184252 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:27.184227    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b6bc68a-c396-44b8-b562-704153888052-kube-api-access-zqjp6" (OuterVolumeSpecName: "kube-api-access-zqjp6") pod "7b6bc68a-c396-44b8-b562-704153888052" (UID: "7b6bc68a-c396-44b8-b562-704153888052"). InnerVolumeSpecName "kube-api-access-zqjp6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:22:27.283952 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:27.283884    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqjp6\" (UniqueName: \"kubernetes.io/projected/7b6bc68a-c396-44b8-b562-704153888052-kube-api-access-zqjp6\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:22:27.283952 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:27.283916    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b6bc68a-c396-44b8-b562-704153888052-plugins-config-volume\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:22:27.352386 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:27.352290    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:27.454085 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:27.454048    2576 generic.go:356] "Generic (PLEG): container finished" podID="7b6bc68a-c396-44b8-b562-704153888052" containerID="4505b199fa8c5befdaaa908453ea7db38f2f4a6369b1c1b8c51a0fda44d1baf3" exitCode=0
Jul 30 04:22:27.454260 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:27.454093    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8d492b-7464844d44-dgdqf"
Jul 30 04:22:27.454260 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:27.454136    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8d492b-7464844d44-dgdqf" event={"ID":"7b6bc68a-c396-44b8-b562-704153888052","Type":"ContainerDied","Data":"4505b199fa8c5befdaaa908453ea7db38f2f4a6369b1c1b8c51a0fda44d1baf3"}
Jul 30 04:22:27.454260 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:27.454192    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8d492b-7464844d44-dgdqf" event={"ID":"7b6bc68a-c396-44b8-b562-704153888052","Type":"ContainerDied","Data":"e0def3739b1bf116dd25562a604b361a0938ec3afa6962f594b53377e893bd16"}
Jul 30 04:22:27.454260 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:27.454218    2576 scope.go:117] "RemoveContainer" containerID="4505b199fa8c5befdaaa908453ea7db38f2f4a6369b1c1b8c51a0fda44d1baf3"
Jul 30 04:22:27.466249 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:27.466228    2576 scope.go:117] "RemoveContainer" containerID="4505b199fa8c5befdaaa908453ea7db38f2f4a6369b1c1b8c51a0fda44d1baf3"
Jul 30 04:22:27.466529 ip-10-0-143-117 kubenswrapper[2576]: E0730 04:22:27.466512    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4505b199fa8c5befdaaa908453ea7db38f2f4a6369b1c1b8c51a0fda44d1baf3\": container with ID starting with 4505b199fa8c5befdaaa908453ea7db38f2f4a6369b1c1b8c51a0fda44d1baf3 not found: ID does not exist" containerID="4505b199fa8c5befdaaa908453ea7db38f2f4a6369b1c1b8c51a0fda44d1baf3"
Jul 30 04:22:27.466568 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:27.466539    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4505b199fa8c5befdaaa908453ea7db38f2f4a6369b1c1b8c51a0fda44d1baf3"} err="failed to get container status \"4505b199fa8c5befdaaa908453ea7db38f2f4a6369b1c1b8c51a0fda44d1baf3\": rpc error: code = NotFound desc = could not find container \"4505b199fa8c5befdaaa908453ea7db38f2f4a6369b1c1b8c51a0fda44d1baf3\": container with ID starting with 4505b199fa8c5befdaaa908453ea7db38f2f4a6369b1c1b8c51a0fda44d1baf3 not found: ID does not exist"
Jul 30 04:22:27.478708 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:27.478682    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-8d492b-7464844d44-dgdqf"]
Jul 30 04:22:27.482590 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:27.482569    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-8d492b-7464844d44-dgdqf"]
Jul 30 04:22:29.051149 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:29.051115    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b6bc68a-c396-44b8-b562-704153888052" path="/var/lib/kubelet/pods/7b6bc68a-c396-44b8-b562-704153888052/volumes"
Jul 30 04:22:29.736907 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:29.736866    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:29.898667 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:29.898616    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-8d492b/test-model-8d492b-kserve-5c4bdf86d7-mkmpv" podUID="8d515889-4686-4a4f-906d-82ed471b9a62" containerName="main" containerID="cri-o://394c3fa1ec764043d41e29f8938680598634e6af937ed851573cb3fc74f37d5e" gracePeriod=45
Jul 30 04:22:30.145610 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:30.145584    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8d492b/test-model-8d492b-kserve-5c4bdf86d7-mkmpv"
Jul 30 04:22:30.215770 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:30.215739    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d515889-4686-4a4f-906d-82ed471b9a62-dshm\") pod \"8d515889-4686-4a4f-906d-82ed471b9a62\" (UID: \"8d515889-4686-4a4f-906d-82ed471b9a62\") "
Jul 30 04:22:30.215981 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:30.215805    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d515889-4686-4a4f-906d-82ed471b9a62-model-cache\") pod \"8d515889-4686-4a4f-906d-82ed471b9a62\" (UID: \"8d515889-4686-4a4f-906d-82ed471b9a62\") "
Jul 30 04:22:30.215981 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:30.215830    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d515889-4686-4a4f-906d-82ed471b9a62-tmp-dir\") pod \"8d515889-4686-4a4f-906d-82ed471b9a62\" (UID: \"8d515889-4686-4a4f-906d-82ed471b9a62\") "
Jul 30 04:22:30.215981 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:30.215972    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d515889-4686-4a4f-906d-82ed471b9a62-tls-certs\") pod \"8d515889-4686-4a4f-906d-82ed471b9a62\" (UID: \"8d515889-4686-4a4f-906d-82ed471b9a62\") "
Jul 30 04:22:30.216175 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:30.215998    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d515889-4686-4a4f-906d-82ed471b9a62-home\") pod \"8d515889-4686-4a4f-906d-82ed471b9a62\" (UID: \"8d515889-4686-4a4f-906d-82ed471b9a62\") "
Jul 30 04:22:30.216175 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:30.216022    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g964r\" (UniqueName: \"kubernetes.io/projected/8d515889-4686-4a4f-906d-82ed471b9a62-kube-api-access-g964r\") pod \"8d515889-4686-4a4f-906d-82ed471b9a62\" (UID: \"8d515889-4686-4a4f-906d-82ed471b9a62\") "
Jul 30 04:22:30.216175 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:30.216043    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d515889-4686-4a4f-906d-82ed471b9a62-model-cache" (OuterVolumeSpecName: "model-cache") pod "8d515889-4686-4a4f-906d-82ed471b9a62" (UID: "8d515889-4686-4a4f-906d-82ed471b9a62"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:22:30.216175 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:30.216140    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d515889-4686-4a4f-906d-82ed471b9a62-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8d515889-4686-4a4f-906d-82ed471b9a62" (UID: "8d515889-4686-4a4f-906d-82ed471b9a62"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:22:30.216365 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:30.216254    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d515889-4686-4a4f-906d-82ed471b9a62-home" (OuterVolumeSpecName: "home") pod "8d515889-4686-4a4f-906d-82ed471b9a62" (UID: "8d515889-4686-4a4f-906d-82ed471b9a62"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:22:30.216456 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:30.216439    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d515889-4686-4a4f-906d-82ed471b9a62-home\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:22:30.216514 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:30.216461    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d515889-4686-4a4f-906d-82ed471b9a62-model-cache\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:22:30.216514 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:30.216477    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d515889-4686-4a4f-906d-82ed471b9a62-tmp-dir\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:22:30.218066 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:30.218043    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d515889-4686-4a4f-906d-82ed471b9a62-dshm" (OuterVolumeSpecName: "dshm") pod "8d515889-4686-4a4f-906d-82ed471b9a62" (UID: "8d515889-4686-4a4f-906d-82ed471b9a62"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:22:30.218170 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:30.218090    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d515889-4686-4a4f-906d-82ed471b9a62-kube-api-access-g964r" (OuterVolumeSpecName: "kube-api-access-g964r") pod "8d515889-4686-4a4f-906d-82ed471b9a62" (UID: "8d515889-4686-4a4f-906d-82ed471b9a62"). InnerVolumeSpecName "kube-api-access-g964r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:22:30.218170 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:30.218119    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d515889-4686-4a4f-906d-82ed471b9a62-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8d515889-4686-4a4f-906d-82ed471b9a62" (UID: "8d515889-4686-4a4f-906d-82ed471b9a62"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 04:22:30.317098 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:30.317065    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d515889-4686-4a4f-906d-82ed471b9a62-tls-certs\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:22:30.317098 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:30.317096    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g964r\" (UniqueName: \"kubernetes.io/projected/8d515889-4686-4a4f-906d-82ed471b9a62-kube-api-access-g964r\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:22:30.317098 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:30.317107    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d515889-4686-4a4f-906d-82ed471b9a62-dshm\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:22:30.472803 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:30.472769    2576 generic.go:356] "Generic (PLEG): container finished" podID="8d515889-4686-4a4f-906d-82ed471b9a62" containerID="394c3fa1ec764043d41e29f8938680598634e6af937ed851573cb3fc74f37d5e" exitCode=0
Jul 30 04:22:30.472994 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:30.472811    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8d492b/test-model-8d492b-kserve-5c4bdf86d7-mkmpv"
Jul 30 04:22:30.472994 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:30.472846    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8d492b/test-model-8d492b-kserve-5c4bdf86d7-mkmpv" event={"ID":"8d515889-4686-4a4f-906d-82ed471b9a62","Type":"ContainerDied","Data":"394c3fa1ec764043d41e29f8938680598634e6af937ed851573cb3fc74f37d5e"}
Jul 30 04:22:30.472994 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:30.472886    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8d492b/test-model-8d492b-kserve-5c4bdf86d7-mkmpv" event={"ID":"8d515889-4686-4a4f-906d-82ed471b9a62","Type":"ContainerDied","Data":"126922f68a91ab305607737265f05b02a234214ac40101ceb66b167b1cf3a3fb"}
Jul 30 04:22:30.472994 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:30.472903    2576 scope.go:117] "RemoveContainer" containerID="394c3fa1ec764043d41e29f8938680598634e6af937ed851573cb3fc74f37d5e"
Jul 30 04:22:30.482654 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:30.482630    2576 scope.go:117] "RemoveContainer" containerID="394c3fa1ec764043d41e29f8938680598634e6af937ed851573cb3fc74f37d5e"
Jul 30 04:22:30.482955 ip-10-0-143-117 kubenswrapper[2576]: E0730 04:22:30.482934    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"394c3fa1ec764043d41e29f8938680598634e6af937ed851573cb3fc74f37d5e\": container with ID starting with 394c3fa1ec764043d41e29f8938680598634e6af937ed851573cb3fc74f37d5e not found: ID does not exist" containerID="394c3fa1ec764043d41e29f8938680598634e6af937ed851573cb3fc74f37d5e"
Jul 30 04:22:30.483018 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:30.482967    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"394c3fa1ec764043d41e29f8938680598634e6af937ed851573cb3fc74f37d5e"} err="failed to get container status \"394c3fa1ec764043d41e29f8938680598634e6af937ed851573cb3fc74f37d5e\": rpc error: code = NotFound desc = could not find container \"394c3fa1ec764043d41e29f8938680598634e6af937ed851573cb3fc74f37d5e\": container with ID starting with 394c3fa1ec764043d41e29f8938680598634e6af937ed851573cb3fc74f37d5e not found: ID does not exist"
Jul 30 04:22:30.494726 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:30.494692    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-8d492b/test-model-8d492b-kserve-5c4bdf86d7-mkmpv"]
Jul 30 04:22:30.498062 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:30.498036    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-8d492b/test-model-8d492b-kserve-5c4bdf86d7-mkmpv"]
Jul 30 04:22:31.051590 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:31.051559    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d515889-4686-4a4f-906d-82ed471b9a62" path="/var/lib/kubelet/pods/8d515889-4686-4a4f-906d-82ed471b9a62/volumes"
Jul 30 04:22:32.333014 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:32.331116    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:22:33.170288 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:33.170247    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:33.231465 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:33.231431    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:33.421314 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:33.421230    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"]
Jul 30 04:22:33.421705 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:33.421446    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn" podUID="5732b28f-b1be-4de2-8708-a535a8d559e7" containerName="istio-proxy" containerID="cri-o://fdd52ad1fcf92153830f9c974886786e7b440f8e628937b4a487f8bb1160544b" gracePeriod=30
Jul 30 04:22:33.433588 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:33.433552    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:33.647528 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:33.647484    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:38.675937 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:38.675905    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:22:38.796752 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:38.796715    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5732b28f-b1be-4de2-8708-a535a8d559e7-workload-certs\") pod \"5732b28f-b1be-4de2-8708-a535a8d559e7\" (UID: \"5732b28f-b1be-4de2-8708-a535a8d559e7\") "
Jul 30 04:22:38.796955 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:38.796771    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jg2rv\" (UniqueName: \"kubernetes.io/projected/5732b28f-b1be-4de2-8708-a535a8d559e7-kube-api-access-jg2rv\") pod \"5732b28f-b1be-4de2-8708-a535a8d559e7\" (UID: \"5732b28f-b1be-4de2-8708-a535a8d559e7\") "
Jul 30 04:22:38.796955 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:38.796808    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5732b28f-b1be-4de2-8708-a535a8d559e7-istiod-ca-cert\") pod \"5732b28f-b1be-4de2-8708-a535a8d559e7\" (UID: \"5732b28f-b1be-4de2-8708-a535a8d559e7\") "
Jul 30 04:22:38.796955 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:38.796855    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5732b28f-b1be-4de2-8708-a535a8d559e7-istio-podinfo\") pod \"5732b28f-b1be-4de2-8708-a535a8d559e7\" (UID: \"5732b28f-b1be-4de2-8708-a535a8d559e7\") "
Jul 30 04:22:38.796955 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:38.796905    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5732b28f-b1be-4de2-8708-a535a8d559e7-istio-envoy\") pod \"5732b28f-b1be-4de2-8708-a535a8d559e7\" (UID: \"5732b28f-b1be-4de2-8708-a535a8d559e7\") "
Jul 30 04:22:38.797153 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:38.796970    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5732b28f-b1be-4de2-8708-a535a8d559e7-istio-data\") pod \"5732b28f-b1be-4de2-8708-a535a8d559e7\" (UID: \"5732b28f-b1be-4de2-8708-a535a8d559e7\") "
Jul 30 04:22:38.797153 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:38.797012    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5732b28f-b1be-4de2-8708-a535a8d559e7-workload-socket\") pod \"5732b28f-b1be-4de2-8708-a535a8d559e7\" (UID: \"5732b28f-b1be-4de2-8708-a535a8d559e7\") "
Jul 30 04:22:38.797153 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:38.797086    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5732b28f-b1be-4de2-8708-a535a8d559e7-istio-token\") pod \"5732b28f-b1be-4de2-8708-a535a8d559e7\" (UID: \"5732b28f-b1be-4de2-8708-a535a8d559e7\") "
Jul 30 04:22:38.797153 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:38.797097    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5732b28f-b1be-4de2-8708-a535a8d559e7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5732b28f-b1be-4de2-8708-a535a8d559e7" (UID: "5732b28f-b1be-4de2-8708-a535a8d559e7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:22:38.797153 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:38.797139    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5732b28f-b1be-4de2-8708-a535a8d559e7-credential-socket\") pod \"5732b28f-b1be-4de2-8708-a535a8d559e7\" (UID: \"5732b28f-b1be-4de2-8708-a535a8d559e7\") "
Jul 30 04:22:38.797423 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:38.797335    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5732b28f-b1be-4de2-8708-a535a8d559e7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5732b28f-b1be-4de2-8708-a535a8d559e7" (UID: "5732b28f-b1be-4de2-8708-a535a8d559e7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:22:38.797423 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:38.797391    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5732b28f-b1be-4de2-8708-a535a8d559e7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5732b28f-b1be-4de2-8708-a535a8d559e7" (UID: "5732b28f-b1be-4de2-8708-a535a8d559e7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:22:38.797524 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:38.797510    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5732b28f-b1be-4de2-8708-a535a8d559e7-workload-certs\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:22:38.797571 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:38.797530    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5732b28f-b1be-4de2-8708-a535a8d559e7-istiod-ca-cert\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:22:38.797571 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:38.797529    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5732b28f-b1be-4de2-8708-a535a8d559e7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5732b28f-b1be-4de2-8708-a535a8d559e7" (UID: "5732b28f-b1be-4de2-8708-a535a8d559e7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:22:38.797571 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:38.797541    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5732b28f-b1be-4de2-8708-a535a8d559e7-workload-socket\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:22:38.798162 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:38.798102    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5732b28f-b1be-4de2-8708-a535a8d559e7-istio-data" (OuterVolumeSpecName: "istio-data") pod "5732b28f-b1be-4de2-8708-a535a8d559e7" (UID: "5732b28f-b1be-4de2-8708-a535a8d559e7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:22:38.799441 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:38.799332    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5732b28f-b1be-4de2-8708-a535a8d559e7-kube-api-access-jg2rv" (OuterVolumeSpecName: "kube-api-access-jg2rv") pod "5732b28f-b1be-4de2-8708-a535a8d559e7" (UID: "5732b28f-b1be-4de2-8708-a535a8d559e7"). InnerVolumeSpecName "kube-api-access-jg2rv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:22:38.799571 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:38.799472    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5732b28f-b1be-4de2-8708-a535a8d559e7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5732b28f-b1be-4de2-8708-a535a8d559e7" (UID: "5732b28f-b1be-4de2-8708-a535a8d559e7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 04:22:38.799981 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:38.799953    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5732b28f-b1be-4de2-8708-a535a8d559e7-istio-token" (OuterVolumeSpecName: "istio-token") pod "5732b28f-b1be-4de2-8708-a535a8d559e7" (UID: "5732b28f-b1be-4de2-8708-a535a8d559e7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:22:38.801026 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:38.801002    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5732b28f-b1be-4de2-8708-a535a8d559e7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5732b28f-b1be-4de2-8708-a535a8d559e7" (UID: "5732b28f-b1be-4de2-8708-a535a8d559e7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:22:38.898162 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:38.898128    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jg2rv\" (UniqueName: \"kubernetes.io/projected/5732b28f-b1be-4de2-8708-a535a8d559e7-kube-api-access-jg2rv\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:22:38.898162 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:38.898156    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5732b28f-b1be-4de2-8708-a535a8d559e7-istio-podinfo\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:22:38.898162 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:38.898166    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5732b28f-b1be-4de2-8708-a535a8d559e7-istio-envoy\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:22:38.898390 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:38.898174    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5732b28f-b1be-4de2-8708-a535a8d559e7-istio-data\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:22:38.898390 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:38.898181    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5732b28f-b1be-4de2-8708-a535a8d559e7-istio-token\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:22:38.898390 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:38.898189    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5732b28f-b1be-4de2-8708-a535a8d559e7-credential-socket\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:22:39.526067 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:39.526036    2576 generic.go:356] "Generic (PLEG): container finished" podID="5732b28f-b1be-4de2-8708-a535a8d559e7" containerID="fdd52ad1fcf92153830f9c974886786e7b440f8e628937b4a487f8bb1160544b" exitCode=0
Jul 30 04:22:39.526233 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:39.526098    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"
Jul 30 04:22:39.526233 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:39.526118    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn" event={"ID":"5732b28f-b1be-4de2-8708-a535a8d559e7","Type":"ContainerDied","Data":"fdd52ad1fcf92153830f9c974886786e7b440f8e628937b4a487f8bb1160544b"}
Jul 30 04:22:39.526233 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:39.526171    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn" event={"ID":"5732b28f-b1be-4de2-8708-a535a8d559e7","Type":"ContainerDied","Data":"229edc00113ab561291d9544353db4f5e5b1deb0f81853e600d78a957963aec1"}
Jul 30 04:22:39.526233 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:39.526195    2576 scope.go:117] "RemoveContainer" containerID="fdd52ad1fcf92153830f9c974886786e7b440f8e628937b4a487f8bb1160544b"
Jul 30 04:22:39.535343 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:39.535331    2576 scope.go:117] "RemoveContainer" containerID="8f11082eaf7df52abe847e02052a0717c8e3792a1d3ce9e646c52448b8f16546"
Jul 30 04:22:39.543402 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:39.543244    2576 scope.go:117] "RemoveContainer" containerID="fdd52ad1fcf92153830f9c974886786e7b440f8e628937b4a487f8bb1160544b"
Jul 30 04:22:39.543522 ip-10-0-143-117 kubenswrapper[2576]: E0730 04:22:39.543504    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdd52ad1fcf92153830f9c974886786e7b440f8e628937b4a487f8bb1160544b\": container with ID starting with fdd52ad1fcf92153830f9c974886786e7b440f8e628937b4a487f8bb1160544b not found: ID does not exist" containerID="fdd52ad1fcf92153830f9c974886786e7b440f8e628937b4a487f8bb1160544b"
Jul 30 04:22:39.543570 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:39.543531    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdd52ad1fcf92153830f9c974886786e7b440f8e628937b4a487f8bb1160544b"} err="failed to get container status \"fdd52ad1fcf92153830f9c974886786e7b440f8e628937b4a487f8bb1160544b\": rpc error: code = NotFound desc = could not find container \"fdd52ad1fcf92153830f9c974886786e7b440f8e628937b4a487f8bb1160544b\": container with ID starting with fdd52ad1fcf92153830f9c974886786e7b440f8e628937b4a487f8bb1160544b not found: ID does not exist"
Jul 30 04:22:39.543570 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:39.543549    2576 scope.go:117] "RemoveContainer" containerID="8f11082eaf7df52abe847e02052a0717c8e3792a1d3ce9e646c52448b8f16546"
Jul 30 04:22:39.543788 ip-10-0-143-117 kubenswrapper[2576]: E0730 04:22:39.543772    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f11082eaf7df52abe847e02052a0717c8e3792a1d3ce9e646c52448b8f16546\": container with ID starting with 8f11082eaf7df52abe847e02052a0717c8e3792a1d3ce9e646c52448b8f16546 not found: ID does not exist" containerID="8f11082eaf7df52abe847e02052a0717c8e3792a1d3ce9e646c52448b8f16546"
Jul 30 04:22:39.543836 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:39.543794    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f11082eaf7df52abe847e02052a0717c8e3792a1d3ce9e646c52448b8f16546"} err="failed to get container status \"8f11082eaf7df52abe847e02052a0717c8e3792a1d3ce9e646c52448b8f16546\": rpc error: code = NotFound desc = could not find container \"8f11082eaf7df52abe847e02052a0717c8e3792a1d3ce9e646c52448b8f16546\": container with ID starting with 8f11082eaf7df52abe847e02052a0717c8e3792a1d3ce9e646c52448b8f16546 not found: ID does not exist"
Jul 30 04:22:39.553173 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:39.553150    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"]
Jul 30 04:22:39.558642 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:39.558621    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-8d492b-openshift-default-c97cd9968-x49qn"]
Jul 30 04:22:41.053367 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:41.053331    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5732b28f-b1be-4de2-8708-a535a8d559e7" path="/var/lib/kubelet/pods/5732b28f-b1be-4de2-8708-a535a8d559e7/volumes"
Jul 30 04:22:44.259349 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:44.259308    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:44.322873 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:44.322834    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:44.388156 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:44.388119    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:44.448098 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:44.448057    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:44.878696 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:44.878655    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:47.133026 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:47.132951    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:22:50.594003 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:50.593968    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:50.665241 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:50.665210    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:50.824284 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:50.824253    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"]
Jul 30 04:22:50.824482 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:50.824460    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5" podUID="6dc89457-4084-4353-9043-3bf110a777a5" containerName="istio-proxy" containerID="cri-o://4aa058c21255aa3841a0d5f5ee3f7c307c7e108eaa071697a7b1fb11f3b57a91" gracePeriod=30
Jul 30 04:22:50.830994 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:50.830955    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:51.051041 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.051003    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:51.381139 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.381057    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:51.653520 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.653396    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"]
Jul 30 04:22:51.653951 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.653867    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d515889-4686-4a4f-906d-82ed471b9a62" containerName="main"
Jul 30 04:22:51.653951 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.653882    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d515889-4686-4a4f-906d-82ed471b9a62" containerName="main"
Jul 30 04:22:51.653951 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.653903    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5732b28f-b1be-4de2-8708-a535a8d559e7" containerName="istio-proxy"
Jul 30 04:22:51.653951 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.653908    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5732b28f-b1be-4de2-8708-a535a8d559e7" containerName="istio-proxy"
Jul 30 04:22:51.653951 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.653914    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b6bc68a-c396-44b8-b562-704153888052" containerName="payload-pre-processing"
Jul 30 04:22:51.653951 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.653934    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b6bc68a-c396-44b8-b562-704153888052" containerName="payload-pre-processing"
Jul 30 04:22:51.653951 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.653952    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5732b28f-b1be-4de2-8708-a535a8d559e7" containerName="istio-proxy"
Jul 30 04:22:51.654209 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.653958    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5732b28f-b1be-4de2-8708-a535a8d559e7" containerName="istio-proxy"
Jul 30 04:22:51.654209 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.654024    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5732b28f-b1be-4de2-8708-a535a8d559e7" containerName="istio-proxy"
Jul 30 04:22:51.654209 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.654033    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d515889-4686-4a4f-906d-82ed471b9a62" containerName="main"
Jul 30 04:22:51.654209 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.654051    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b6bc68a-c396-44b8-b562-704153888052" containerName="payload-pre-processing"
Jul 30 04:22:51.658893 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.658875    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:22:51.676342 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.676316    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:51.682500 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.682480    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-9c7b98-openshift-default-dockercfg-sslgf\""
Jul 30 04:22:51.685476 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.685442    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"]
Jul 30 04:22:51.823213 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.823175    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07f7c7ec-b1b6-4a89-a761-08c3af23436c-istiod-ca-cert\") pod \"e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn\" (UID: \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\") " pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:22:51.823401 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.823228    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f7c7ec-b1b6-4a89-a761-08c3af23436c-credential-socket\") pod \"e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn\" (UID: \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\") " pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:22:51.823401 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.823306    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07f7c7ec-b1b6-4a89-a761-08c3af23436c-istio-token\") pod \"e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn\" (UID: \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\") " pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:22:51.823401 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.823362    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f7c7ec-b1b6-4a89-a761-08c3af23436c-workload-socket\") pod \"e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn\" (UID: \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\") " pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:22:51.823552 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.823406    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwczm\" (UniqueName: \"kubernetes.io/projected/07f7c7ec-b1b6-4a89-a761-08c3af23436c-kube-api-access-bwczm\") pod \"e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn\" (UID: \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\") " pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:22:51.823552 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.823450    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07f7c7ec-b1b6-4a89-a761-08c3af23436c-istio-podinfo\") pod \"e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn\" (UID: \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\") " pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:22:51.823552 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.823479    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07f7c7ec-b1b6-4a89-a761-08c3af23436c-istio-envoy\") pod \"e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn\" (UID: \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\") " pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:22:51.823552 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.823501    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07f7c7ec-b1b6-4a89-a761-08c3af23436c-workload-certs\") pod \"e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn\" (UID: \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\") " pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:22:51.823552 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.823519    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07f7c7ec-b1b6-4a89-a761-08c3af23436c-istio-data\") pod \"e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn\" (UID: \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\") " pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:22:51.924197 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.924104    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f7c7ec-b1b6-4a89-a761-08c3af23436c-workload-socket\") pod \"e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn\" (UID: \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\") " pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:22:51.924197 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.924154    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwczm\" (UniqueName: \"kubernetes.io/projected/07f7c7ec-b1b6-4a89-a761-08c3af23436c-kube-api-access-bwczm\") pod \"e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn\" (UID: \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\") " pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:22:51.924425 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.924203    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07f7c7ec-b1b6-4a89-a761-08c3af23436c-istio-podinfo\") pod \"e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn\" (UID: \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\") " pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:22:51.924425 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.924240    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07f7c7ec-b1b6-4a89-a761-08c3af23436c-istio-envoy\") pod \"e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn\" (UID: \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\") " pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:22:51.924425 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.924370    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07f7c7ec-b1b6-4a89-a761-08c3af23436c-workload-certs\") pod \"e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn\" (UID: \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\") " pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:22:51.924425 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.924420    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07f7c7ec-b1b6-4a89-a761-08c3af23436c-istio-data\") pod \"e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn\" (UID: \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\") " pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:22:51.924630 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.924462    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07f7c7ec-b1b6-4a89-a761-08c3af23436c-istiod-ca-cert\") pod \"e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn\" (UID: \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\") " pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:22:51.924630 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.924514    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f7c7ec-b1b6-4a89-a761-08c3af23436c-credential-socket\") pod \"e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn\" (UID: \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\") " pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:22:51.924630 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.924540    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07f7c7ec-b1b6-4a89-a761-08c3af23436c-istio-token\") pod \"e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn\" (UID: \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\") " pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:22:51.924630 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.924559    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f7c7ec-b1b6-4a89-a761-08c3af23436c-workload-socket\") pod \"e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn\" (UID: \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\") " pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:22:51.924834 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.924776    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07f7c7ec-b1b6-4a89-a761-08c3af23436c-istio-data\") pod \"e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn\" (UID: \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\") " pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:22:51.924834 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.924821    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07f7c7ec-b1b6-4a89-a761-08c3af23436c-workload-certs\") pod \"e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn\" (UID: \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\") " pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:22:51.924965 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.924892    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f7c7ec-b1b6-4a89-a761-08c3af23436c-credential-socket\") pod \"e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn\" (UID: \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\") " pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:22:51.925360 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.925334    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07f7c7ec-b1b6-4a89-a761-08c3af23436c-istiod-ca-cert\") pod \"e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn\" (UID: \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\") " pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:22:51.926513 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.926494    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07f7c7ec-b1b6-4a89-a761-08c3af23436c-istio-envoy\") pod \"e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn\" (UID: \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\") " pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:22:51.926764 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.926745    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07f7c7ec-b1b6-4a89-a761-08c3af23436c-istio-podinfo\") pod \"e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn\" (UID: \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\") " pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:22:51.933213 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.933186    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07f7c7ec-b1b6-4a89-a761-08c3af23436c-istio-token\") pod \"e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn\" (UID: \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\") " pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:22:51.933322 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.933285    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwczm\" (UniqueName: \"kubernetes.io/projected/07f7c7ec-b1b6-4a89-a761-08c3af23436c-kube-api-access-bwczm\") pod \"e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn\" (UID: \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\") " pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:22:51.972315 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:51.972287    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:22:52.106297 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:52.106269    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"]
Jul 30 04:22:52.107628 ip-10-0-143-117 kubenswrapper[2576]: W0730 04:22:52.107601    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07f7c7ec_b1b6_4a89_a761_08c3af23436c.slice/crio-0b37ccd6ccabf9184cd384716647edd617bc3971a76b0dc0a2983435164b93f7 WatchSource:0}: Error finding container 0b37ccd6ccabf9184cd384716647edd617bc3971a76b0dc0a2983435164b93f7: Status 404 returned error can't find the container with id 0b37ccd6ccabf9184cd384716647edd617bc3971a76b0dc0a2983435164b93f7
Jul 30 04:22:52.109798 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:52.109767    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:22:52.109894 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:52.109862    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:22:52.110028 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:52.109907    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:22:52.114797 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:52.114771    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:52.598995 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:52.598957    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn" event={"ID":"07f7c7ec-b1b6-4a89-a761-08c3af23436c","Type":"ContainerStarted","Data":"a4f3926cbe5f329e485bfddd3f776555443c8c5934ee9a8dbefd109c7b178a78"}
Jul 30 04:22:52.599201 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:52.599022    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn" event={"ID":"07f7c7ec-b1b6-4a89-a761-08c3af23436c","Type":"ContainerStarted","Data":"0b37ccd6ccabf9184cd384716647edd617bc3971a76b0dc0a2983435164b93f7"}
Jul 30 04:22:52.622220 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:52.622173    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn" podStartSLOduration=1.622157993 podStartE2EDuration="1.622157993s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:22:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:22:52.619513635 +0000 UTC m=+2878.237745876" watchObservedRunningTime="2026-07-30 04:22:52.622157993 +0000 UTC m=+2878.240390250"
Jul 30 04:22:52.972459 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:52.972365    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:22:53.977487 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:53.977460    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:22:54.471536 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:54.471504    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:54.574357 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:54.574324    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:54.610731 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:54.610701    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:22:54.611741 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:54.611725    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:22:55.745746 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:55.745656    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:55.838425 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:55.838394    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:56.078833 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.078810    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:22:56.164244 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.164205    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dc89457-4084-4353-9043-3bf110a777a5-workload-certs\") pod \"6dc89457-4084-4353-9043-3bf110a777a5\" (UID: \"6dc89457-4084-4353-9043-3bf110a777a5\") "
Jul 30 04:22:56.164442 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.164312    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dc89457-4084-4353-9043-3bf110a777a5-istiod-ca-cert\") pod \"6dc89457-4084-4353-9043-3bf110a777a5\" (UID: \"6dc89457-4084-4353-9043-3bf110a777a5\") "
Jul 30 04:22:56.164442 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.164339    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9tjr\" (UniqueName: \"kubernetes.io/projected/6dc89457-4084-4353-9043-3bf110a777a5-kube-api-access-p9tjr\") pod \"6dc89457-4084-4353-9043-3bf110a777a5\" (UID: \"6dc89457-4084-4353-9043-3bf110a777a5\") "
Jul 30 04:22:56.164442 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.164356    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dc89457-4084-4353-9043-3bf110a777a5-workload-socket\") pod \"6dc89457-4084-4353-9043-3bf110a777a5\" (UID: \"6dc89457-4084-4353-9043-3bf110a777a5\") "
Jul 30 04:22:56.164442 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.164374    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dc89457-4084-4353-9043-3bf110a777a5-credential-socket\") pod \"6dc89457-4084-4353-9043-3bf110a777a5\" (UID: \"6dc89457-4084-4353-9043-3bf110a777a5\") "
Jul 30 04:22:56.164442 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.164401    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dc89457-4084-4353-9043-3bf110a777a5-istio-envoy\") pod \"6dc89457-4084-4353-9043-3bf110a777a5\" (UID: \"6dc89457-4084-4353-9043-3bf110a777a5\") "
Jul 30 04:22:56.164719 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.164455    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dc89457-4084-4353-9043-3bf110a777a5-istio-podinfo\") pod \"6dc89457-4084-4353-9043-3bf110a777a5\" (UID: \"6dc89457-4084-4353-9043-3bf110a777a5\") "
Jul 30 04:22:56.164719 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.164478    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dc89457-4084-4353-9043-3bf110a777a5-istio-data\") pod \"6dc89457-4084-4353-9043-3bf110a777a5\" (UID: \"6dc89457-4084-4353-9043-3bf110a777a5\") "
Jul 30 04:22:56.164719 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.164532    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dc89457-4084-4353-9043-3bf110a777a5-istio-token\") pod \"6dc89457-4084-4353-9043-3bf110a777a5\" (UID: \"6dc89457-4084-4353-9043-3bf110a777a5\") "
Jul 30 04:22:56.164719 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.164562    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dc89457-4084-4353-9043-3bf110a777a5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6dc89457-4084-4353-9043-3bf110a777a5" (UID: "6dc89457-4084-4353-9043-3bf110a777a5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:22:56.164719 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.164693    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dc89457-4084-4353-9043-3bf110a777a5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6dc89457-4084-4353-9043-3bf110a777a5" (UID: "6dc89457-4084-4353-9043-3bf110a777a5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:22:56.164719 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.164706    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dc89457-4084-4353-9043-3bf110a777a5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6dc89457-4084-4353-9043-3bf110a777a5" (UID: "6dc89457-4084-4353-9043-3bf110a777a5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:22:56.165062 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.164811    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6dc89457-4084-4353-9043-3bf110a777a5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6dc89457-4084-4353-9043-3bf110a777a5" (UID: "6dc89457-4084-4353-9043-3bf110a777a5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:22:56.165119 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.165071    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dc89457-4084-4353-9043-3bf110a777a5-istiod-ca-cert\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:22:56.165119 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.165093    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dc89457-4084-4353-9043-3bf110a777a5-workload-socket\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:22:56.165119 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.165109    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dc89457-4084-4353-9043-3bf110a777a5-credential-socket\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:22:56.165274 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.165122    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dc89457-4084-4353-9043-3bf110a777a5-workload-certs\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:22:56.165481 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.165434    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dc89457-4084-4353-9043-3bf110a777a5-istio-data" (OuterVolumeSpecName: "istio-data") pod "6dc89457-4084-4353-9043-3bf110a777a5" (UID: "6dc89457-4084-4353-9043-3bf110a777a5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:22:56.166745 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.166719    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6dc89457-4084-4353-9043-3bf110a777a5-kube-api-access-p9tjr" (OuterVolumeSpecName: "kube-api-access-p9tjr") pod "6dc89457-4084-4353-9043-3bf110a777a5" (UID: "6dc89457-4084-4353-9043-3bf110a777a5"). InnerVolumeSpecName "kube-api-access-p9tjr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:22:56.167042 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.167018    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6dc89457-4084-4353-9043-3bf110a777a5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6dc89457-4084-4353-9043-3bf110a777a5" (UID: "6dc89457-4084-4353-9043-3bf110a777a5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 04:22:56.167135 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.167018    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dc89457-4084-4353-9043-3bf110a777a5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6dc89457-4084-4353-9043-3bf110a777a5" (UID: "6dc89457-4084-4353-9043-3bf110a777a5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:22:56.167179 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.167154    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6dc89457-4084-4353-9043-3bf110a777a5-istio-token" (OuterVolumeSpecName: "istio-token") pod "6dc89457-4084-4353-9043-3bf110a777a5" (UID: "6dc89457-4084-4353-9043-3bf110a777a5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:22:56.266018 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.265988    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dc89457-4084-4353-9043-3bf110a777a5-istio-token\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:22:56.266018 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.266015    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9tjr\" (UniqueName: \"kubernetes.io/projected/6dc89457-4084-4353-9043-3bf110a777a5-kube-api-access-p9tjr\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:22:56.266209 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.266026    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dc89457-4084-4353-9043-3bf110a777a5-istio-envoy\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:22:56.266209 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.266035    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dc89457-4084-4353-9043-3bf110a777a5-istio-podinfo\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:22:56.266209 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.266043    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dc89457-4084-4353-9043-3bf110a777a5-istio-data\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:22:56.272802 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.272772    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:56.369569 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.369491    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:22:56.373877 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.373854    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-9c7b98-5fcdd5d79b-5p6hd"]
Jul 30 04:22:56.374213 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.374202    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6dc89457-4084-4353-9043-3bf110a777a5" containerName="istio-proxy"
Jul 30 04:22:56.374257 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.374215    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6dc89457-4084-4353-9043-3bf110a777a5" containerName="istio-proxy"
Jul 30 04:22:56.374257 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.374240    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6dc89457-4084-4353-9043-3bf110a777a5" containerName="istio-proxy"
Jul 30 04:22:56.374257 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.374246    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6dc89457-4084-4353-9043-3bf110a777a5" containerName="istio-proxy"
Jul 30 04:22:56.374347 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.374300    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5732b28f-b1be-4de2-8708-a535a8d559e7" containerName="istio-proxy"
Jul 30 04:22:56.374347 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.374310    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6dc89457-4084-4353-9043-3bf110a777a5" containerName="istio-proxy"
Jul 30 04:22:56.374347 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.374317    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6dc89457-4084-4353-9043-3bf110a777a5" containerName="istio-proxy"
Jul 30 04:22:56.377569 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.377555    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c7b98-5fcdd5d79b-5p6hd"
Jul 30 04:22:56.380785 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.380766    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-9c7b98-dockercfg-t5cnd\""
Jul 30 04:22:56.380785 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.380781    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-9c7b98\""
Jul 30 04:22:56.387097 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.387077    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-9c7b98-5fcdd5d79b-5p6hd"]
Jul 30 04:22:56.467999 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.467968    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxg4n\" (UniqueName: \"kubernetes.io/projected/2b796602-5c31-40a7-b39a-c35b2c993ed0-kube-api-access-wxg4n\") pod \"payload-pre-processing-e2e-api-a-9c7b98-5fcdd5d79b-5p6hd\" (UID: \"2b796602-5c31-40a7-b39a-c35b2c993ed0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c7b98-5fcdd5d79b-5p6hd"
Jul 30 04:22:56.468171 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.468133    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b796602-5c31-40a7-b39a-c35b2c993ed0-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-9c7b98-5fcdd5d79b-5p6hd\" (UID: \"2b796602-5c31-40a7-b39a-c35b2c993ed0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c7b98-5fcdd5d79b-5p6hd"
Jul 30 04:22:56.569141 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.569101    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxg4n\" (UniqueName: \"kubernetes.io/projected/2b796602-5c31-40a7-b39a-c35b2c993ed0-kube-api-access-wxg4n\") pod \"payload-pre-processing-e2e-api-a-9c7b98-5fcdd5d79b-5p6hd\" (UID: \"2b796602-5c31-40a7-b39a-c35b2c993ed0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c7b98-5fcdd5d79b-5p6hd"
Jul 30 04:22:56.569325 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.569186    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b796602-5c31-40a7-b39a-c35b2c993ed0-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-9c7b98-5fcdd5d79b-5p6hd\" (UID: \"2b796602-5c31-40a7-b39a-c35b2c993ed0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c7b98-5fcdd5d79b-5p6hd"
Jul 30 04:22:56.569834 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.569816    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b796602-5c31-40a7-b39a-c35b2c993ed0-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-9c7b98-5fcdd5d79b-5p6hd\" (UID: \"2b796602-5c31-40a7-b39a-c35b2c993ed0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c7b98-5fcdd5d79b-5p6hd"
Jul 30 04:22:56.576985 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.576962    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxg4n\" (UniqueName: \"kubernetes.io/projected/2b796602-5c31-40a7-b39a-c35b2c993ed0-kube-api-access-wxg4n\") pod \"payload-pre-processing-e2e-api-a-9c7b98-5fcdd5d79b-5p6hd\" (UID: \"2b796602-5c31-40a7-b39a-c35b2c993ed0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c7b98-5fcdd5d79b-5p6hd"
Jul 30 04:22:56.626897 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.626819    2576 generic.go:356] "Generic (PLEG): container finished" podID="6dc89457-4084-4353-9043-3bf110a777a5" containerID="4aa058c21255aa3841a0d5f5ee3f7c307c7e108eaa071697a7b1fb11f3b57a91" exitCode=0
Jul 30 04:22:56.626897 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.626878    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"
Jul 30 04:22:56.627104 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.626906    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5" event={"ID":"6dc89457-4084-4353-9043-3bf110a777a5","Type":"ContainerDied","Data":"4aa058c21255aa3841a0d5f5ee3f7c307c7e108eaa071697a7b1fb11f3b57a91"}
Jul 30 04:22:56.627104 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.626974    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5" event={"ID":"6dc89457-4084-4353-9043-3bf110a777a5","Type":"ContainerDied","Data":"61bfa04c3e965e0a02b80f68373747d87d8dacf147119cb28e063561f8675825"}
Jul 30 04:22:56.627104 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.626998    2576 scope.go:117] "RemoveContainer" containerID="4aa058c21255aa3841a0d5f5ee3f7c307c7e108eaa071697a7b1fb11f3b57a91"
Jul 30 04:22:56.638642 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.638625    2576 scope.go:117] "RemoveContainer" containerID="d0719ddbf22f362c2b789cc6184bfe3f2418f2efad751bd6144c6ca79ad2c78a"
Jul 30 04:22:56.646249 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.646234    2576 scope.go:117] "RemoveContainer" containerID="4aa058c21255aa3841a0d5f5ee3f7c307c7e108eaa071697a7b1fb11f3b57a91"
Jul 30 04:22:56.646479 ip-10-0-143-117 kubenswrapper[2576]: E0730 04:22:56.646459    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4aa058c21255aa3841a0d5f5ee3f7c307c7e108eaa071697a7b1fb11f3b57a91\": container with ID starting with 4aa058c21255aa3841a0d5f5ee3f7c307c7e108eaa071697a7b1fb11f3b57a91 not found: ID does not exist" containerID="4aa058c21255aa3841a0d5f5ee3f7c307c7e108eaa071697a7b1fb11f3b57a91"
Jul 30 04:22:56.646521 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.646487    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4aa058c21255aa3841a0d5f5ee3f7c307c7e108eaa071697a7b1fb11f3b57a91"} err="failed to get container status \"4aa058c21255aa3841a0d5f5ee3f7c307c7e108eaa071697a7b1fb11f3b57a91\": rpc error: code = NotFound desc = could not find container \"4aa058c21255aa3841a0d5f5ee3f7c307c7e108eaa071697a7b1fb11f3b57a91\": container with ID starting with 4aa058c21255aa3841a0d5f5ee3f7c307c7e108eaa071697a7b1fb11f3b57a91 not found: ID does not exist"
Jul 30 04:22:56.646521 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.646502    2576 scope.go:117] "RemoveContainer" containerID="d0719ddbf22f362c2b789cc6184bfe3f2418f2efad751bd6144c6ca79ad2c78a"
Jul 30 04:22:56.646702 ip-10-0-143-117 kubenswrapper[2576]: E0730 04:22:56.646686    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0719ddbf22f362c2b789cc6184bfe3f2418f2efad751bd6144c6ca79ad2c78a\": container with ID starting with d0719ddbf22f362c2b789cc6184bfe3f2418f2efad751bd6144c6ca79ad2c78a not found: ID does not exist" containerID="d0719ddbf22f362c2b789cc6184bfe3f2418f2efad751bd6144c6ca79ad2c78a"
Jul 30 04:22:56.646739 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.646707    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0719ddbf22f362c2b789cc6184bfe3f2418f2efad751bd6144c6ca79ad2c78a"} err="failed to get container status \"d0719ddbf22f362c2b789cc6184bfe3f2418f2efad751bd6144c6ca79ad2c78a\": rpc error: code = NotFound desc = could not find container \"d0719ddbf22f362c2b789cc6184bfe3f2418f2efad751bd6144c6ca79ad2c78a\": container with ID starting with d0719ddbf22f362c2b789cc6184bfe3f2418f2efad751bd6144c6ca79ad2c78a not found: ID does not exist"
Jul 30 04:22:56.672117 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.672090    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"]
Jul 30 04:22:56.688168 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.688144    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-5f61e5-openshift-default-788fbfbb4-5whd5"]
Jul 30 04:22:56.688724 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.688706    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c7b98-5fcdd5d79b-5p6hd"
Jul 30 04:22:56.819780 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:56.819757    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-9c7b98-5fcdd5d79b-5p6hd"]
Jul 30 04:22:56.821388 ip-10-0-143-117 kubenswrapper[2576]: W0730 04:22:56.821361    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b796602_5c31_40a7_b39a_c35b2c993ed0.slice/crio-2c45b9cfecdeb822093bce723e70c7ef6bcf39ba0ce665a8e6a756d4124d8bef WatchSource:0}: Error finding container 2c45b9cfecdeb822093bce723e70c7ef6bcf39ba0ce665a8e6a756d4124d8bef: Status 404 returned error can't find the container with id 2c45b9cfecdeb822093bce723e70c7ef6bcf39ba0ce665a8e6a756d4124d8bef
Jul 30 04:22:57.052014 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:57.051984    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6dc89457-4084-4353-9043-3bf110a777a5" path="/var/lib/kubelet/pods/6dc89457-4084-4353-9043-3bf110a777a5/volumes"
Jul 30 04:22:57.634381 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:57.634347    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c7b98-5fcdd5d79b-5p6hd" event={"ID":"2b796602-5c31-40a7-b39a-c35b2c993ed0","Type":"ContainerStarted","Data":"fee2c989e8577f4b3b75ca803a5e05434011cf20a87ab96625b63f1a045bea20"}
Jul 30 04:22:57.634581 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:57.634412    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c7b98-5fcdd5d79b-5p6hd" event={"ID":"2b796602-5c31-40a7-b39a-c35b2c993ed0","Type":"ContainerStarted","Data":"2c45b9cfecdeb822093bce723e70c7ef6bcf39ba0ce665a8e6a756d4124d8bef"}
Jul 30 04:22:57.634581 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:57.634445    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c7b98-5fcdd5d79b-5p6hd"
Jul 30 04:22:57.654686 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:22:57.654636    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c7b98-5fcdd5d79b-5p6hd" podStartSLOduration=1.654620575 podStartE2EDuration="1.654620575s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:22:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:22:57.65185482 +0000 UTC m=+2883.270087060" watchObservedRunningTime="2026-07-30 04:22:57.654620575 +0000 UTC m=+2883.272853137"
Jul 30 04:23:00.136168 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:00.136132    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:23:07.691961 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:07.691903    2576 generic.go:356] "Generic (PLEG): container finished" podID="07f7c7ec-b1b6-4a89-a761-08c3af23436c" containerID="a4f3926cbe5f329e485bfddd3f776555443c8c5934ee9a8dbefd109c7b178a78" exitCode=0
Jul 30 04:23:07.692446 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:07.691988    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn" event={"ID":"07f7c7ec-b1b6-4a89-a761-08c3af23436c","Type":"ContainerDied","Data":"a4f3926cbe5f329e485bfddd3f776555443c8c5934ee9a8dbefd109c7b178a78"}
Jul 30 04:23:07.692446 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:07.692374    2576 scope.go:117] "RemoveContainer" containerID="a4f3926cbe5f329e485bfddd3f776555443c8c5934ee9a8dbefd109c7b178a78"
Jul 30 04:23:07.693366 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:07.693337    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:23:07.693470 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:07.693424    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:23:07.693540 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:07.693480    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:23:08.643343 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:08.643305    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c7b98-5fcdd5d79b-5p6hd"
Jul 30 04:23:08.704099 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:08.704057    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn" event={"ID":"07f7c7ec-b1b6-4a89-a761-08c3af23436c","Type":"ContainerStarted","Data":"5be2bcabb95a98432df2e5c60a5b8cbf72422261dc2ed144e4ed0dc4d382cdcd"}
Jul 30 04:23:08.973364 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:08.973273    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:23:08.974531 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:08.974503    2576 patch_prober.go:28] interesting pod/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Jul 30 04:23:08.974658 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:08.974567    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn" podUID="07f7c7ec-b1b6-4a89-a761-08c3af23436c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Jul 30 04:23:09.611174 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:09.611140    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:23:09.973715 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:09.973632    2576 patch_prober.go:28] interesting pod/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Jul 30 04:23:09.973715 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:09.973696    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn" podUID="07f7c7ec-b1b6-4a89-a761-08c3af23436c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Jul 30 04:23:10.973704 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:10.973667    2576 patch_prober.go:28] interesting pod/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Jul 30 04:23:10.973884 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:10.973725    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn" podUID="07f7c7ec-b1b6-4a89-a761-08c3af23436c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Jul 30 04:23:11.972731 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:11.972694    2576 patch_prober.go:28] interesting pod/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Jul 30 04:23:11.972944 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:11.972751    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn" podUID="07f7c7ec-b1b6-4a89-a761-08c3af23436c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Jul 30 04:23:12.976839 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:12.976807    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:23:12.977850 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:12.977830    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:23:19.436576 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:19.436536    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:23:20.130197 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.130166    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"]
Jul 30 04:23:20.133765 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.133746    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:23:20.136236 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.136220    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-2a2d39-openshift-default-dockercfg-dtnqb\""
Jul 30 04:23:20.143553 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.143527    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"]
Jul 30 04:23:20.199165 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.199139    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmlhd\" (UniqueName: \"kubernetes.io/projected/a05e918c-cf1c-48a5-b57f-888f27c239e3-kube-api-access-tmlhd\") pod \"e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd\" (UID: \"a05e918c-cf1c-48a5-b57f-888f27c239e3\") " pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:23:20.199340 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.199171    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a05e918c-cf1c-48a5-b57f-888f27c239e3-istio-data\") pod \"e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd\" (UID: \"a05e918c-cf1c-48a5-b57f-888f27c239e3\") " pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:23:20.199340 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.199191    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a05e918c-cf1c-48a5-b57f-888f27c239e3-workload-socket\") pod \"e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd\" (UID: \"a05e918c-cf1c-48a5-b57f-888f27c239e3\") " pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:23:20.199340 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.199294    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a05e918c-cf1c-48a5-b57f-888f27c239e3-workload-certs\") pod \"e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd\" (UID: \"a05e918c-cf1c-48a5-b57f-888f27c239e3\") " pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:23:20.199340 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.199328    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a05e918c-cf1c-48a5-b57f-888f27c239e3-istio-podinfo\") pod \"e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd\" (UID: \"a05e918c-cf1c-48a5-b57f-888f27c239e3\") " pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:23:20.199566 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.199382    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a05e918c-cf1c-48a5-b57f-888f27c239e3-istio-envoy\") pod \"e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd\" (UID: \"a05e918c-cf1c-48a5-b57f-888f27c239e3\") " pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:23:20.199566 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.199425    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a05e918c-cf1c-48a5-b57f-888f27c239e3-istio-token\") pod \"e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd\" (UID: \"a05e918c-cf1c-48a5-b57f-888f27c239e3\") " pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:23:20.199566 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.199492    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a05e918c-cf1c-48a5-b57f-888f27c239e3-credential-socket\") pod \"e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd\" (UID: \"a05e918c-cf1c-48a5-b57f-888f27c239e3\") " pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:23:20.199566 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.199544    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a05e918c-cf1c-48a5-b57f-888f27c239e3-istiod-ca-cert\") pod \"e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd\" (UID: \"a05e918c-cf1c-48a5-b57f-888f27c239e3\") " pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:23:20.303474 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.303434    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a05e918c-cf1c-48a5-b57f-888f27c239e3-istiod-ca-cert\") pod \"e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd\" (UID: \"a05e918c-cf1c-48a5-b57f-888f27c239e3\") " pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:23:20.303674 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.303602    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmlhd\" (UniqueName: \"kubernetes.io/projected/a05e918c-cf1c-48a5-b57f-888f27c239e3-kube-api-access-tmlhd\") pod \"e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd\" (UID: \"a05e918c-cf1c-48a5-b57f-888f27c239e3\") " pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:23:20.303674 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.303661    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a05e918c-cf1c-48a5-b57f-888f27c239e3-istio-data\") pod \"e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd\" (UID: \"a05e918c-cf1c-48a5-b57f-888f27c239e3\") " pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:23:20.303815 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.303695    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a05e918c-cf1c-48a5-b57f-888f27c239e3-workload-socket\") pod \"e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd\" (UID: \"a05e918c-cf1c-48a5-b57f-888f27c239e3\") " pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:23:20.303815 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.303764    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a05e918c-cf1c-48a5-b57f-888f27c239e3-workload-certs\") pod \"e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd\" (UID: \"a05e918c-cf1c-48a5-b57f-888f27c239e3\") " pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:23:20.303815 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.303797    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a05e918c-cf1c-48a5-b57f-888f27c239e3-istio-podinfo\") pod \"e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd\" (UID: \"a05e918c-cf1c-48a5-b57f-888f27c239e3\") " pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:23:20.304015 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.303839    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a05e918c-cf1c-48a5-b57f-888f27c239e3-istio-envoy\") pod \"e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd\" (UID: \"a05e918c-cf1c-48a5-b57f-888f27c239e3\") " pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:23:20.304015 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.303876    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a05e918c-cf1c-48a5-b57f-888f27c239e3-istio-token\") pod \"e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd\" (UID: \"a05e918c-cf1c-48a5-b57f-888f27c239e3\") " pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:23:20.304015 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.303975    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a05e918c-cf1c-48a5-b57f-888f27c239e3-credential-socket\") pod \"e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd\" (UID: \"a05e918c-cf1c-48a5-b57f-888f27c239e3\") " pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:23:20.304180 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.304114    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a05e918c-cf1c-48a5-b57f-888f27c239e3-istio-data\") pod \"e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd\" (UID: \"a05e918c-cf1c-48a5-b57f-888f27c239e3\") " pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:23:20.304180 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.304164    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a05e918c-cf1c-48a5-b57f-888f27c239e3-workload-socket\") pod \"e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd\" (UID: \"a05e918c-cf1c-48a5-b57f-888f27c239e3\") " pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:23:20.304689 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.304664    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a05e918c-cf1c-48a5-b57f-888f27c239e3-istiod-ca-cert\") pod \"e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd\" (UID: \"a05e918c-cf1c-48a5-b57f-888f27c239e3\") " pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:23:20.304810 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.304746    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a05e918c-cf1c-48a5-b57f-888f27c239e3-credential-socket\") pod \"e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd\" (UID: \"a05e918c-cf1c-48a5-b57f-888f27c239e3\") " pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:23:20.304898 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.304879    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a05e918c-cf1c-48a5-b57f-888f27c239e3-workload-certs\") pod \"e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd\" (UID: \"a05e918c-cf1c-48a5-b57f-888f27c239e3\") " pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:23:20.306876 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.306851    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a05e918c-cf1c-48a5-b57f-888f27c239e3-istio-podinfo\") pod \"e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd\" (UID: \"a05e918c-cf1c-48a5-b57f-888f27c239e3\") " pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:23:20.307014 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.306917    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a05e918c-cf1c-48a5-b57f-888f27c239e3-istio-envoy\") pod \"e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd\" (UID: \"a05e918c-cf1c-48a5-b57f-888f27c239e3\") " pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:23:20.311607 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.311586    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a05e918c-cf1c-48a5-b57f-888f27c239e3-istio-token\") pod \"e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd\" (UID: \"a05e918c-cf1c-48a5-b57f-888f27c239e3\") " pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:23:20.311823 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.311801    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmlhd\" (UniqueName: \"kubernetes.io/projected/a05e918c-cf1c-48a5-b57f-888f27c239e3-kube-api-access-tmlhd\") pod \"e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd\" (UID: \"a05e918c-cf1c-48a5-b57f-888f27c239e3\") " pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:23:20.444908 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.444822    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:23:20.620689 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.620661    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"]
Jul 30 04:23:20.622073 ip-10-0-143-117 kubenswrapper[2576]: W0730 04:23:20.622046    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda05e918c_cf1c_48a5_b57f_888f27c239e3.slice/crio-7da329d21397ad5f0ca1cba4b989f801718469f678151287cb839749e00fcb14 WatchSource:0}: Error finding container 7da329d21397ad5f0ca1cba4b989f801718469f678151287cb839749e00fcb14: Status 404 returned error can't find the container with id 7da329d21397ad5f0ca1cba4b989f801718469f678151287cb839749e00fcb14
Jul 30 04:23:20.624234 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.624202    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:23:20.624326 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.624266    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:23:20.624326 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.624296    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:23:20.743092 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.742999    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:23:20.772141 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.772105    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd" event={"ID":"a05e918c-cf1c-48a5-b57f-888f27c239e3","Type":"ContainerStarted","Data":"731ee09c89cc03a1ac16fc2e906031af61bdd301dd524aef92f7e41ed21a3a43"}
Jul 30 04:23:20.772299 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.772172    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd" event={"ID":"a05e918c-cf1c-48a5-b57f-888f27c239e3","Type":"ContainerStarted","Data":"7da329d21397ad5f0ca1cba4b989f801718469f678151287cb839749e00fcb14"}
Jul 30 04:23:20.793187 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:20.793138    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd" podStartSLOduration=0.793122152 podStartE2EDuration="793.122152ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:23:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:23:20.790606135 +0000 UTC m=+2906.408838375" watchObservedRunningTime="2026-07-30 04:23:20.793122152 +0000 UTC m=+2906.411354392"
Jul 30 04:23:21.445334 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:21.445298    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:23:21.450306 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:21.450284    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:23:21.778023 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:21.777996    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:23:21.778906 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:21.778882    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:23:22.648354 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:22.648320    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:23:22.738401 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:22.738368    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:23:23.777597 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:23.777559    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:23:24.114090 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:24.114013    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:23:24.298417 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:24.298388    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:23:24.397849 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:24.397767    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:23:24.452995 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:24.452966    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-2a2d39-8f8cbfbbd-6gmqc"]
Jul 30 04:23:24.456595 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:24.456580    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-2a2d39-8f8cbfbbd-6gmqc"
Jul 30 04:23:24.459172 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:24.459150    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-2a2d39-dockercfg-m96g4\""
Jul 30 04:23:24.459289 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:24.459170    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-2a2d39\""
Jul 30 04:23:24.466101 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:24.466076    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-2a2d39-8f8cbfbbd-6gmqc"]
Jul 30 04:23:24.549407 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:24.549369    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmp2s\" (UniqueName: \"kubernetes.io/projected/c11f29f9-a542-4ce7-b3a2-6b4147891549-kube-api-access-qmp2s\") pod \"payload-processing-e2e-api-b-2a2d39-8f8cbfbbd-6gmqc\" (UID: \"c11f29f9-a542-4ce7-b3a2-6b4147891549\") " pod="openshift-ingress/payload-processing-e2e-api-b-2a2d39-8f8cbfbbd-6gmqc"
Jul 30 04:23:24.549593 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:24.549418    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c11f29f9-a542-4ce7-b3a2-6b4147891549-plugins-config-volume\") pod \"payload-processing-e2e-api-b-2a2d39-8f8cbfbbd-6gmqc\" (UID: \"c11f29f9-a542-4ce7-b3a2-6b4147891549\") " pod="openshift-ingress/payload-processing-e2e-api-b-2a2d39-8f8cbfbbd-6gmqc"
Jul 30 04:23:24.650683 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:24.650598    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmp2s\" (UniqueName: \"kubernetes.io/projected/c11f29f9-a542-4ce7-b3a2-6b4147891549-kube-api-access-qmp2s\") pod \"payload-processing-e2e-api-b-2a2d39-8f8cbfbbd-6gmqc\" (UID: \"c11f29f9-a542-4ce7-b3a2-6b4147891549\") " pod="openshift-ingress/payload-processing-e2e-api-b-2a2d39-8f8cbfbbd-6gmqc"
Jul 30 04:23:24.650683 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:24.650637    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c11f29f9-a542-4ce7-b3a2-6b4147891549-plugins-config-volume\") pod \"payload-processing-e2e-api-b-2a2d39-8f8cbfbbd-6gmqc\" (UID: \"c11f29f9-a542-4ce7-b3a2-6b4147891549\") " pod="openshift-ingress/payload-processing-e2e-api-b-2a2d39-8f8cbfbbd-6gmqc"
Jul 30 04:23:24.651296 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:24.651278    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c11f29f9-a542-4ce7-b3a2-6b4147891549-plugins-config-volume\") pod \"payload-processing-e2e-api-b-2a2d39-8f8cbfbbd-6gmqc\" (UID: \"c11f29f9-a542-4ce7-b3a2-6b4147891549\") " pod="openshift-ingress/payload-processing-e2e-api-b-2a2d39-8f8cbfbbd-6gmqc"
Jul 30 04:23:24.658359 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:24.658341    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmp2s\" (UniqueName: \"kubernetes.io/projected/c11f29f9-a542-4ce7-b3a2-6b4147891549-kube-api-access-qmp2s\") pod \"payload-processing-e2e-api-b-2a2d39-8f8cbfbbd-6gmqc\" (UID: \"c11f29f9-a542-4ce7-b3a2-6b4147891549\") " pod="openshift-ingress/payload-processing-e2e-api-b-2a2d39-8f8cbfbbd-6gmqc"
Jul 30 04:23:24.767974 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:24.767944    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-2a2d39-8f8cbfbbd-6gmqc"
Jul 30 04:23:24.900843 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:24.900789    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-2a2d39-8f8cbfbbd-6gmqc"]
Jul 30 04:23:24.901870 ip-10-0-143-117 kubenswrapper[2576]: W0730 04:23:24.901840    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc11f29f9_a542_4ce7_b3a2_6b4147891549.slice/crio-d86b4cb377c480dd8d37a8f5ee49bf436414993316f1f265ac5e6f934eb75e8e WatchSource:0}: Error finding container d86b4cb377c480dd8d37a8f5ee49bf436414993316f1f265ac5e6f934eb75e8e: Status 404 returned error can't find the container with id d86b4cb377c480dd8d37a8f5ee49bf436414993316f1f265ac5e6f934eb75e8e
Jul 30 04:23:25.801516 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:25.801484    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-2a2d39-8f8cbfbbd-6gmqc" event={"ID":"c11f29f9-a542-4ce7-b3a2-6b4147891549","Type":"ContainerStarted","Data":"f9dbd43d5b6a6c3ba5d6d2085e2cf3bcb7e37946380cf8ae8d4ad48c4296fa94"}
Jul 30 04:23:25.801691 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:25.801537    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-2a2d39-8f8cbfbbd-6gmqc" event={"ID":"c11f29f9-a542-4ce7-b3a2-6b4147891549","Type":"ContainerStarted","Data":"d86b4cb377c480dd8d37a8f5ee49bf436414993316f1f265ac5e6f934eb75e8e"}
Jul 30 04:23:25.801691 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:25.801614    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-2a2d39-8f8cbfbbd-6gmqc"
Jul 30 04:23:25.818029 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:25.817990    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-2a2d39-8f8cbfbbd-6gmqc" podStartSLOduration=1.81797867 podStartE2EDuration="1.81797867s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:23:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:23:25.817084937 +0000 UTC m=+2911.435317179" watchObservedRunningTime="2026-07-30 04:23:25.81797867 +0000 UTC m=+2911.436210908"
Jul 30 04:23:31.045240 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:31.045207    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:23:36.808554 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:36.808523    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-2a2d39-8f8cbfbbd-6gmqc"
Jul 30 04:23:37.820096 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:37.820056    2576 patch_prober.go:28] interesting pod/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 30 04:23:37.820501 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:37.820143    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd" podUID="a05e918c-cf1c-48a5-b57f-888f27c239e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 30 04:23:39.888192 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:39.888136    2576 generic.go:356] "Generic (PLEG): container finished" podID="a05e918c-cf1c-48a5-b57f-888f27c239e3" containerID="731ee09c89cc03a1ac16fc2e906031af61bdd301dd524aef92f7e41ed21a3a43" exitCode=0
Jul 30 04:23:39.888192 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:39.888174    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd" event={"ID":"a05e918c-cf1c-48a5-b57f-888f27c239e3","Type":"ContainerDied","Data":"731ee09c89cc03a1ac16fc2e906031af61bdd301dd524aef92f7e41ed21a3a43"}
Jul 30 04:23:39.888698 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:39.888568    2576 scope.go:117] "RemoveContainer" containerID="731ee09c89cc03a1ac16fc2e906031af61bdd301dd524aef92f7e41ed21a3a43"
Jul 30 04:23:39.889663 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:39.889619    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:23:39.889717 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:39.889704    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:23:39.889753 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:39.889741    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:23:40.896363 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:40.896323    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd" event={"ID":"a05e918c-cf1c-48a5-b57f-888f27c239e3","Type":"ContainerStarted","Data":"1053b6384503cb363967695abd19a43cbd824a6c7e3066d12dbb151b0b94a7ba"}
Jul 30 04:23:41.445992 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:41.445951    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:23:41.446280 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:41.445998    2576 patch_prober.go:28] interesting pod/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Jul 30 04:23:41.446432 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:41.446407    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd" podUID="a05e918c-cf1c-48a5-b57f-888f27c239e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Jul 30 04:23:42.445963 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:42.445906    2576 patch_prober.go:28] interesting pod/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Jul 30 04:23:42.446558 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:42.446525    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd" podUID="a05e918c-cf1c-48a5-b57f-888f27c239e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Jul 30 04:23:43.445867 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:43.445401    2576 patch_prober.go:28] interesting pod/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Jul 30 04:23:43.445867 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:43.445473    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd" podUID="a05e918c-cf1c-48a5-b57f-888f27c239e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Jul 30 04:23:44.445987 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:44.445949    2576 patch_prober.go:28] interesting pod/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Jul 30 04:23:44.446266 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:44.446237    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd" podUID="a05e918c-cf1c-48a5-b57f-888f27c239e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Jul 30 04:23:45.449364 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:45.449335    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:23:45.449771 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:45.449629    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:23:45.450436 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:45.450420    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:23:47.566646 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:23:47.566616    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:05.801598 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:05.801565    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:05.840718 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:05.840681    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-9c7b98-5fcdd5d79b-5p6hd"]
Jul 30 04:24:05.841220 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:05.841032    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c7b98-5fcdd5d79b-5p6hd" podUID="2b796602-5c31-40a7-b39a-c35b2c993ed0" containerName="payload-pre-processing" containerID="cri-o://fee2c989e8577f4b3b75ca803a5e05434011cf20a87ab96625b63f1a045bea20" gracePeriod=30
Jul 30 04:24:05.867095 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:05.867063    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:05.930388 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:05.930358    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:05.993245 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:05.993206    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:06.041282 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:06.041239    2576 generic.go:356] "Generic (PLEG): container finished" podID="2b796602-5c31-40a7-b39a-c35b2c993ed0" containerID="fee2c989e8577f4b3b75ca803a5e05434011cf20a87ab96625b63f1a045bea20" exitCode=0
Jul 30 04:24:06.041455 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:06.041309    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c7b98-5fcdd5d79b-5p6hd" event={"ID":"2b796602-5c31-40a7-b39a-c35b2c993ed0","Type":"ContainerDied","Data":"fee2c989e8577f4b3b75ca803a5e05434011cf20a87ab96625b63f1a045bea20"}
Jul 30 04:24:06.099950 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:06.099906    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c7b98-5fcdd5d79b-5p6hd"
Jul 30 04:24:06.271983 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:06.271946    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wxg4n\" (UniqueName: \"kubernetes.io/projected/2b796602-5c31-40a7-b39a-c35b2c993ed0-kube-api-access-wxg4n\") pod \"2b796602-5c31-40a7-b39a-c35b2c993ed0\" (UID: \"2b796602-5c31-40a7-b39a-c35b2c993ed0\") "
Jul 30 04:24:06.272173 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:06.272009    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b796602-5c31-40a7-b39a-c35b2c993ed0-plugins-config-volume\") pod \"2b796602-5c31-40a7-b39a-c35b2c993ed0\" (UID: \"2b796602-5c31-40a7-b39a-c35b2c993ed0\") "
Jul 30 04:24:06.272398 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:06.272375    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b796602-5c31-40a7-b39a-c35b2c993ed0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2b796602-5c31-40a7-b39a-c35b2c993ed0" (UID: "2b796602-5c31-40a7-b39a-c35b2c993ed0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:24:06.274116 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:06.274090    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b796602-5c31-40a7-b39a-c35b2c993ed0-kube-api-access-wxg4n" (OuterVolumeSpecName: "kube-api-access-wxg4n") pod "2b796602-5c31-40a7-b39a-c35b2c993ed0" (UID: "2b796602-5c31-40a7-b39a-c35b2c993ed0"). InnerVolumeSpecName "kube-api-access-wxg4n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:24:06.359386 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:06.359299    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:06.373424 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:06.373396    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxg4n\" (UniqueName: \"kubernetes.io/projected/2b796602-5c31-40a7-b39a-c35b2c993ed0-kube-api-access-wxg4n\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:24:06.373424 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:06.373421    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b796602-5c31-40a7-b39a-c35b2c993ed0-plugins-config-volume\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:24:07.047996 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:07.047966    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c7b98-5fcdd5d79b-5p6hd"
Jul 30 04:24:07.052777 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:07.052747    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c7b98-5fcdd5d79b-5p6hd" event={"ID":"2b796602-5c31-40a7-b39a-c35b2c993ed0","Type":"ContainerDied","Data":"2c45b9cfecdeb822093bce723e70c7ef6bcf39ba0ce665a8e6a756d4124d8bef"}
Jul 30 04:24:07.052958 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:07.052801    2576 scope.go:117] "RemoveContainer" containerID="fee2c989e8577f4b3b75ca803a5e05434011cf20a87ab96625b63f1a045bea20"
Jul 30 04:24:07.073388 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:07.073365    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-9c7b98-5fcdd5d79b-5p6hd"]
Jul 30 04:24:07.075487 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:07.075466    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-9c7b98-5fcdd5d79b-5p6hd"]
Jul 30 04:24:09.051847 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:09.051815    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b796602-5c31-40a7-b39a-c35b2c993ed0" path="/var/lib/kubelet/pods/2b796602-5c31-40a7-b39a-c35b2c993ed0/volumes"
Jul 30 04:24:12.047844 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:12.047812    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:12.159226 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:12.159187    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:12.231426 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:12.231392    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:24:12.260177 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:12.260148    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"]
Jul 30 04:24:12.260386 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:12.260361    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn" podUID="07f7c7ec-b1b6-4a89-a761-08c3af23436c" containerName="istio-proxy" containerID="cri-o://5be2bcabb95a98432df2e5c60a5b8cbf72422261dc2ed144e4ed0dc4d382cdcd" gracePeriod=30
Jul 30 04:24:12.269154 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:12.269125    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:12.493778 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:12.493684    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:12.980154 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:12.980119    2576 patch_prober.go:28] interesting pod/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 30 04:24:12.980328 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:12.980187    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn" podUID="07f7c7ec-b1b6-4a89-a761-08c3af23436c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 04:24:17.511974 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:17.511950    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:24:17.582888 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:17.582798    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07f7c7ec-b1b6-4a89-a761-08c3af23436c-istiod-ca-cert\") pod \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\" (UID: \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\") "
Jul 30 04:24:17.582888 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:17.582841    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07f7c7ec-b1b6-4a89-a761-08c3af23436c-workload-certs\") pod \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\" (UID: \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\") "
Jul 30 04:24:17.582888 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:17.582870    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07f7c7ec-b1b6-4a89-a761-08c3af23436c-istio-podinfo\") pod \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\" (UID: \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\") "
Jul 30 04:24:17.583214 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:17.582893    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f7c7ec-b1b6-4a89-a761-08c3af23436c-credential-socket\") pod \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\" (UID: \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\") "
Jul 30 04:24:17.583214 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:17.582958    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07f7c7ec-b1b6-4a89-a761-08c3af23436c-istio-data\") pod \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\" (UID: \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\") "
Jul 30 04:24:17.583214 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:17.582992    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07f7c7ec-b1b6-4a89-a761-08c3af23436c-istio-envoy\") pod \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\" (UID: \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\") "
Jul 30 04:24:17.583214 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:17.583035    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bwczm\" (UniqueName: \"kubernetes.io/projected/07f7c7ec-b1b6-4a89-a761-08c3af23436c-kube-api-access-bwczm\") pod \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\" (UID: \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\") "
Jul 30 04:24:17.583214 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:17.583063    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07f7c7ec-b1b6-4a89-a761-08c3af23436c-istio-token\") pod \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\" (UID: \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\") "
Jul 30 04:24:17.583214 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:17.583088    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f7c7ec-b1b6-4a89-a761-08c3af23436c-workload-socket\") pod \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\" (UID: \"07f7c7ec-b1b6-4a89-a761-08c3af23436c\") "
Jul 30 04:24:17.583515 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:17.583211    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07f7c7ec-b1b6-4a89-a761-08c3af23436c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "07f7c7ec-b1b6-4a89-a761-08c3af23436c" (UID: "07f7c7ec-b1b6-4a89-a761-08c3af23436c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:24:17.583515 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:17.583221    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07f7c7ec-b1b6-4a89-a761-08c3af23436c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "07f7c7ec-b1b6-4a89-a761-08c3af23436c" (UID: "07f7c7ec-b1b6-4a89-a761-08c3af23436c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:24:17.583515 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:17.583297    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07f7c7ec-b1b6-4a89-a761-08c3af23436c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "07f7c7ec-b1b6-4a89-a761-08c3af23436c" (UID: "07f7c7ec-b1b6-4a89-a761-08c3af23436c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:24:17.583515 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:17.583386    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07f7c7ec-b1b6-4a89-a761-08c3af23436c-workload-certs\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:24:17.583515 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:17.583480    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07f7c7ec-b1b6-4a89-a761-08c3af23436c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "07f7c7ec-b1b6-4a89-a761-08c3af23436c" (UID: "07f7c7ec-b1b6-4a89-a761-08c3af23436c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:24:17.584247 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:17.584223    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07f7c7ec-b1b6-4a89-a761-08c3af23436c-istio-data" (OuterVolumeSpecName: "istio-data") pod "07f7c7ec-b1b6-4a89-a761-08c3af23436c" (UID: "07f7c7ec-b1b6-4a89-a761-08c3af23436c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:24:17.585191 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:17.585165    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/07f7c7ec-b1b6-4a89-a761-08c3af23436c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "07f7c7ec-b1b6-4a89-a761-08c3af23436c" (UID: "07f7c7ec-b1b6-4a89-a761-08c3af23436c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 04:24:17.585336 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:17.585305    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07f7c7ec-b1b6-4a89-a761-08c3af23436c-kube-api-access-bwczm" (OuterVolumeSpecName: "kube-api-access-bwczm") pod "07f7c7ec-b1b6-4a89-a761-08c3af23436c" (UID: "07f7c7ec-b1b6-4a89-a761-08c3af23436c"). InnerVolumeSpecName "kube-api-access-bwczm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:24:17.585599 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:17.585579    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07f7c7ec-b1b6-4a89-a761-08c3af23436c-istio-token" (OuterVolumeSpecName: "istio-token") pod "07f7c7ec-b1b6-4a89-a761-08c3af23436c" (UID: "07f7c7ec-b1b6-4a89-a761-08c3af23436c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:24:17.585599 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:17.585581    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07f7c7ec-b1b6-4a89-a761-08c3af23436c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "07f7c7ec-b1b6-4a89-a761-08c3af23436c" (UID: "07f7c7ec-b1b6-4a89-a761-08c3af23436c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:24:17.684314 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:17.684275    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07f7c7ec-b1b6-4a89-a761-08c3af23436c-istiod-ca-cert\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:24:17.684314 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:17.684305    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07f7c7ec-b1b6-4a89-a761-08c3af23436c-istio-podinfo\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:24:17.684314 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:17.684314    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f7c7ec-b1b6-4a89-a761-08c3af23436c-credential-socket\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:24:17.684314 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:17.684324    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07f7c7ec-b1b6-4a89-a761-08c3af23436c-istio-data\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:24:17.684587 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:17.684333    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07f7c7ec-b1b6-4a89-a761-08c3af23436c-istio-envoy\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:24:17.684587 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:17.684340    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bwczm\" (UniqueName: \"kubernetes.io/projected/07f7c7ec-b1b6-4a89-a761-08c3af23436c-kube-api-access-bwczm\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:24:17.684587 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:17.684349    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07f7c7ec-b1b6-4a89-a761-08c3af23436c-istio-token\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:24:17.684587 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:17.684357    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f7c7ec-b1b6-4a89-a761-08c3af23436c-workload-socket\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:24:18.105441 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:18.105406    2576 generic.go:356] "Generic (PLEG): container finished" podID="07f7c7ec-b1b6-4a89-a761-08c3af23436c" containerID="5be2bcabb95a98432df2e5c60a5b8cbf72422261dc2ed144e4ed0dc4d382cdcd" exitCode=0
Jul 30 04:24:18.105604 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:18.105453    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"
Jul 30 04:24:18.105604 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:18.105488    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn" event={"ID":"07f7c7ec-b1b6-4a89-a761-08c3af23436c","Type":"ContainerDied","Data":"5be2bcabb95a98432df2e5c60a5b8cbf72422261dc2ed144e4ed0dc4d382cdcd"}
Jul 30 04:24:18.105604 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:18.105558    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn" event={"ID":"07f7c7ec-b1b6-4a89-a761-08c3af23436c","Type":"ContainerDied","Data":"0b37ccd6ccabf9184cd384716647edd617bc3971a76b0dc0a2983435164b93f7"}
Jul 30 04:24:18.105604 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:18.105589    2576 scope.go:117] "RemoveContainer" containerID="5be2bcabb95a98432df2e5c60a5b8cbf72422261dc2ed144e4ed0dc4d382cdcd"
Jul 30 04:24:18.118150 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:18.118067    2576 scope.go:117] "RemoveContainer" containerID="a4f3926cbe5f329e485bfddd3f776555443c8c5934ee9a8dbefd109c7b178a78"
Jul 30 04:24:18.126915 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:18.126895    2576 scope.go:117] "RemoveContainer" containerID="5be2bcabb95a98432df2e5c60a5b8cbf72422261dc2ed144e4ed0dc4d382cdcd"
Jul 30 04:24:18.127196 ip-10-0-143-117 kubenswrapper[2576]: E0730 04:24:18.127179    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5be2bcabb95a98432df2e5c60a5b8cbf72422261dc2ed144e4ed0dc4d382cdcd\": container with ID starting with 5be2bcabb95a98432df2e5c60a5b8cbf72422261dc2ed144e4ed0dc4d382cdcd not found: ID does not exist" containerID="5be2bcabb95a98432df2e5c60a5b8cbf72422261dc2ed144e4ed0dc4d382cdcd"
Jul 30 04:24:18.127248 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:18.127205    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5be2bcabb95a98432df2e5c60a5b8cbf72422261dc2ed144e4ed0dc4d382cdcd"} err="failed to get container status \"5be2bcabb95a98432df2e5c60a5b8cbf72422261dc2ed144e4ed0dc4d382cdcd\": rpc error: code = NotFound desc = could not find container \"5be2bcabb95a98432df2e5c60a5b8cbf72422261dc2ed144e4ed0dc4d382cdcd\": container with ID starting with 5be2bcabb95a98432df2e5c60a5b8cbf72422261dc2ed144e4ed0dc4d382cdcd not found: ID does not exist"
Jul 30 04:24:18.127248 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:18.127223    2576 scope.go:117] "RemoveContainer" containerID="a4f3926cbe5f329e485bfddd3f776555443c8c5934ee9a8dbefd109c7b178a78"
Jul 30 04:24:18.127441 ip-10-0-143-117 kubenswrapper[2576]: E0730 04:24:18.127427    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4f3926cbe5f329e485bfddd3f776555443c8c5934ee9a8dbefd109c7b178a78\": container with ID starting with a4f3926cbe5f329e485bfddd3f776555443c8c5934ee9a8dbefd109c7b178a78 not found: ID does not exist" containerID="a4f3926cbe5f329e485bfddd3f776555443c8c5934ee9a8dbefd109c7b178a78"
Jul 30 04:24:18.127484 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:18.127443    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4f3926cbe5f329e485bfddd3f776555443c8c5934ee9a8dbefd109c7b178a78"} err="failed to get container status \"a4f3926cbe5f329e485bfddd3f776555443c8c5934ee9a8dbefd109c7b178a78\": rpc error: code = NotFound desc = could not find container \"a4f3926cbe5f329e485bfddd3f776555443c8c5934ee9a8dbefd109c7b178a78\": container with ID starting with a4f3926cbe5f329e485bfddd3f776555443c8c5934ee9a8dbefd109c7b178a78 not found: ID does not exist"
Jul 30 04:24:18.128913 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:18.128893    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"]
Jul 30 04:24:18.133078 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:18.133058    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-9c7b98-openshift-default-7b9f8c94b4-g7gnn"]
Jul 30 04:24:19.052091 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:19.052058    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07f7c7ec-b1b6-4a89-a761-08c3af23436c" path="/var/lib/kubelet/pods/07f7c7ec-b1b6-4a89-a761-08c3af23436c/volumes"
Jul 30 04:24:28.202704 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:28.202667    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:28.238586 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:28.238557    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-2a2d39-8f8cbfbbd-6gmqc"]
Jul 30 04:24:28.238791 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:28.238764    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-2a2d39-8f8cbfbbd-6gmqc" podUID="c11f29f9-a542-4ce7-b3a2-6b4147891549" containerName="payload-processing" containerID="cri-o://f9dbd43d5b6a6c3ba5d6d2085e2cf3bcb7e37946380cf8ae8d4ad48c4296fa94" gracePeriod=30
Jul 30 04:24:28.280129 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:28.280096    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:28.338063 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:28.338032    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:28.400508 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:28.400478    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:28.500559 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:28.500535    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-2a2d39-8f8cbfbbd-6gmqc"
Jul 30 04:24:28.583155 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:28.583119    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qmp2s\" (UniqueName: \"kubernetes.io/projected/c11f29f9-a542-4ce7-b3a2-6b4147891549-kube-api-access-qmp2s\") pod \"c11f29f9-a542-4ce7-b3a2-6b4147891549\" (UID: \"c11f29f9-a542-4ce7-b3a2-6b4147891549\") "
Jul 30 04:24:28.583329 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:28.583240    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c11f29f9-a542-4ce7-b3a2-6b4147891549-plugins-config-volume\") pod \"c11f29f9-a542-4ce7-b3a2-6b4147891549\" (UID: \"c11f29f9-a542-4ce7-b3a2-6b4147891549\") "
Jul 30 04:24:28.583619 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:28.583593    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c11f29f9-a542-4ce7-b3a2-6b4147891549-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c11f29f9-a542-4ce7-b3a2-6b4147891549" (UID: "c11f29f9-a542-4ce7-b3a2-6b4147891549"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:24:28.585225 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:28.585191    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c11f29f9-a542-4ce7-b3a2-6b4147891549-kube-api-access-qmp2s" (OuterVolumeSpecName: "kube-api-access-qmp2s") pod "c11f29f9-a542-4ce7-b3a2-6b4147891549" (UID: "c11f29f9-a542-4ce7-b3a2-6b4147891549"). InnerVolumeSpecName "kube-api-access-qmp2s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:24:28.684195 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:28.684163    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmp2s\" (UniqueName: \"kubernetes.io/projected/c11f29f9-a542-4ce7-b3a2-6b4147891549-kube-api-access-qmp2s\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:24:28.684195 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:28.684190    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c11f29f9-a542-4ce7-b3a2-6b4147891549-plugins-config-volume\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:24:28.791117 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:28.791086    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:29.165387 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:29.165346    2576 generic.go:356] "Generic (PLEG): container finished" podID="c11f29f9-a542-4ce7-b3a2-6b4147891549" containerID="f9dbd43d5b6a6c3ba5d6d2085e2cf3bcb7e37946380cf8ae8d4ad48c4296fa94" exitCode=0
Jul 30 04:24:29.165571 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:29.165408    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-2a2d39-8f8cbfbbd-6gmqc"
Jul 30 04:24:29.165571 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:29.165425    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-2a2d39-8f8cbfbbd-6gmqc" event={"ID":"c11f29f9-a542-4ce7-b3a2-6b4147891549","Type":"ContainerDied","Data":"f9dbd43d5b6a6c3ba5d6d2085e2cf3bcb7e37946380cf8ae8d4ad48c4296fa94"}
Jul 30 04:24:29.165571 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:29.165509    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-2a2d39-8f8cbfbbd-6gmqc" event={"ID":"c11f29f9-a542-4ce7-b3a2-6b4147891549","Type":"ContainerDied","Data":"d86b4cb377c480dd8d37a8f5ee49bf436414993316f1f265ac5e6f934eb75e8e"}
Jul 30 04:24:29.165571 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:29.165533    2576 scope.go:117] "RemoveContainer" containerID="f9dbd43d5b6a6c3ba5d6d2085e2cf3bcb7e37946380cf8ae8d4ad48c4296fa94"
Jul 30 04:24:29.174471 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:29.174454    2576 scope.go:117] "RemoveContainer" containerID="f9dbd43d5b6a6c3ba5d6d2085e2cf3bcb7e37946380cf8ae8d4ad48c4296fa94"
Jul 30 04:24:29.174740 ip-10-0-143-117 kubenswrapper[2576]: E0730 04:24:29.174722    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9dbd43d5b6a6c3ba5d6d2085e2cf3bcb7e37946380cf8ae8d4ad48c4296fa94\": container with ID starting with f9dbd43d5b6a6c3ba5d6d2085e2cf3bcb7e37946380cf8ae8d4ad48c4296fa94 not found: ID does not exist" containerID="f9dbd43d5b6a6c3ba5d6d2085e2cf3bcb7e37946380cf8ae8d4ad48c4296fa94"
Jul 30 04:24:29.174801 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:29.174751    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9dbd43d5b6a6c3ba5d6d2085e2cf3bcb7e37946380cf8ae8d4ad48c4296fa94"} err="failed to get container status \"f9dbd43d5b6a6c3ba5d6d2085e2cf3bcb7e37946380cf8ae8d4ad48c4296fa94\": rpc error: code = NotFound desc = could not find container \"f9dbd43d5b6a6c3ba5d6d2085e2cf3bcb7e37946380cf8ae8d4ad48c4296fa94\": container with ID starting with f9dbd43d5b6a6c3ba5d6d2085e2cf3bcb7e37946380cf8ae8d4ad48c4296fa94 not found: ID does not exist"
Jul 30 04:24:29.182698 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:29.182675    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-2a2d39-8f8cbfbbd-6gmqc"]
Jul 30 04:24:29.184801 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:29.184782    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-2a2d39-8f8cbfbbd-6gmqc"]
Jul 30 04:24:31.051726 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:31.051693    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c11f29f9-a542-4ce7-b3a2-6b4147891549" path="/var/lib/kubelet/pods/c11f29f9-a542-4ce7-b3a2-6b4147891549/volumes"
Jul 30 04:24:33.333301 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:33.333270    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:24:34.734879 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:34.734843    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:34.835619 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:34.835584    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:34.973902 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:34.968298    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"]
Jul 30 04:24:34.973902 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:34.968618    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd" podUID="a05e918c-cf1c-48a5-b57f-888f27c239e3" containerName="istio-proxy" containerID="cri-o://1053b6384503cb363967695abd19a43cbd824a6c7e3066d12dbb151b0b94a7ba" gracePeriod=30
Jul 30 04:24:34.990705 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:34.990609    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:35.202281 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.202236    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:35.533703 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.533669    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:35.779817 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.779786    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"]
Jul 30 04:24:35.780206 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.780185    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07f7c7ec-b1b6-4a89-a761-08c3af23436c" containerName="istio-proxy"
Jul 30 04:24:35.780206 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.780198    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="07f7c7ec-b1b6-4a89-a761-08c3af23436c" containerName="istio-proxy"
Jul 30 04:24:35.780206 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.780209    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c11f29f9-a542-4ce7-b3a2-6b4147891549" containerName="payload-processing"
Jul 30 04:24:35.780314 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.780215    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c11f29f9-a542-4ce7-b3a2-6b4147891549" containerName="payload-processing"
Jul 30 04:24:35.780314 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.780226    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b796602-5c31-40a7-b39a-c35b2c993ed0" containerName="payload-pre-processing"
Jul 30 04:24:35.780314 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.780231    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b796602-5c31-40a7-b39a-c35b2c993ed0" containerName="payload-pre-processing"
Jul 30 04:24:35.780314 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.780240    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07f7c7ec-b1b6-4a89-a761-08c3af23436c" containerName="istio-proxy"
Jul 30 04:24:35.780314 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.780245    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="07f7c7ec-b1b6-4a89-a761-08c3af23436c" containerName="istio-proxy"
Jul 30 04:24:35.780314 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.780299    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="07f7c7ec-b1b6-4a89-a761-08c3af23436c" containerName="istio-proxy"
Jul 30 04:24:35.780314 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.780306    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b796602-5c31-40a7-b39a-c35b2c993ed0" containerName="payload-pre-processing"
Jul 30 04:24:35.780314 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.780313    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c11f29f9-a542-4ce7-b3a2-6b4147891549" containerName="payload-processing"
Jul 30 04:24:35.780549 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.780322    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="07f7c7ec-b1b6-4a89-a761-08c3af23436c" containerName="istio-proxy"
Jul 30 04:24:35.783189 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.783174    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:24:35.785713 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.785663    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-19ef02-openshift-default-dockercfg-2fmrb\""
Jul 30 04:24:35.797001 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.796983    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"]
Jul 30 04:24:35.807536 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.807508    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:35.847835 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.847796    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a596f883-2431-4345-aaba-d22c0f30ae8a-istio-token\") pod \"e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh\" (UID: \"a596f883-2431-4345-aaba-d22c0f30ae8a\") " pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:24:35.847835 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.847827    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a596f883-2431-4345-aaba-d22c0f30ae8a-istio-podinfo\") pod \"e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh\" (UID: \"a596f883-2431-4345-aaba-d22c0f30ae8a\") " pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:24:35.848038 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.847851    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a596f883-2431-4345-aaba-d22c0f30ae8a-workload-certs\") pod \"e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh\" (UID: \"a596f883-2431-4345-aaba-d22c0f30ae8a\") " pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:24:35.848038 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.847874    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a596f883-2431-4345-aaba-d22c0f30ae8a-workload-socket\") pod \"e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh\" (UID: \"a596f883-2431-4345-aaba-d22c0f30ae8a\") " pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:24:35.848038 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.847890    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a596f883-2431-4345-aaba-d22c0f30ae8a-istiod-ca-cert\") pod \"e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh\" (UID: \"a596f883-2431-4345-aaba-d22c0f30ae8a\") " pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:24:35.848038 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.847910    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qft77\" (UniqueName: \"kubernetes.io/projected/a596f883-2431-4345-aaba-d22c0f30ae8a-kube-api-access-qft77\") pod \"e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh\" (UID: \"a596f883-2431-4345-aaba-d22c0f30ae8a\") " pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:24:35.848038 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.848005    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a596f883-2431-4345-aaba-d22c0f30ae8a-credential-socket\") pod \"e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh\" (UID: \"a596f883-2431-4345-aaba-d22c0f30ae8a\") " pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:24:35.848206 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.848062    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a596f883-2431-4345-aaba-d22c0f30ae8a-istio-envoy\") pod \"e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh\" (UID: \"a596f883-2431-4345-aaba-d22c0f30ae8a\") " pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:24:35.848206 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.848113    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a596f883-2431-4345-aaba-d22c0f30ae8a-istio-data\") pod \"e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh\" (UID: \"a596f883-2431-4345-aaba-d22c0f30ae8a\") " pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:24:35.948980 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.948948    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a596f883-2431-4345-aaba-d22c0f30ae8a-istio-data\") pod \"e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh\" (UID: \"a596f883-2431-4345-aaba-d22c0f30ae8a\") " pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:24:35.949140 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.948995    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a596f883-2431-4345-aaba-d22c0f30ae8a-istio-token\") pod \"e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh\" (UID: \"a596f883-2431-4345-aaba-d22c0f30ae8a\") " pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:24:35.949140 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.949100    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a596f883-2431-4345-aaba-d22c0f30ae8a-istio-podinfo\") pod \"e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh\" (UID: \"a596f883-2431-4345-aaba-d22c0f30ae8a\") " pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:24:35.949215 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.949151    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a596f883-2431-4345-aaba-d22c0f30ae8a-workload-certs\") pod \"e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh\" (UID: \"a596f883-2431-4345-aaba-d22c0f30ae8a\") " pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:24:35.949215 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.949190    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a596f883-2431-4345-aaba-d22c0f30ae8a-workload-socket\") pod \"e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh\" (UID: \"a596f883-2431-4345-aaba-d22c0f30ae8a\") " pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:24:35.949215 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.949212    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a596f883-2431-4345-aaba-d22c0f30ae8a-istiod-ca-cert\") pod \"e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh\" (UID: \"a596f883-2431-4345-aaba-d22c0f30ae8a\") " pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:24:35.949358 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.949250    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qft77\" (UniqueName: \"kubernetes.io/projected/a596f883-2431-4345-aaba-d22c0f30ae8a-kube-api-access-qft77\") pod \"e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh\" (UID: \"a596f883-2431-4345-aaba-d22c0f30ae8a\") " pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:24:35.949358 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.949311    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a596f883-2431-4345-aaba-d22c0f30ae8a-credential-socket\") pod \"e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh\" (UID: \"a596f883-2431-4345-aaba-d22c0f30ae8a\") " pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:24:35.949441 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.949390    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a596f883-2431-4345-aaba-d22c0f30ae8a-istio-envoy\") pod \"e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh\" (UID: \"a596f883-2431-4345-aaba-d22c0f30ae8a\") " pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:24:35.949565 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.949538    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a596f883-2431-4345-aaba-d22c0f30ae8a-workload-certs\") pod \"e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh\" (UID: \"a596f883-2431-4345-aaba-d22c0f30ae8a\") " pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:24:35.949649 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.949579    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a596f883-2431-4345-aaba-d22c0f30ae8a-workload-socket\") pod \"e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh\" (UID: \"a596f883-2431-4345-aaba-d22c0f30ae8a\") " pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:24:35.949706 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.949677    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a596f883-2431-4345-aaba-d22c0f30ae8a-istio-data\") pod \"e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh\" (UID: \"a596f883-2431-4345-aaba-d22c0f30ae8a\") " pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:24:35.949793 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.949774    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a596f883-2431-4345-aaba-d22c0f30ae8a-credential-socket\") pod \"e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh\" (UID: \"a596f883-2431-4345-aaba-d22c0f30ae8a\") " pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:24:35.949900 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.949880    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a596f883-2431-4345-aaba-d22c0f30ae8a-istiod-ca-cert\") pod \"e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh\" (UID: \"a596f883-2431-4345-aaba-d22c0f30ae8a\") " pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:24:35.951387 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.951366    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a596f883-2431-4345-aaba-d22c0f30ae8a-istio-envoy\") pod \"e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh\" (UID: \"a596f883-2431-4345-aaba-d22c0f30ae8a\") " pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:24:35.951484 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.951439    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a596f883-2431-4345-aaba-d22c0f30ae8a-istio-podinfo\") pod \"e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh\" (UID: \"a596f883-2431-4345-aaba-d22c0f30ae8a\") " pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:24:35.956590 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.956569    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a596f883-2431-4345-aaba-d22c0f30ae8a-istio-token\") pod \"e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh\" (UID: \"a596f883-2431-4345-aaba-d22c0f30ae8a\") " pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:24:35.956778 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:35.956761    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qft77\" (UniqueName: \"kubernetes.io/projected/a596f883-2431-4345-aaba-d22c0f30ae8a-kube-api-access-qft77\") pod \"e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh\" (UID: \"a596f883-2431-4345-aaba-d22c0f30ae8a\") " pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:24:36.082087 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:36.082003    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:36.092970 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:36.092934    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:24:36.231488 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:36.228828    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"]
Jul 30 04:24:36.232672 ip-10-0-143-117 kubenswrapper[2576]: W0730 04:24:36.232626    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda596f883_2431_4345_aaba_d22c0f30ae8a.slice/crio-840a7bb685a3049349f0e3a7e6b0f14d7af02dcbb0cdb9daf2f0751f772b1a9c WatchSource:0}: Error finding container 840a7bb685a3049349f0e3a7e6b0f14d7af02dcbb0cdb9daf2f0751f772b1a9c: Status 404 returned error can't find the container with id 840a7bb685a3049349f0e3a7e6b0f14d7af02dcbb0cdb9daf2f0751f772b1a9c
Jul 30 04:24:36.234994 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:36.234961    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:24:36.235083 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:36.235029    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:24:36.235083 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:36.235059    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:24:37.211720 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:37.211688    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh" event={"ID":"a596f883-2431-4345-aaba-d22c0f30ae8a","Type":"ContainerStarted","Data":"c880b496ff64ac85b1360e5c42da8d40fe9f6573a3c9316342c628ab44f81eae"}
Jul 30 04:24:37.212134 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:37.211737    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh" event={"ID":"a596f883-2431-4345-aaba-d22c0f30ae8a","Type":"ContainerStarted","Data":"840a7bb685a3049349f0e3a7e6b0f14d7af02dcbb0cdb9daf2f0751f772b1a9c"}
Jul 30 04:24:37.232358 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:37.232305    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh" podStartSLOduration=2.232288832 podStartE2EDuration="2.232288832s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:24:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:24:37.229989684 +0000 UTC m=+2982.848221926" watchObservedRunningTime="2026-07-30 04:24:37.232288832 +0000 UTC m=+2982.850521074"
Jul 30 04:24:38.093992 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:38.093964    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:24:38.098403 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:38.098380    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:24:38.216648 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:38.216616    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:24:38.217685 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:38.217663    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:24:39.101270 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:39.101239    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:39.246420 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:39.246386    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:40.225521 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.225497    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:24:40.228107 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.228081    2576 generic.go:356] "Generic (PLEG): container finished" podID="a05e918c-cf1c-48a5-b57f-888f27c239e3" containerID="1053b6384503cb363967695abd19a43cbd824a6c7e3066d12dbb151b0b94a7ba" exitCode=0
Jul 30 04:24:40.228212 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.228119    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"
Jul 30 04:24:40.228212 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.228157    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd" event={"ID":"a05e918c-cf1c-48a5-b57f-888f27c239e3","Type":"ContainerDied","Data":"1053b6384503cb363967695abd19a43cbd824a6c7e3066d12dbb151b0b94a7ba"}
Jul 30 04:24:40.228301 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.228211    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd" event={"ID":"a05e918c-cf1c-48a5-b57f-888f27c239e3","Type":"ContainerDied","Data":"7da329d21397ad5f0ca1cba4b989f801718469f678151287cb839749e00fcb14"}
Jul 30 04:24:40.228301 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.228235    2576 scope.go:117] "RemoveContainer" containerID="1053b6384503cb363967695abd19a43cbd824a6c7e3066d12dbb151b0b94a7ba"
Jul 30 04:24:40.231555 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.231528    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:40.237176 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.237157    2576 scope.go:117] "RemoveContainer" containerID="731ee09c89cc03a1ac16fc2e906031af61bdd301dd524aef92f7e41ed21a3a43"
Jul 30 04:24:40.245292 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.245277    2576 scope.go:117] "RemoveContainer" containerID="1053b6384503cb363967695abd19a43cbd824a6c7e3066d12dbb151b0b94a7ba"
Jul 30 04:24:40.245571 ip-10-0-143-117 kubenswrapper[2576]: E0730 04:24:40.245544    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1053b6384503cb363967695abd19a43cbd824a6c7e3066d12dbb151b0b94a7ba\": container with ID starting with 1053b6384503cb363967695abd19a43cbd824a6c7e3066d12dbb151b0b94a7ba not found: ID does not exist" containerID="1053b6384503cb363967695abd19a43cbd824a6c7e3066d12dbb151b0b94a7ba"
Jul 30 04:24:40.245618 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.245580    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1053b6384503cb363967695abd19a43cbd824a6c7e3066d12dbb151b0b94a7ba"} err="failed to get container status \"1053b6384503cb363967695abd19a43cbd824a6c7e3066d12dbb151b0b94a7ba\": rpc error: code = NotFound desc = could not find container \"1053b6384503cb363967695abd19a43cbd824a6c7e3066d12dbb151b0b94a7ba\": container with ID starting with 1053b6384503cb363967695abd19a43cbd824a6c7e3066d12dbb151b0b94a7ba not found: ID does not exist"
Jul 30 04:24:40.245618 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.245598    2576 scope.go:117] "RemoveContainer" containerID="731ee09c89cc03a1ac16fc2e906031af61bdd301dd524aef92f7e41ed21a3a43"
Jul 30 04:24:40.245847 ip-10-0-143-117 kubenswrapper[2576]: E0730 04:24:40.245830    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"731ee09c89cc03a1ac16fc2e906031af61bdd301dd524aef92f7e41ed21a3a43\": container with ID starting with 731ee09c89cc03a1ac16fc2e906031af61bdd301dd524aef92f7e41ed21a3a43 not found: ID does not exist" containerID="731ee09c89cc03a1ac16fc2e906031af61bdd301dd524aef92f7e41ed21a3a43"
Jul 30 04:24:40.245888 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.245857    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"731ee09c89cc03a1ac16fc2e906031af61bdd301dd524aef92f7e41ed21a3a43"} err="failed to get container status \"731ee09c89cc03a1ac16fc2e906031af61bdd301dd524aef92f7e41ed21a3a43\": rpc error: code = NotFound desc = could not find container \"731ee09c89cc03a1ac16fc2e906031af61bdd301dd524aef92f7e41ed21a3a43\": container with ID starting with 731ee09c89cc03a1ac16fc2e906031af61bdd301dd524aef92f7e41ed21a3a43 not found: ID does not exist"
Jul 30 04:24:40.392475 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.392449    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tmlhd\" (UniqueName: \"kubernetes.io/projected/a05e918c-cf1c-48a5-b57f-888f27c239e3-kube-api-access-tmlhd\") pod \"a05e918c-cf1c-48a5-b57f-888f27c239e3\" (UID: \"a05e918c-cf1c-48a5-b57f-888f27c239e3\") "
Jul 30 04:24:40.392956 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.392503    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a05e918c-cf1c-48a5-b57f-888f27c239e3-istio-envoy\") pod \"a05e918c-cf1c-48a5-b57f-888f27c239e3\" (UID: \"a05e918c-cf1c-48a5-b57f-888f27c239e3\") "
Jul 30 04:24:40.392956 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.392529    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a05e918c-cf1c-48a5-b57f-888f27c239e3-istio-data\") pod \"a05e918c-cf1c-48a5-b57f-888f27c239e3\" (UID: \"a05e918c-cf1c-48a5-b57f-888f27c239e3\") "
Jul 30 04:24:40.392956 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.392559    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a05e918c-cf1c-48a5-b57f-888f27c239e3-credential-socket\") pod \"a05e918c-cf1c-48a5-b57f-888f27c239e3\" (UID: \"a05e918c-cf1c-48a5-b57f-888f27c239e3\") "
Jul 30 04:24:40.392956 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.392584    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a05e918c-cf1c-48a5-b57f-888f27c239e3-istio-token\") pod \"a05e918c-cf1c-48a5-b57f-888f27c239e3\" (UID: \"a05e918c-cf1c-48a5-b57f-888f27c239e3\") "
Jul 30 04:24:40.392956 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.392628    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a05e918c-cf1c-48a5-b57f-888f27c239e3-istio-podinfo\") pod \"a05e918c-cf1c-48a5-b57f-888f27c239e3\" (UID: \"a05e918c-cf1c-48a5-b57f-888f27c239e3\") "
Jul 30 04:24:40.392956 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.392656    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a05e918c-cf1c-48a5-b57f-888f27c239e3-istiod-ca-cert\") pod \"a05e918c-cf1c-48a5-b57f-888f27c239e3\" (UID: \"a05e918c-cf1c-48a5-b57f-888f27c239e3\") "
Jul 30 04:24:40.392956 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.392695    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a05e918c-cf1c-48a5-b57f-888f27c239e3-workload-socket\") pod \"a05e918c-cf1c-48a5-b57f-888f27c239e3\" (UID: \"a05e918c-cf1c-48a5-b57f-888f27c239e3\") "
Jul 30 04:24:40.392956 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.392736    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a05e918c-cf1c-48a5-b57f-888f27c239e3-workload-certs\") pod \"a05e918c-cf1c-48a5-b57f-888f27c239e3\" (UID: \"a05e918c-cf1c-48a5-b57f-888f27c239e3\") "
Jul 30 04:24:40.392956 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.392904    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a05e918c-cf1c-48a5-b57f-888f27c239e3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a05e918c-cf1c-48a5-b57f-888f27c239e3" (UID: "a05e918c-cf1c-48a5-b57f-888f27c239e3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:24:40.393458 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.392978    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a05e918c-cf1c-48a5-b57f-888f27c239e3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a05e918c-cf1c-48a5-b57f-888f27c239e3" (UID: "a05e918c-cf1c-48a5-b57f-888f27c239e3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:24:40.393458 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.393078    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a05e918c-cf1c-48a5-b57f-888f27c239e3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a05e918c-cf1c-48a5-b57f-888f27c239e3" (UID: "a05e918c-cf1c-48a5-b57f-888f27c239e3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:24:40.393458 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.393233    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a05e918c-cf1c-48a5-b57f-888f27c239e3-credential-socket\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:24:40.393458 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.393254    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a05e918c-cf1c-48a5-b57f-888f27c239e3-istiod-ca-cert\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:24:40.393458 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.393269    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a05e918c-cf1c-48a5-b57f-888f27c239e3-workload-socket\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:24:40.393458 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.393366    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a05e918c-cf1c-48a5-b57f-888f27c239e3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a05e918c-cf1c-48a5-b57f-888f27c239e3" (UID: "a05e918c-cf1c-48a5-b57f-888f27c239e3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:24:40.394032 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.394011    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a05e918c-cf1c-48a5-b57f-888f27c239e3-istio-data" (OuterVolumeSpecName: "istio-data") pod "a05e918c-cf1c-48a5-b57f-888f27c239e3" (UID: "a05e918c-cf1c-48a5-b57f-888f27c239e3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:24:40.395069 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.395033    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a05e918c-cf1c-48a5-b57f-888f27c239e3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a05e918c-cf1c-48a5-b57f-888f27c239e3" (UID: "a05e918c-cf1c-48a5-b57f-888f27c239e3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:24:40.395291 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.395262    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a05e918c-cf1c-48a5-b57f-888f27c239e3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a05e918c-cf1c-48a5-b57f-888f27c239e3" (UID: "a05e918c-cf1c-48a5-b57f-888f27c239e3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 04:24:40.395367 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.395325    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a05e918c-cf1c-48a5-b57f-888f27c239e3-istio-token" (OuterVolumeSpecName: "istio-token") pod "a05e918c-cf1c-48a5-b57f-888f27c239e3" (UID: "a05e918c-cf1c-48a5-b57f-888f27c239e3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:24:40.395497 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.395477    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a05e918c-cf1c-48a5-b57f-888f27c239e3-kube-api-access-tmlhd" (OuterVolumeSpecName: "kube-api-access-tmlhd") pod "a05e918c-cf1c-48a5-b57f-888f27c239e3" (UID: "a05e918c-cf1c-48a5-b57f-888f27c239e3"). InnerVolumeSpecName "kube-api-access-tmlhd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:24:40.444806 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.444771    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:40.494741 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.494648    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a05e918c-cf1c-48a5-b57f-888f27c239e3-istio-podinfo\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:24:40.494741 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.494680    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a05e918c-cf1c-48a5-b57f-888f27c239e3-workload-certs\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:24:40.494741 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.494694    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tmlhd\" (UniqueName: \"kubernetes.io/projected/a05e918c-cf1c-48a5-b57f-888f27c239e3-kube-api-access-tmlhd\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:24:40.494741 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.494708    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a05e918c-cf1c-48a5-b57f-888f27c239e3-istio-envoy\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:24:40.494741 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.494720    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a05e918c-cf1c-48a5-b57f-888f27c239e3-istio-data\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:24:40.494741 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.494732    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a05e918c-cf1c-48a5-b57f-888f27c239e3-istio-token\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:24:40.554874 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.554840    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"]
Jul 30 04:24:40.562013 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.561980    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-2a2d39-openshift-default-c787b4964-m7sgd"]
Jul 30 04:24:40.752183 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.752082    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:40.854771 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.854742    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:24:40.866779 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.866752    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-19ef02-77955db588-b2gpj"]
Jul 30 04:24:40.867138 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.867125    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a05e918c-cf1c-48a5-b57f-888f27c239e3" containerName="istio-proxy"
Jul 30 04:24:40.867193 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.867140    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a05e918c-cf1c-48a5-b57f-888f27c239e3" containerName="istio-proxy"
Jul 30 04:24:40.867193 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.867149    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a05e918c-cf1c-48a5-b57f-888f27c239e3" containerName="istio-proxy"
Jul 30 04:24:40.867193 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.867155    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a05e918c-cf1c-48a5-b57f-888f27c239e3" containerName="istio-proxy"
Jul 30 04:24:40.867289 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.867239    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a05e918c-cf1c-48a5-b57f-888f27c239e3" containerName="istio-proxy"
Jul 30 04:24:40.871530 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.871511    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-19ef02-77955db588-b2gpj"
Jul 30 04:24:40.874703 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.874683    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-19ef02-dockercfg-nbw2s\""
Jul 30 04:24:40.874827 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.874766    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-19ef02\""
Jul 30 04:24:40.881287 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.881266    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-19ef02-77955db588-b2gpj"]
Jul 30 04:24:40.999373 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.999339    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b34a72cb-89b7-45d0-91fd-0e6add8db5b6-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-19ef02-77955db588-b2gpj\" (UID: \"b34a72cb-89b7-45d0-91fd-0e6add8db5b6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-19ef02-77955db588-b2gpj"
Jul 30 04:24:40.999567 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:40.999454    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jg6dh\" (UniqueName: \"kubernetes.io/projected/b34a72cb-89b7-45d0-91fd-0e6add8db5b6-kube-api-access-jg6dh\") pod \"payload-pre-processing-e2e-shared-a-19ef02-77955db588-b2gpj\" (UID: \"b34a72cb-89b7-45d0-91fd-0e6add8db5b6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-19ef02-77955db588-b2gpj"
Jul 30 04:24:41.052076 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:41.052048    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a05e918c-cf1c-48a5-b57f-888f27c239e3" path="/var/lib/kubelet/pods/a05e918c-cf1c-48a5-b57f-888f27c239e3/volumes"
Jul 30 04:24:41.100956 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:41.100881    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jg6dh\" (UniqueName: \"kubernetes.io/projected/b34a72cb-89b7-45d0-91fd-0e6add8db5b6-kube-api-access-jg6dh\") pod \"payload-pre-processing-e2e-shared-a-19ef02-77955db588-b2gpj\" (UID: \"b34a72cb-89b7-45d0-91fd-0e6add8db5b6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-19ef02-77955db588-b2gpj"
Jul 30 04:24:41.101170 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:41.101032    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b34a72cb-89b7-45d0-91fd-0e6add8db5b6-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-19ef02-77955db588-b2gpj\" (UID: \"b34a72cb-89b7-45d0-91fd-0e6add8db5b6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-19ef02-77955db588-b2gpj"
Jul 30 04:24:41.101719 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:41.101700    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b34a72cb-89b7-45d0-91fd-0e6add8db5b6-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-19ef02-77955db588-b2gpj\" (UID: \"b34a72cb-89b7-45d0-91fd-0e6add8db5b6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-19ef02-77955db588-b2gpj"
Jul 30 04:24:41.117050 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:41.117021    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jg6dh\" (UniqueName: \"kubernetes.io/projected/b34a72cb-89b7-45d0-91fd-0e6add8db5b6-kube-api-access-jg6dh\") pod \"payload-pre-processing-e2e-shared-a-19ef02-77955db588-b2gpj\" (UID: \"b34a72cb-89b7-45d0-91fd-0e6add8db5b6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-19ef02-77955db588-b2gpj"
Jul 30 04:24:41.183151 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:41.183117    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-19ef02-77955db588-b2gpj"
Jul 30 04:24:41.334472 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:41.334444    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-19ef02-77955db588-b2gpj"]
Jul 30 04:24:41.336531 ip-10-0-143-117 kubenswrapper[2576]: W0730 04:24:41.336495    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb34a72cb_89b7_45d0_91fd_0e6add8db5b6.slice/crio-b4d9cb6e96d122adbb8301aad9f0eda066c3cb2aec415e389385a45e1b7fd2a3 WatchSource:0}: Error finding container b4d9cb6e96d122adbb8301aad9f0eda066c3cb2aec415e389385a45e1b7fd2a3: Status 404 returned error can't find the container with id b4d9cb6e96d122adbb8301aad9f0eda066c3cb2aec415e389385a45e1b7fd2a3
Jul 30 04:24:42.243900 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:42.243862    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-19ef02-77955db588-b2gpj" event={"ID":"b34a72cb-89b7-45d0-91fd-0e6add8db5b6","Type":"ContainerStarted","Data":"7944a37a71d4f12bcf200490c141ca45a1bc926c082b92b1c1fa4e59c5147d56"}
Jul 30 04:24:42.244288 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:42.243916    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-19ef02-77955db588-b2gpj" event={"ID":"b34a72cb-89b7-45d0-91fd-0e6add8db5b6","Type":"ContainerStarted","Data":"b4d9cb6e96d122adbb8301aad9f0eda066c3cb2aec415e389385a45e1b7fd2a3"}
Jul 30 04:24:42.244288 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:42.243983    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-19ef02-77955db588-b2gpj"
Jul 30 04:24:42.261234 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:42.261187    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-19ef02-77955db588-b2gpj" podStartSLOduration=2.261173275 podStartE2EDuration="2.261173275s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:24:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:24:42.258890518 +0000 UTC m=+2987.877122759" watchObservedRunningTime="2026-07-30 04:24:42.261173275 +0000 UTC m=+2987.879405515"
Jul 30 04:24:44.926303 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:44.926199    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:24:53.217778 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:53.217738    2576 patch_prober.go:28] interesting pod/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Jul 30 04:24:53.218233 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:53.217804    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh" podUID="a596f883-2431-4345-aaba-d22c0f30ae8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Jul 30 04:24:53.250966 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:53.250917    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-19ef02-77955db588-b2gpj"
Jul 30 04:24:53.302976 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:53.302938    2576 generic.go:356] "Generic (PLEG): container finished" podID="a596f883-2431-4345-aaba-d22c0f30ae8a" containerID="c880b496ff64ac85b1360e5c42da8d40fe9f6573a3c9316342c628ab44f81eae" exitCode=0
Jul 30 04:24:53.303131 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:53.302963    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh" event={"ID":"a596f883-2431-4345-aaba-d22c0f30ae8a","Type":"ContainerDied","Data":"c880b496ff64ac85b1360e5c42da8d40fe9f6573a3c9316342c628ab44f81eae"}
Jul 30 04:24:53.303472 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:53.303456    2576 scope.go:117] "RemoveContainer" containerID="c880b496ff64ac85b1360e5c42da8d40fe9f6573a3c9316342c628ab44f81eae"
Jul 30 04:24:53.304806 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:53.304767    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:24:53.304888 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:53.304866    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:24:53.304950 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:53.304916    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:24:54.310700 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:54.310663    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh" event={"ID":"a596f883-2431-4345-aaba-d22c0f30ae8a","Type":"ContainerStarted","Data":"3ba8d8cbb1150f003dc9733ae9757b292863096e0872df578759dec66b89f177"}
Jul 30 04:24:55.093300 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:55.093268    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:24:55.093571 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:55.093480    2576 patch_prober.go:28] interesting pod/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Jul 30 04:24:55.093656 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:55.093560    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh" podUID="a596f883-2431-4345-aaba-d22c0f30ae8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Jul 30 04:24:56.093567 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:56.093532    2576 patch_prober.go:28] interesting pod/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Jul 30 04:24:56.094011 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:56.093586    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh" podUID="a596f883-2431-4345-aaba-d22c0f30ae8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Jul 30 04:24:57.093524 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:57.093481    2576 patch_prober.go:28] interesting pod/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Jul 30 04:24:57.093712 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:57.093547    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh" podUID="a596f883-2431-4345-aaba-d22c0f30ae8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Jul 30 04:24:58.097278 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:58.097246    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:24:58.097664 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:58.097607    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:24:58.098363 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:24:58.098345    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:25:04.050423 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:04.050390    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:25:04.382262 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:04.382173    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:25:04.637539 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:04.637450    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"]
Jul 30 04:25:04.637944 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:04.637909    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a05e918c-cf1c-48a5-b57f-888f27c239e3" containerName="istio-proxy"
Jul 30 04:25:04.640726 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:04.640707    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:25:04.643833 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:04.643799    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-fcf3e6-openshift-default-dockercfg-5rxbj\""
Jul 30 04:25:04.652737 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:04.652712    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"]
Jul 30 04:25:04.724977 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:04.724946    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1352281-7174-49b9-b639-60c39cb4ccb0-istio-token\") pod \"e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd\" (UID: \"c1352281-7174-49b9-b639-60c39cb4ccb0\") " pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:25:04.725164 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:04.725008    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1352281-7174-49b9-b639-60c39cb4ccb0-credential-socket\") pod \"e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd\" (UID: \"c1352281-7174-49b9-b639-60c39cb4ccb0\") " pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:25:04.725164 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:04.725036    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1352281-7174-49b9-b639-60c39cb4ccb0-istio-podinfo\") pod \"e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd\" (UID: \"c1352281-7174-49b9-b639-60c39cb4ccb0\") " pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:25:04.725164 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:04.725073    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1352281-7174-49b9-b639-60c39cb4ccb0-workload-certs\") pod \"e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd\" (UID: \"c1352281-7174-49b9-b639-60c39cb4ccb0\") " pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:25:04.725164 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:04.725105    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1352281-7174-49b9-b639-60c39cb4ccb0-istio-data\") pod \"e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd\" (UID: \"c1352281-7174-49b9-b639-60c39cb4ccb0\") " pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:25:04.725164 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:04.725140    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1352281-7174-49b9-b639-60c39cb4ccb0-istiod-ca-cert\") pod \"e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd\" (UID: \"c1352281-7174-49b9-b639-60c39cb4ccb0\") " pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:25:04.725164 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:04.725163    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1352281-7174-49b9-b639-60c39cb4ccb0-istio-envoy\") pod \"e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd\" (UID: \"c1352281-7174-49b9-b639-60c39cb4ccb0\") " pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:25:04.725396 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:04.725178    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7np6\" (UniqueName: \"kubernetes.io/projected/c1352281-7174-49b9-b639-60c39cb4ccb0-kube-api-access-k7np6\") pod \"e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd\" (UID: \"c1352281-7174-49b9-b639-60c39cb4ccb0\") " pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:25:04.725396 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:04.725209    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1352281-7174-49b9-b639-60c39cb4ccb0-workload-socket\") pod \"e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd\" (UID: \"c1352281-7174-49b9-b639-60c39cb4ccb0\") " pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:25:04.730324 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:04.730297    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:25:04.825866 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:04.825828    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1352281-7174-49b9-b639-60c39cb4ccb0-credential-socket\") pod \"e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd\" (UID: \"c1352281-7174-49b9-b639-60c39cb4ccb0\") " pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:25:04.826111 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:04.825883    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1352281-7174-49b9-b639-60c39cb4ccb0-istio-podinfo\") pod \"e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd\" (UID: \"c1352281-7174-49b9-b639-60c39cb4ccb0\") " pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:25:04.826111 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:04.825943    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1352281-7174-49b9-b639-60c39cb4ccb0-workload-certs\") pod \"e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd\" (UID: \"c1352281-7174-49b9-b639-60c39cb4ccb0\") " pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:25:04.826111 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:04.825970    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1352281-7174-49b9-b639-60c39cb4ccb0-istio-data\") pod \"e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd\" (UID: \"c1352281-7174-49b9-b639-60c39cb4ccb0\") " pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:25:04.826111 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:04.826026    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1352281-7174-49b9-b639-60c39cb4ccb0-istiod-ca-cert\") pod \"e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd\" (UID: \"c1352281-7174-49b9-b639-60c39cb4ccb0\") " pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:25:04.826111 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:04.826061    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1352281-7174-49b9-b639-60c39cb4ccb0-istio-envoy\") pod \"e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd\" (UID: \"c1352281-7174-49b9-b639-60c39cb4ccb0\") " pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:25:04.826111 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:04.826089    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7np6\" (UniqueName: \"kubernetes.io/projected/c1352281-7174-49b9-b639-60c39cb4ccb0-kube-api-access-k7np6\") pod \"e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd\" (UID: \"c1352281-7174-49b9-b639-60c39cb4ccb0\") " pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:25:04.826441 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:04.826128    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1352281-7174-49b9-b639-60c39cb4ccb0-workload-socket\") pod \"e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd\" (UID: \"c1352281-7174-49b9-b639-60c39cb4ccb0\") " pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:25:04.826441 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:04.826395    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1352281-7174-49b9-b639-60c39cb4ccb0-workload-certs\") pod \"e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd\" (UID: \"c1352281-7174-49b9-b639-60c39cb4ccb0\") " pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:25:04.826441 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:04.826420    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1352281-7174-49b9-b639-60c39cb4ccb0-credential-socket\") pod \"e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd\" (UID: \"c1352281-7174-49b9-b639-60c39cb4ccb0\") " pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:25:04.826597 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:04.826468    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1352281-7174-49b9-b639-60c39cb4ccb0-istio-token\") pod \"e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd\" (UID: \"c1352281-7174-49b9-b639-60c39cb4ccb0\") " pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:25:04.826907 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:04.826878    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1352281-7174-49b9-b639-60c39cb4ccb0-istio-data\") pod \"e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd\" (UID: \"c1352281-7174-49b9-b639-60c39cb4ccb0\") " pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:25:04.827060 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:04.826882    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1352281-7174-49b9-b639-60c39cb4ccb0-istiod-ca-cert\") pod \"e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd\" (UID: \"c1352281-7174-49b9-b639-60c39cb4ccb0\") " pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:25:04.827137 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:04.827102    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1352281-7174-49b9-b639-60c39cb4ccb0-workload-socket\") pod \"e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd\" (UID: \"c1352281-7174-49b9-b639-60c39cb4ccb0\") " pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:25:04.828718 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:04.828701    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1352281-7174-49b9-b639-60c39cb4ccb0-istio-podinfo\") pod \"e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd\" (UID: \"c1352281-7174-49b9-b639-60c39cb4ccb0\") " pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:25:04.828813 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:04.828796    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1352281-7174-49b9-b639-60c39cb4ccb0-istio-envoy\") pod \"e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd\" (UID: \"c1352281-7174-49b9-b639-60c39cb4ccb0\") " pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:25:04.834058 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:04.834036    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1352281-7174-49b9-b639-60c39cb4ccb0-istio-token\") pod \"e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd\" (UID: \"c1352281-7174-49b9-b639-60c39cb4ccb0\") " pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:25:04.834177 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:04.834159    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7np6\" (UniqueName: \"kubernetes.io/projected/c1352281-7174-49b9-b639-60c39cb4ccb0-kube-api-access-k7np6\") pod \"e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd\" (UID: \"c1352281-7174-49b9-b639-60c39cb4ccb0\") " pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:25:04.951761 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:04.951663    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:25:04.971156 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:04.971125    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:25:05.088182 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:05.088156    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"]
Jul 30 04:25:05.089481 ip-10-0-143-117 kubenswrapper[2576]: W0730 04:25:05.089456    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1352281_7174_49b9_b639_60c39cb4ccb0.slice/crio-28d270c07361ff16beb3be8c920b95fd8e2452a2ac6cd88d001d910680fc6b0b WatchSource:0}: Error finding container 28d270c07361ff16beb3be8c920b95fd8e2452a2ac6cd88d001d910680fc6b0b: Status 404 returned error can't find the container with id 28d270c07361ff16beb3be8c920b95fd8e2452a2ac6cd88d001d910680fc6b0b
Jul 30 04:25:05.091575 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:05.091547    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:25:05.091646 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:05.091611    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:25:05.091690 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:05.091659    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:25:05.371351 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:05.371314    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd" event={"ID":"c1352281-7174-49b9-b639-60c39cb4ccb0","Type":"ContainerStarted","Data":"47b53757e44918f24fb13a76c2b53a72e9ff26867312aa6d50311e90daf5adde"}
Jul 30 04:25:05.371546 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:05.371380    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd" event={"ID":"c1352281-7174-49b9-b639-60c39cb4ccb0","Type":"ContainerStarted","Data":"28d270c07361ff16beb3be8c920b95fd8e2452a2ac6cd88d001d910680fc6b0b"}
Jul 30 04:25:05.393910 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:05.393846    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd" podStartSLOduration=1.39382027 podStartE2EDuration="1.39382027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:25:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:25:05.390017201 +0000 UTC m=+3011.008249453" watchObservedRunningTime="2026-07-30 04:25:05.39382027 +0000 UTC m=+3011.012052516"
Jul 30 04:25:05.952128 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:05.952100    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:25:06.956713 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:06.956685    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:25:07.266723 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:07.266612    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:25:07.365255 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:07.365224    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:25:07.382976 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:07.382948    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:25:07.383997 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:07.383974    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:25:08.402352 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:08.402319    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:25:08.773200 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:08.773169    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:25:08.894763 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:08.894730    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:25:09.035747 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:09.035643    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:25:09.207663 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:09.207631    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-fcf3e6-596fc7d866-lw8rm"]
Jul 30 04:25:09.210075 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:09.210059    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-fcf3e6-596fc7d866-lw8rm"
Jul 30 04:25:09.213860 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:09.213837    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-fcf3e6\""
Jul 30 04:25:09.214008 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:09.213973    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-fcf3e6-dockercfg-pggx7\""
Jul 30 04:25:09.224503 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:09.224479    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-fcf3e6-596fc7d866-lw8rm"]
Jul 30 04:25:09.268671 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:09.268636    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgsc4\" (UniqueName: \"kubernetes.io/projected/aa7e6f7d-c522-48d9-aba9-ccaaad76d08d-kube-api-access-rgsc4\") pod \"payload-processing-e2e-shared-b-fcf3e6-596fc7d866-lw8rm\" (UID: \"aa7e6f7d-c522-48d9-aba9-ccaaad76d08d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-fcf3e6-596fc7d866-lw8rm"
Jul 30 04:25:09.268842 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:09.268740    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa7e6f7d-c522-48d9-aba9-ccaaad76d08d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-fcf3e6-596fc7d866-lw8rm\" (UID: \"aa7e6f7d-c522-48d9-aba9-ccaaad76d08d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-fcf3e6-596fc7d866-lw8rm"
Jul 30 04:25:09.369905 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:09.369809    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa7e6f7d-c522-48d9-aba9-ccaaad76d08d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-fcf3e6-596fc7d866-lw8rm\" (UID: \"aa7e6f7d-c522-48d9-aba9-ccaaad76d08d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-fcf3e6-596fc7d866-lw8rm"
Jul 30 04:25:09.369905 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:09.369872    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgsc4\" (UniqueName: \"kubernetes.io/projected/aa7e6f7d-c522-48d9-aba9-ccaaad76d08d-kube-api-access-rgsc4\") pod \"payload-processing-e2e-shared-b-fcf3e6-596fc7d866-lw8rm\" (UID: \"aa7e6f7d-c522-48d9-aba9-ccaaad76d08d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-fcf3e6-596fc7d866-lw8rm"
Jul 30 04:25:09.370578 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:09.370559    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa7e6f7d-c522-48d9-aba9-ccaaad76d08d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-fcf3e6-596fc7d866-lw8rm\" (UID: \"aa7e6f7d-c522-48d9-aba9-ccaaad76d08d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-fcf3e6-596fc7d866-lw8rm"
Jul 30 04:25:09.378394 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:09.378364    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgsc4\" (UniqueName: \"kubernetes.io/projected/aa7e6f7d-c522-48d9-aba9-ccaaad76d08d-kube-api-access-rgsc4\") pod \"payload-processing-e2e-shared-b-fcf3e6-596fc7d866-lw8rm\" (UID: \"aa7e6f7d-c522-48d9-aba9-ccaaad76d08d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-fcf3e6-596fc7d866-lw8rm"
Jul 30 04:25:09.520780 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:09.520748    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-fcf3e6-596fc7d866-lw8rm"
Jul 30 04:25:09.858483 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:09.858454    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-fcf3e6-596fc7d866-lw8rm"]
Jul 30 04:25:09.859637 ip-10-0-143-117 kubenswrapper[2576]: W0730 04:25:09.859613    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa7e6f7d_c522_48d9_aba9_ccaaad76d08d.slice/crio-d004e047c6c1a8aff169a82ae27e764950408d75c792094b1698f8ba19dd6f03 WatchSource:0}: Error finding container d004e047c6c1a8aff169a82ae27e764950408d75c792094b1698f8ba19dd6f03: Status 404 returned error can't find the container with id d004e047c6c1a8aff169a82ae27e764950408d75c792094b1698f8ba19dd6f03
Jul 30 04:25:10.400783 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:10.400748    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-fcf3e6-596fc7d866-lw8rm" event={"ID":"aa7e6f7d-c522-48d9-aba9-ccaaad76d08d","Type":"ContainerStarted","Data":"4f3c80cafd1e6983705f52df3ba4ea06c872f87523c2881462f1575ff3295cb0"}
Jul 30 04:25:10.400990 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:10.400806    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-fcf3e6-596fc7d866-lw8rm" event={"ID":"aa7e6f7d-c522-48d9-aba9-ccaaad76d08d","Type":"ContainerStarted","Data":"d004e047c6c1a8aff169a82ae27e764950408d75c792094b1698f8ba19dd6f03"}
Jul 30 04:25:10.400990 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:10.400942    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-fcf3e6-596fc7d866-lw8rm"
Jul 30 04:25:10.418953 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:10.418892    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-fcf3e6-596fc7d866-lw8rm" podStartSLOduration=1.418878295 podStartE2EDuration="1.418878295s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:25:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:25:10.417632087 +0000 UTC m=+3016.035864322" watchObservedRunningTime="2026-07-30 04:25:10.418878295 +0000 UTC m=+3016.037110554"
Jul 30 04:25:12.118547 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:12.118503    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:25:12.171103 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:12.171067    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:25:15.727817 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:15.727785    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:25:21.414106 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:21.414075    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-fcf3e6-596fc7d866-lw8rm"
Jul 30 04:25:23.288581 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:23.288538    2576 patch_prober.go:28] interesting pod/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": read tcp 10.134.0.2:36538->10.134.0.76:15021: read: connection reset by peer" start-of-body=
Jul 30 04:25:23.289015 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:23.288630    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd" podUID="c1352281-7174-49b9-b639-60c39cb4ccb0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": read tcp 10.134.0.2:36538->10.134.0.76:15021: read: connection reset by peer"
Jul 30 04:25:23.487857 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:23.487774    2576 generic.go:356] "Generic (PLEG): container finished" podID="c1352281-7174-49b9-b639-60c39cb4ccb0" containerID="47b53757e44918f24fb13a76c2b53a72e9ff26867312aa6d50311e90daf5adde" exitCode=0
Jul 30 04:25:23.488021 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:23.487852    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd" event={"ID":"c1352281-7174-49b9-b639-60c39cb4ccb0","Type":"ContainerDied","Data":"47b53757e44918f24fb13a76c2b53a72e9ff26867312aa6d50311e90daf5adde"}
Jul 30 04:25:23.488293 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:23.488275    2576 scope.go:117] "RemoveContainer" containerID="47b53757e44918f24fb13a76c2b53a72e9ff26867312aa6d50311e90daf5adde"
Jul 30 04:25:23.489457 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:23.489426    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:25:23.489560 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:23.489537    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:25:23.489606 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:23.489594    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:25:24.494414 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:24.494376    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd" event={"ID":"c1352281-7174-49b9-b639-60c39cb4ccb0","Type":"ContainerStarted","Data":"b7291ba5bd852526c8fae3220f127d6476689ebc10ea243986180a1a166ffd80"}
Jul 30 04:25:24.952777 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:24.952733    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:25:24.953016 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:24.952867    2576 patch_prober.go:28] interesting pod/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Jul 30 04:25:24.953016 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:24.952911    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd" podUID="c1352281-7174-49b9-b639-60c39cb4ccb0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Jul 30 04:25:25.952302 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:25.952260    2576 patch_prober.go:28] interesting pod/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Jul 30 04:25:25.952795 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:25.952342    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd" podUID="c1352281-7174-49b9-b639-60c39cb4ccb0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Jul 30 04:25:26.952115 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:26.952073    2576 patch_prober.go:28] interesting pod/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Jul 30 04:25:26.952421 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:26.952389    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd" podUID="c1352281-7174-49b9-b639-60c39cb4ccb0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Jul 30 04:25:27.952209 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:27.952163    2576 patch_prober.go:28] interesting pod/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Jul 30 04:25:27.952425 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:27.952244    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd" podUID="c1352281-7174-49b9-b639-60c39cb4ccb0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Jul 30 04:25:28.955297 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:28.955269    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:25:28.955658 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:28.955531    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:25:28.956283 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:28.956269    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:25:32.213544 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:32.213498    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:25:55.626477 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:25:55.626371    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:26:16.445788 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:26:16.445752    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:26:19.626779 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:26:19.626740    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:26:23.441486 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:26:23.441444    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:26:26.452107 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:26:26.452073    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:26:29.329253 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:26:29.329219    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:27:03.929904 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:27:03.929867    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:27:30.129115 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:27:30.129030    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:27:48.626745 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:27:48.626712    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:28:16.923433 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:28:16.923397    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:28:33.231831 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:28:33.231795    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:29:00.730451 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:29:00.730372    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:29:19.671402 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:29:19.671367    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:29:46.626094 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:29:46.626056    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:30:05.227226 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:30:05.227191    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:30:06.024599 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:30:06.024561    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:30:30.336307 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:30:30.336272    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:30:47.632165 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:30:47.632128    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:31:04.596653 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:31:04.596609    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:31:14.527689 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:31:14.527658    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:31:17.283766 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:31:17.283730    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:31:18.295290 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:31:18.295256    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:31:30.928756 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:31:30.928722    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:31:31.262798 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:31:31.262705    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:31:40.633025 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:31:40.632992    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:31:52.634562 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:31:52.634457    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:31:56.231536 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:31:56.231498    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:32:52.132109 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:32:52.132073    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:33:07.438626 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:33:07.438586    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:33:33.136987 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:33:33.136885    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:33:38.433068 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:33:38.433032    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:33:39.422976 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:33:39.422942    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:33:53.429517 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:33:53.429483    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:33:57.335380 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:33:57.335328    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:33:59.416173 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:33:59.416128    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:34:03.931378 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:34:03.931333    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:34:21.055859 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:34:21.055824    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:34:24.621255 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:34:24.621214    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:34:25.087831 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:34:25.087795    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:35:18.925181 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:35:18.925145    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:35:40.536053 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:35:40.536016    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:36:04.529277 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:04.529244    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:36:10.558883 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:10.558853    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:36:12.182478 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:12.182443    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:36:12.369550 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:12.369513    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7322-140422138461232"]
Jul 30 04:36:12.373092 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:12.373077    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7322-140422138461232"
Jul 30 04:36:12.376474 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:12.376452    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-69vlb\""
Jul 30 04:36:12.385078 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:12.385054    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7322-140422138461232"]
Jul 30 04:36:12.544096 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:12.544064    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lh7fm\" (UniqueName: \"kubernetes.io/projected/ea6c6291-7223-4241-8054-1253cb0a6628-kube-api-access-lh7fm\") pod \"test-curl-7322-140422138461232\" (UID: \"ea6c6291-7223-4241-8054-1253cb0a6628\") " pod="opendatahub/test-curl-7322-140422138461232"
Jul 30 04:36:12.645169 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:12.645134    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lh7fm\" (UniqueName: \"kubernetes.io/projected/ea6c6291-7223-4241-8054-1253cb0a6628-kube-api-access-lh7fm\") pod \"test-curl-7322-140422138461232\" (UID: \"ea6c6291-7223-4241-8054-1253cb0a6628\") " pod="opendatahub/test-curl-7322-140422138461232"
Jul 30 04:36:12.654218 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:12.654189    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lh7fm\" (UniqueName: \"kubernetes.io/projected/ea6c6291-7223-4241-8054-1253cb0a6628-kube-api-access-lh7fm\") pod \"test-curl-7322-140422138461232\" (UID: \"ea6c6291-7223-4241-8054-1253cb0a6628\") " pod="opendatahub/test-curl-7322-140422138461232"
Jul 30 04:36:12.682025 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:12.681990    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7322-140422138461232"
Jul 30 04:36:12.815127 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:12.815105    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7322-140422138461232"]
Jul 30 04:36:12.816946 ip-10-0-143-117 kubenswrapper[2576]: W0730 04:36:12.816895    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podea6c6291_7223_4241_8054_1253cb0a6628.slice/crio-ded34b741c256bf2ffa49978e38720f170f408d723868a9fed9eebe483be8e9e WatchSource:0}: Error finding container ded34b741c256bf2ffa49978e38720f170f408d723868a9fed9eebe483be8e9e: Status 404 returned error can't find the container with id ded34b741c256bf2ffa49978e38720f170f408d723868a9fed9eebe483be8e9e
Jul 30 04:36:12.818436 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:12.818416    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 04:36:12.968372 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:12.968337    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7322-140422138461232" event={"ID":"ea6c6291-7223-4241-8054-1253cb0a6628","Type":"ContainerStarted","Data":"ded34b741c256bf2ffa49978e38720f170f408d723868a9fed9eebe483be8e9e"}
Jul 30 04:36:13.975275 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:13.975241    2576 generic.go:356] "Generic (PLEG): container finished" podID="ea6c6291-7223-4241-8054-1253cb0a6628" containerID="e4c8a4d6f9f86f0225b3260982dcc53cd3ddc2bca547a545895bacb9009b78e5" exitCode=0
Jul 30 04:36:13.975711 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:13.975322    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7322-140422138461232" event={"ID":"ea6c6291-7223-4241-8054-1253cb0a6628","Type":"ContainerDied","Data":"e4c8a4d6f9f86f0225b3260982dcc53cd3ddc2bca547a545895bacb9009b78e5"}
Jul 30 04:36:15.116042 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:15.116019    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7322-140422138461232"
Jul 30 04:36:15.135040 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:15.135018    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7322-140422138461232_ea6c6291-7223-4241-8054-1253cb0a6628/test-curl-7322-140422138461232/0.log"
Jul 30 04:36:15.155908 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:15.155885    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7322-140422138461232"]
Jul 30 04:36:15.159372 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:15.159349    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7322-140422138461232"]
Jul 30 04:36:15.167496 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:15.167471    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lh7fm\" (UniqueName: \"kubernetes.io/projected/ea6c6291-7223-4241-8054-1253cb0a6628-kube-api-access-lh7fm\") pod \"ea6c6291-7223-4241-8054-1253cb0a6628\" (UID: \"ea6c6291-7223-4241-8054-1253cb0a6628\") "
Jul 30 04:36:15.169534 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:15.169512    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea6c6291-7223-4241-8054-1253cb0a6628-kube-api-access-lh7fm" (OuterVolumeSpecName: "kube-api-access-lh7fm") pod "ea6c6291-7223-4241-8054-1253cb0a6628" (UID: "ea6c6291-7223-4241-8054-1253cb0a6628"). InnerVolumeSpecName "kube-api-access-lh7fm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:36:15.268546 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:15.268464    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lh7fm\" (UniqueName: \"kubernetes.io/projected/ea6c6291-7223-4241-8054-1253cb0a6628-kube-api-access-lh7fm\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:36:15.277872 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:15.277846    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7322-140422138461104"]
Jul 30 04:36:15.278237 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:15.278224    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea6c6291-7223-4241-8054-1253cb0a6628" containerName="test-curl-7322-140422138461232"
Jul 30 04:36:15.278285 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:15.278239    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea6c6291-7223-4241-8054-1253cb0a6628" containerName="test-curl-7322-140422138461232"
Jul 30 04:36:15.278335 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:15.278325    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea6c6291-7223-4241-8054-1253cb0a6628" containerName="test-curl-7322-140422138461232"
Jul 30 04:36:15.281366 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:15.281352    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7322-140422138461104"
Jul 30 04:36:15.285705 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:15.285684    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7322-140422138461104"]
Jul 30 04:36:15.369216 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:15.369177    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4z4r\" (UniqueName: \"kubernetes.io/projected/3cdf0fdd-3369-4718-8850-1acfd6262708-kube-api-access-b4z4r\") pod \"test-curl-7322-140422138461104\" (UID: \"3cdf0fdd-3369-4718-8850-1acfd6262708\") " pod="opendatahub/test-curl-7322-140422138461104"
Jul 30 04:36:15.469918 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:15.469888    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4z4r\" (UniqueName: \"kubernetes.io/projected/3cdf0fdd-3369-4718-8850-1acfd6262708-kube-api-access-b4z4r\") pod \"test-curl-7322-140422138461104\" (UID: \"3cdf0fdd-3369-4718-8850-1acfd6262708\") " pod="opendatahub/test-curl-7322-140422138461104"
Jul 30 04:36:15.478260 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:15.478237    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4z4r\" (UniqueName: \"kubernetes.io/projected/3cdf0fdd-3369-4718-8850-1acfd6262708-kube-api-access-b4z4r\") pod \"test-curl-7322-140422138461104\" (UID: \"3cdf0fdd-3369-4718-8850-1acfd6262708\") " pod="opendatahub/test-curl-7322-140422138461104"
Jul 30 04:36:15.592417 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:15.592377    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7322-140422138461104"
Jul 30 04:36:15.718256 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:15.718231    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7322-140422138461104"]
Jul 30 04:36:15.720582 ip-10-0-143-117 kubenswrapper[2576]: W0730 04:36:15.720553    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3cdf0fdd_3369_4718_8850_1acfd6262708.slice/crio-a40eb8e95a564d27681b78b37c73b315f84d090c654995cff1b62a8f6ae42b0e WatchSource:0}: Error finding container a40eb8e95a564d27681b78b37c73b315f84d090c654995cff1b62a8f6ae42b0e: Status 404 returned error can't find the container with id a40eb8e95a564d27681b78b37c73b315f84d090c654995cff1b62a8f6ae42b0e
Jul 30 04:36:15.986268 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:15.986194    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7322-140422138461232"
Jul 30 04:36:15.986425 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:15.986204    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ded34b741c256bf2ffa49978e38720f170f408d723868a9fed9eebe483be8e9e"
Jul 30 04:36:15.987508 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:15.987478    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7322-140422138461104" event={"ID":"3cdf0fdd-3369-4718-8850-1acfd6262708","Type":"ContainerStarted","Data":"a40eb8e95a564d27681b78b37c73b315f84d090c654995cff1b62a8f6ae42b0e"}
Jul 30 04:36:15.997833 ip-10-0-143-117 kubenswrapper[2576]: E0730 04:36:15.997805    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7322-140422138461232\" is forbidden: User \"system:node:ip-10-0-143-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-117.ec2.internal' and this object" podUID="ea6c6291-7223-4241-8054-1253cb0a6628" pod="opendatahub/test-curl-7322-140422138461232"
Jul 30 04:36:16.994386 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:16.994351    2576 generic.go:356] "Generic (PLEG): container finished" podID="3cdf0fdd-3369-4718-8850-1acfd6262708" containerID="85f7a56354ccd4c8ff5b4bfa899f984389f6e027fa272f34ee55e14551ad8bef" exitCode=0
Jul 30 04:36:16.994819 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:16.994440    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7322-140422138461104" event={"ID":"3cdf0fdd-3369-4718-8850-1acfd6262708","Type":"ContainerDied","Data":"85f7a56354ccd4c8ff5b4bfa899f984389f6e027fa272f34ee55e14551ad8bef"}
Jul 30 04:36:16.996796 ip-10-0-143-117 kubenswrapper[2576]: E0730 04:36:16.996773    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7322-140422138461232\" is forbidden: User \"system:node:ip-10-0-143-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-117.ec2.internal' and this object" podUID="ea6c6291-7223-4241-8054-1253cb0a6628" pod="opendatahub/test-curl-7322-140422138461232"
Jul 30 04:36:17.051163 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:17.051135    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea6c6291-7223-4241-8054-1253cb0a6628" path="/var/lib/kubelet/pods/ea6c6291-7223-4241-8054-1253cb0a6628/volumes"
Jul 30 04:36:18.142126 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:18.142099    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7322-140422138461104"
Jul 30 04:36:18.161519 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:18.161493    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7322-140422138461104_3cdf0fdd-3369-4718-8850-1acfd6262708/test-curl-7322-140422138461104/0.log"
Jul 30 04:36:18.182860 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:18.182835    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7322-140422138461104"]
Jul 30 04:36:18.186266 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:18.186245    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7322-140422138461104"]
Jul 30 04:36:18.194411 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:18.194393    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4z4r\" (UniqueName: \"kubernetes.io/projected/3cdf0fdd-3369-4718-8850-1acfd6262708-kube-api-access-b4z4r\") pod \"3cdf0fdd-3369-4718-8850-1acfd6262708\" (UID: \"3cdf0fdd-3369-4718-8850-1acfd6262708\") "
Jul 30 04:36:18.196421 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:18.196389    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cdf0fdd-3369-4718-8850-1acfd6262708-kube-api-access-b4z4r" (OuterVolumeSpecName: "kube-api-access-b4z4r") pod "3cdf0fdd-3369-4718-8850-1acfd6262708" (UID: "3cdf0fdd-3369-4718-8850-1acfd6262708"). InnerVolumeSpecName "kube-api-access-b4z4r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:36:18.295152 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:18.295107    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4z4r\" (UniqueName: \"kubernetes.io/projected/3cdf0fdd-3369-4718-8850-1acfd6262708-kube-api-access-b4z4r\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:36:18.341084 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:18.341056    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7322-140422138631200"]
Jul 30 04:36:18.341481 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:18.341467    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cdf0fdd-3369-4718-8850-1acfd6262708" containerName="test-curl-7322-140422138461104"
Jul 30 04:36:18.341527 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:18.341483    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cdf0fdd-3369-4718-8850-1acfd6262708" containerName="test-curl-7322-140422138461104"
Jul 30 04:36:18.341568 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:18.341548    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cdf0fdd-3369-4718-8850-1acfd6262708" containerName="test-curl-7322-140422138461104"
Jul 30 04:36:18.344670 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:18.344650    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7322-140422138631200"
Jul 30 04:36:18.347858 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:18.347833    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7322-140422138631200"]
Jul 30 04:36:18.396053 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:18.396024    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2wwj\" (UniqueName: \"kubernetes.io/projected/21c0b925-cbe7-4f9a-867c-5471157e564d-kube-api-access-j2wwj\") pod \"test-curl-7322-140422138631200\" (UID: \"21c0b925-cbe7-4f9a-867c-5471157e564d\") " pod="opendatahub/test-curl-7322-140422138631200"
Jul 30 04:36:18.427439 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:18.427410    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:36:18.496850 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:18.496817    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2wwj\" (UniqueName: \"kubernetes.io/projected/21c0b925-cbe7-4f9a-867c-5471157e564d-kube-api-access-j2wwj\") pod \"test-curl-7322-140422138631200\" (UID: \"21c0b925-cbe7-4f9a-867c-5471157e564d\") " pod="opendatahub/test-curl-7322-140422138631200"
Jul 30 04:36:18.505394 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:18.505368    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2wwj\" (UniqueName: \"kubernetes.io/projected/21c0b925-cbe7-4f9a-867c-5471157e564d-kube-api-access-j2wwj\") pod \"test-curl-7322-140422138631200\" (UID: \"21c0b925-cbe7-4f9a-867c-5471157e564d\") " pod="opendatahub/test-curl-7322-140422138631200"
Jul 30 04:36:18.655170 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:18.655092    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7322-140422138631200"
Jul 30 04:36:18.792809 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:18.792773    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7322-140422138631200"]
Jul 30 04:36:18.792974 ip-10-0-143-117 kubenswrapper[2576]: W0730 04:36:18.792891    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod21c0b925_cbe7_4f9a_867c_5471157e564d.slice/crio-126becbe0340bf5b2c509d151f71c89bda3e200f677ca3fdcff7da889de2c89d WatchSource:0}: Error finding container 126becbe0340bf5b2c509d151f71c89bda3e200f677ca3fdcff7da889de2c89d: Status 404 returned error can't find the container with id 126becbe0340bf5b2c509d151f71c89bda3e200f677ca3fdcff7da889de2c89d
Jul 30 04:36:19.009162 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:19.009074    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7322-140422138631200" event={"ID":"21c0b925-cbe7-4f9a-867c-5471157e564d","Type":"ContainerStarted","Data":"126becbe0340bf5b2c509d151f71c89bda3e200f677ca3fdcff7da889de2c89d"}
Jul 30 04:36:19.010240 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:19.010218    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7322-140422138461104"
Jul 30 04:36:19.010384 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:19.010218    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a40eb8e95a564d27681b78b37c73b315f84d090c654995cff1b62a8f6ae42b0e"
Jul 30 04:36:19.021162 ip-10-0-143-117 kubenswrapper[2576]: E0730 04:36:19.021133    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7322-140422138461104\" is forbidden: User \"system:node:ip-10-0-143-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-117.ec2.internal' and this object" podUID="3cdf0fdd-3369-4718-8850-1acfd6262708" pod="opendatahub/test-curl-7322-140422138461104"
Jul 30 04:36:19.052069 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:19.052048    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cdf0fdd-3369-4718-8850-1acfd6262708" path="/var/lib/kubelet/pods/3cdf0fdd-3369-4718-8850-1acfd6262708/volumes"
Jul 30 04:36:20.017586 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:20.017549    2576 generic.go:356] "Generic (PLEG): container finished" podID="21c0b925-cbe7-4f9a-867c-5471157e564d" containerID="7c60045d4dfae168c0b17bffcaa35a598ca4a66e8016c8738cf0224e44d4f588" exitCode=0
Jul 30 04:36:20.018010 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:20.017595    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7322-140422138631200" event={"ID":"21c0b925-cbe7-4f9a-867c-5471157e564d","Type":"ContainerDied","Data":"7c60045d4dfae168c0b17bffcaa35a598ca4a66e8016c8738cf0224e44d4f588"}
Jul 30 04:36:21.160661 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:21.160635    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7322-140422138631200"
Jul 30 04:36:21.180058 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:21.180036    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7322-140422138631200_21c0b925-cbe7-4f9a-867c-5471157e564d/test-curl-7322-140422138631200/0.log"
Jul 30 04:36:21.202026 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:21.201997    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7322-140422138631200"]
Jul 30 04:36:21.205829 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:21.205806    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7322-140422138631200"]
Jul 30 04:36:21.220861 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:21.220839    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2wwj\" (UniqueName: \"kubernetes.io/projected/21c0b925-cbe7-4f9a-867c-5471157e564d-kube-api-access-j2wwj\") pod \"21c0b925-cbe7-4f9a-867c-5471157e564d\" (UID: \"21c0b925-cbe7-4f9a-867c-5471157e564d\") "
Jul 30 04:36:21.222853 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:21.222829    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21c0b925-cbe7-4f9a-867c-5471157e564d-kube-api-access-j2wwj" (OuterVolumeSpecName: "kube-api-access-j2wwj") pod "21c0b925-cbe7-4f9a-867c-5471157e564d" (UID: "21c0b925-cbe7-4f9a-867c-5471157e564d"). InnerVolumeSpecName "kube-api-access-j2wwj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:36:21.318887 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:21.318808    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7322-140422138631200"]
Jul 30 04:36:21.319239 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:21.319222    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21c0b925-cbe7-4f9a-867c-5471157e564d" containerName="test-curl-7322-140422138631200"
Jul 30 04:36:21.319282 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:21.319241    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="21c0b925-cbe7-4f9a-867c-5471157e564d" containerName="test-curl-7322-140422138631200"
Jul 30 04:36:21.319318 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:21.319312    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="21c0b925-cbe7-4f9a-867c-5471157e564d" containerName="test-curl-7322-140422138631200"
Jul 30 04:36:21.321646 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:21.321629    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2wwj\" (UniqueName: \"kubernetes.io/projected/21c0b925-cbe7-4f9a-867c-5471157e564d-kube-api-access-j2wwj\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:36:21.322437 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:21.322422    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7322-140422138631200"
Jul 30 04:36:21.328686 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:21.328663    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7322-140422138631200"]
Jul 30 04:36:21.422109 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:21.422078    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndznh\" (UniqueName: \"kubernetes.io/projected/49841ab2-cc2e-4ea9-b8ed-1345ea2f9980-kube-api-access-ndznh\") pod \"test-curl-7322-140422138631200\" (UID: \"49841ab2-cc2e-4ea9-b8ed-1345ea2f9980\") " pod="opendatahub/test-curl-7322-140422138631200"
Jul 30 04:36:21.523189 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:21.523138    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndznh\" (UniqueName: \"kubernetes.io/projected/49841ab2-cc2e-4ea9-b8ed-1345ea2f9980-kube-api-access-ndznh\") pod \"test-curl-7322-140422138631200\" (UID: \"49841ab2-cc2e-4ea9-b8ed-1345ea2f9980\") " pod="opendatahub/test-curl-7322-140422138631200"
Jul 30 04:36:21.531398 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:21.531366    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndznh\" (UniqueName: \"kubernetes.io/projected/49841ab2-cc2e-4ea9-b8ed-1345ea2f9980-kube-api-access-ndznh\") pod \"test-curl-7322-140422138631200\" (UID: \"49841ab2-cc2e-4ea9-b8ed-1345ea2f9980\") " pod="opendatahub/test-curl-7322-140422138631200"
Jul 30 04:36:21.632991 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:21.632889    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7322-140422138631200"
Jul 30 04:36:21.761624 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:21.761601    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7322-140422138631200"]
Jul 30 04:36:21.763578 ip-10-0-143-117 kubenswrapper[2576]: W0730 04:36:21.763553    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod49841ab2_cc2e_4ea9_b8ed_1345ea2f9980.slice/crio-a9089877ac7931339f8cb02d4bbcc15978b15e753f0cf7b12ad42e8cc5ec74d4 WatchSource:0}: Error finding container a9089877ac7931339f8cb02d4bbcc15978b15e753f0cf7b12ad42e8cc5ec74d4: Status 404 returned error can't find the container with id a9089877ac7931339f8cb02d4bbcc15978b15e753f0cf7b12ad42e8cc5ec74d4
Jul 30 04:36:22.030256 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:22.030232    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7322-140422138631200"
Jul 30 04:36:22.030256 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:22.030243    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="126becbe0340bf5b2c509d151f71c89bda3e200f677ca3fdcff7da889de2c89d"
Jul 30 04:36:22.031451 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:22.031427    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7322-140422138631200" event={"ID":"49841ab2-cc2e-4ea9-b8ed-1345ea2f9980","Type":"ContainerStarted","Data":"a9089877ac7931339f8cb02d4bbcc15978b15e753f0cf7b12ad42e8cc5ec74d4"}
Jul 30 04:36:22.043241 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:22.043184    2576 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7322-140422138631200" oldPodUID="21c0b925-cbe7-4f9a-867c-5471157e564d" podUID="49841ab2-cc2e-4ea9-b8ed-1345ea2f9980"
Jul 30 04:36:23.038791 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:23.038755    2576 generic.go:356] "Generic (PLEG): container finished" podID="49841ab2-cc2e-4ea9-b8ed-1345ea2f9980" containerID="44fdcde10ea9d32d6c3d1e6471ebd732d13c3dc4abd720181a2e95f22e129489" exitCode=0
Jul 30 04:36:23.039210 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:23.038837    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7322-140422138631200" event={"ID":"49841ab2-cc2e-4ea9-b8ed-1345ea2f9980","Type":"ContainerDied","Data":"44fdcde10ea9d32d6c3d1e6471ebd732d13c3dc4abd720181a2e95f22e129489"}
Jul 30 04:36:23.050820 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:23.050797    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21c0b925-cbe7-4f9a-867c-5471157e564d" path="/var/lib/kubelet/pods/21c0b925-cbe7-4f9a-867c-5471157e564d/volumes"
Jul 30 04:36:24.191302 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:24.191279    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7322-140422138631200"
Jul 30 04:36:24.210599 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:24.210574    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7322-140422138631200_49841ab2-cc2e-4ea9-b8ed-1345ea2f9980/test-curl-7322-140422138631200/0.log"
Jul 30 04:36:24.232207 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:24.232181    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7322-140422138631200"]
Jul 30 04:36:24.235338 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:24.235315    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7322-140422138631200"]
Jul 30 04:36:24.250329 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:24.250310    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ndznh\" (UniqueName: \"kubernetes.io/projected/49841ab2-cc2e-4ea9-b8ed-1345ea2f9980-kube-api-access-ndznh\") pod \"49841ab2-cc2e-4ea9-b8ed-1345ea2f9980\" (UID: \"49841ab2-cc2e-4ea9-b8ed-1345ea2f9980\") "
Jul 30 04:36:24.252251 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:24.252224    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49841ab2-cc2e-4ea9-b8ed-1345ea2f9980-kube-api-access-ndznh" (OuterVolumeSpecName: "kube-api-access-ndznh") pod "49841ab2-cc2e-4ea9-b8ed-1345ea2f9980" (UID: "49841ab2-cc2e-4ea9-b8ed-1345ea2f9980"). InnerVolumeSpecName "kube-api-access-ndznh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:36:24.344746 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:24.344665    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7322-140422138632208"]
Jul 30 04:36:24.345087 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:24.345073    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49841ab2-cc2e-4ea9-b8ed-1345ea2f9980" containerName="test-curl-7322-140422138631200"
Jul 30 04:36:24.345149 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:24.345088    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="49841ab2-cc2e-4ea9-b8ed-1345ea2f9980" containerName="test-curl-7322-140422138631200"
Jul 30 04:36:24.345186 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:24.345163    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="49841ab2-cc2e-4ea9-b8ed-1345ea2f9980" containerName="test-curl-7322-140422138631200"
Jul 30 04:36:24.348272 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:24.348256    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7322-140422138632208"
Jul 30 04:36:24.351658 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:24.351626    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndznh\" (UniqueName: \"kubernetes.io/projected/49841ab2-cc2e-4ea9-b8ed-1345ea2f9980-kube-api-access-ndznh\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:36:24.354503 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:24.354482    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7322-140422138632208"]
Jul 30 04:36:24.452992 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:24.452967    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7f4q\" (UniqueName: \"kubernetes.io/projected/d56314c4-1d05-4a18-b32b-46342da30fbd-kube-api-access-n7f4q\") pod \"test-curl-7322-140422138632208\" (UID: \"d56314c4-1d05-4a18-b32b-46342da30fbd\") " pod="opendatahub/test-curl-7322-140422138632208"
Jul 30 04:36:24.554478 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:24.554437    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7f4q\" (UniqueName: \"kubernetes.io/projected/d56314c4-1d05-4a18-b32b-46342da30fbd-kube-api-access-n7f4q\") pod \"test-curl-7322-140422138632208\" (UID: \"d56314c4-1d05-4a18-b32b-46342da30fbd\") " pod="opendatahub/test-curl-7322-140422138632208"
Jul 30 04:36:24.562960 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:24.562917    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7f4q\" (UniqueName: \"kubernetes.io/projected/d56314c4-1d05-4a18-b32b-46342da30fbd-kube-api-access-n7f4q\") pod \"test-curl-7322-140422138632208\" (UID: \"d56314c4-1d05-4a18-b32b-46342da30fbd\") " pod="opendatahub/test-curl-7322-140422138632208"
Jul 30 04:36:24.658998 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:24.658877    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7322-140422138632208"
Jul 30 04:36:24.790522 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:24.790492    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7322-140422138632208"]
Jul 30 04:36:24.791820 ip-10-0-143-117 kubenswrapper[2576]: W0730 04:36:24.791797    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd56314c4_1d05_4a18_b32b_46342da30fbd.slice/crio-c295be231e15535d4db4cd54129a29754c5ee02b8f2bef23a541bc61257af1a2 WatchSource:0}: Error finding container c295be231e15535d4db4cd54129a29754c5ee02b8f2bef23a541bc61257af1a2: Status 404 returned error can't find the container with id c295be231e15535d4db4cd54129a29754c5ee02b8f2bef23a541bc61257af1a2
Jul 30 04:36:25.054513 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:25.054478    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49841ab2-cc2e-4ea9-b8ed-1345ea2f9980" path="/var/lib/kubelet/pods/49841ab2-cc2e-4ea9-b8ed-1345ea2f9980/volumes"
Jul 30 04:36:25.057893 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:25.057872    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7322-140422138631200"
Jul 30 04:36:25.058075 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:25.057942    2576 scope.go:117] "RemoveContainer" containerID="44fdcde10ea9d32d6c3d1e6471ebd732d13c3dc4abd720181a2e95f22e129489"
Jul 30 04:36:25.059256 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:25.059197    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7322-140422138632208" event={"ID":"d56314c4-1d05-4a18-b32b-46342da30fbd","Type":"ContainerStarted","Data":"c295be231e15535d4db4cd54129a29754c5ee02b8f2bef23a541bc61257af1a2"}
Jul 30 04:36:26.067419 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:26.067392    2576 generic.go:356] "Generic (PLEG): container finished" podID="d56314c4-1d05-4a18-b32b-46342da30fbd" containerID="cb09c624e82254266d9a97775024b9b21ebae4358903f1b9e3e61505ee4b20a7" exitCode=0
Jul 30 04:36:26.067792 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:26.067476    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7322-140422138632208" event={"ID":"d56314c4-1d05-4a18-b32b-46342da30fbd","Type":"ContainerDied","Data":"cb09c624e82254266d9a97775024b9b21ebae4358903f1b9e3e61505ee4b20a7"}
Jul 30 04:36:27.222967 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:27.222940    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7322-140422138632208"
Jul 30 04:36:27.242240 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:27.242214    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7322-140422138632208_d56314c4-1d05-4a18-b32b-46342da30fbd/test-curl-7322-140422138632208/0.log"
Jul 30 04:36:27.263316 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:27.263287    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7322-140422138632208"]
Jul 30 04:36:27.266164 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:27.266140    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7322-140422138632208"]
Jul 30 04:36:27.278032 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:27.278014    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n7f4q\" (UniqueName: \"kubernetes.io/projected/d56314c4-1d05-4a18-b32b-46342da30fbd-kube-api-access-n7f4q\") pod \"d56314c4-1d05-4a18-b32b-46342da30fbd\" (UID: \"d56314c4-1d05-4a18-b32b-46342da30fbd\") "
Jul 30 04:36:27.280053 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:27.280021    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d56314c4-1d05-4a18-b32b-46342da30fbd-kube-api-access-n7f4q" (OuterVolumeSpecName: "kube-api-access-n7f4q") pod "d56314c4-1d05-4a18-b32b-46342da30fbd" (UID: "d56314c4-1d05-4a18-b32b-46342da30fbd"). InnerVolumeSpecName "kube-api-access-n7f4q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:36:27.379558 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:27.379472    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n7f4q\" (UniqueName: \"kubernetes.io/projected/d56314c4-1d05-4a18-b32b-46342da30fbd-kube-api-access-n7f4q\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:36:28.078631 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:28.078600    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7322-140422138632208"
Jul 30 04:36:28.078786 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:28.078611    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c295be231e15535d4db4cd54129a29754c5ee02b8f2bef23a541bc61257af1a2"
Jul 30 04:36:28.088527 ip-10-0-143-117 kubenswrapper[2576]: E0730 04:36:28.088498    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7322-140422138632208\" is forbidden: User \"system:node:ip-10-0-143-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-117.ec2.internal' and this object" podUID="d56314c4-1d05-4a18-b32b-46342da30fbd" pod="opendatahub/test-curl-7322-140422138632208"
Jul 30 04:36:29.051297 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:29.051251    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d56314c4-1d05-4a18-b32b-46342da30fbd" path="/var/lib/kubelet/pods/d56314c4-1d05-4a18-b32b-46342da30fbd/volumes"
Jul 30 04:36:30.698902 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:30.698871    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:36:30.976684 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:30.976601    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"]
Jul 30 04:36:30.977038 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:30.977023    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d56314c4-1d05-4a18-b32b-46342da30fbd" containerName="test-curl-7322-140422138632208"
Jul 30 04:36:30.977088 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:30.977042    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d56314c4-1d05-4a18-b32b-46342da30fbd" containerName="test-curl-7322-140422138632208"
Jul 30 04:36:30.977124 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:30.977113    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d56314c4-1d05-4a18-b32b-46342da30fbd" containerName="test-curl-7322-140422138632208"
Jul 30 04:36:30.980370 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:30.980350    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:36:30.983173 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:30.983155    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-77b2c7-openshift-default-dockercfg-4lgwv\""
Jul 30 04:36:30.989841 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:30.989822    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"]
Jul 30 04:36:30.996365 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:30.996340    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:36:31.012996 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:31.012971    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/270b4e59-f874-4227-9207-7fee83b78a28-credential-socket\") pod \"e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8\" (UID: \"270b4e59-f874-4227-9207-7fee83b78a28\") " pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:36:31.013107 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:31.013012    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/270b4e59-f874-4227-9207-7fee83b78a28-workload-socket\") pod \"e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8\" (UID: \"270b4e59-f874-4227-9207-7fee83b78a28\") " pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:36:31.013107 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:31.013031    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/270b4e59-f874-4227-9207-7fee83b78a28-istiod-ca-cert\") pod \"e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8\" (UID: \"270b4e59-f874-4227-9207-7fee83b78a28\") " pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:36:31.013188 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:31.013163    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/270b4e59-f874-4227-9207-7fee83b78a28-istio-token\") pod \"e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8\" (UID: \"270b4e59-f874-4227-9207-7fee83b78a28\") " pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:36:31.013232 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:31.013214    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/270b4e59-f874-4227-9207-7fee83b78a28-istio-envoy\") pod \"e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8\" (UID: \"270b4e59-f874-4227-9207-7fee83b78a28\") " pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:36:31.013288 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:31.013273    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dg8gm\" (UniqueName: \"kubernetes.io/projected/270b4e59-f874-4227-9207-7fee83b78a28-kube-api-access-dg8gm\") pod \"e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8\" (UID: \"270b4e59-f874-4227-9207-7fee83b78a28\") " pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:36:31.013328 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:31.013314    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/270b4e59-f874-4227-9207-7fee83b78a28-workload-certs\") pod \"e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8\" (UID: \"270b4e59-f874-4227-9207-7fee83b78a28\") " pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:36:31.013367 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:31.013342    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/270b4e59-f874-4227-9207-7fee83b78a28-istio-podinfo\") pod \"e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8\" (UID: \"270b4e59-f874-4227-9207-7fee83b78a28\") " pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:36:31.013405 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:31.013383    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/270b4e59-f874-4227-9207-7fee83b78a28-istio-data\") pod \"e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8\" (UID: \"270b4e59-f874-4227-9207-7fee83b78a28\") " pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:36:31.114847 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:31.114817    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/270b4e59-f874-4227-9207-7fee83b78a28-workload-socket\") pod \"e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8\" (UID: \"270b4e59-f874-4227-9207-7fee83b78a28\") " pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:36:31.114847 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:31.114851    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/270b4e59-f874-4227-9207-7fee83b78a28-istiod-ca-cert\") pod \"e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8\" (UID: \"270b4e59-f874-4227-9207-7fee83b78a28\") " pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:36:31.115117 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:31.114897    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/270b4e59-f874-4227-9207-7fee83b78a28-istio-token\") pod \"e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8\" (UID: \"270b4e59-f874-4227-9207-7fee83b78a28\") " pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:36:31.115117 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:31.115100    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/270b4e59-f874-4227-9207-7fee83b78a28-istio-envoy\") pod \"e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8\" (UID: \"270b4e59-f874-4227-9207-7fee83b78a28\") " pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:36:31.115236 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:31.115162    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dg8gm\" (UniqueName: \"kubernetes.io/projected/270b4e59-f874-4227-9207-7fee83b78a28-kube-api-access-dg8gm\") pod \"e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8\" (UID: \"270b4e59-f874-4227-9207-7fee83b78a28\") " pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:36:31.115236 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:31.115202    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/270b4e59-f874-4227-9207-7fee83b78a28-workload-certs\") pod \"e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8\" (UID: \"270b4e59-f874-4227-9207-7fee83b78a28\") " pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:36:31.115236 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:31.115231    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/270b4e59-f874-4227-9207-7fee83b78a28-istio-podinfo\") pod \"e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8\" (UID: \"270b4e59-f874-4227-9207-7fee83b78a28\") " pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:36:31.115433 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:31.115264    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/270b4e59-f874-4227-9207-7fee83b78a28-workload-socket\") pod \"e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8\" (UID: \"270b4e59-f874-4227-9207-7fee83b78a28\") " pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:36:31.115433 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:31.115276    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/270b4e59-f874-4227-9207-7fee83b78a28-istio-data\") pod \"e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8\" (UID: \"270b4e59-f874-4227-9207-7fee83b78a28\") " pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:36:31.115433 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:31.115389    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/270b4e59-f874-4227-9207-7fee83b78a28-credential-socket\") pod \"e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8\" (UID: \"270b4e59-f874-4227-9207-7fee83b78a28\") " pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:36:31.115601 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:31.115483    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/270b4e59-f874-4227-9207-7fee83b78a28-workload-certs\") pod \"e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8\" (UID: \"270b4e59-f874-4227-9207-7fee83b78a28\") " pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:36:31.115671 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:31.115651    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/270b4e59-f874-4227-9207-7fee83b78a28-credential-socket\") pod \"e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8\" (UID: \"270b4e59-f874-4227-9207-7fee83b78a28\") " pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:36:31.115714 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:31.115666    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/270b4e59-f874-4227-9207-7fee83b78a28-istiod-ca-cert\") pod \"e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8\" (UID: \"270b4e59-f874-4227-9207-7fee83b78a28\") " pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:36:31.115754 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:31.115711    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/270b4e59-f874-4227-9207-7fee83b78a28-istio-data\") pod \"e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8\" (UID: \"270b4e59-f874-4227-9207-7fee83b78a28\") " pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:36:31.117368 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:31.117349    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/270b4e59-f874-4227-9207-7fee83b78a28-istio-envoy\") pod \"e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8\" (UID: \"270b4e59-f874-4227-9207-7fee83b78a28\") " pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:36:31.117566 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:31.117550    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/270b4e59-f874-4227-9207-7fee83b78a28-istio-podinfo\") pod \"e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8\" (UID: \"270b4e59-f874-4227-9207-7fee83b78a28\") " pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:36:31.122966 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:31.122940    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dg8gm\" (UniqueName: \"kubernetes.io/projected/270b4e59-f874-4227-9207-7fee83b78a28-kube-api-access-dg8gm\") pod \"e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8\" (UID: \"270b4e59-f874-4227-9207-7fee83b78a28\") " pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:36:31.123101 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:31.123048    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/270b4e59-f874-4227-9207-7fee83b78a28-istio-token\") pod \"e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8\" (UID: \"270b4e59-f874-4227-9207-7fee83b78a28\") " pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:36:31.163761 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:31.163734    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:36:31.291425 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:31.291397    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:36:31.426835 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:31.426814    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"]
Jul 30 04:36:31.428930 ip-10-0-143-117 kubenswrapper[2576]: W0730 04:36:31.428880    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod270b4e59_f874_4227_9207_7fee83b78a28.slice/crio-b48467aed6de9d476b801887f02d1328a40e5b5789b34de03c99d31b16eb9876 WatchSource:0}: Error finding container b48467aed6de9d476b801887f02d1328a40e5b5789b34de03c99d31b16eb9876: Status 404 returned error can't find the container with id b48467aed6de9d476b801887f02d1328a40e5b5789b34de03c99d31b16eb9876
Jul 30 04:36:31.431272 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:31.431235    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:36:31.431343 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:31.431324    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:36:31.431389 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:31.431372    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:36:32.104651 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:32.104611    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8" event={"ID":"270b4e59-f874-4227-9207-7fee83b78a28","Type":"ContainerStarted","Data":"c49f770156d6d69abed16e174e8f368d239196c0c9991572463e51b87c5d32ba"}
Jul 30 04:36:32.105049 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:32.104696    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8" event={"ID":"270b4e59-f874-4227-9207-7fee83b78a28","Type":"ContainerStarted","Data":"b48467aed6de9d476b801887f02d1328a40e5b5789b34de03c99d31b16eb9876"}
Jul 30 04:36:32.125222 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:32.125172    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8" podStartSLOduration=2.125158043 podStartE2EDuration="2.125158043s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:36:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:36:32.123547522 +0000 UTC m=+3697.741779764" watchObservedRunningTime="2026-07-30 04:36:32.125158043 +0000 UTC m=+3697.743390284"
Jul 30 04:36:32.291938 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:32.291898    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:36:32.296831 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:32.296802    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:36:33.110174 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:33.110146    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:36:33.110975 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:33.110957    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:36:33.989033 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:33.989003    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:36:34.090707 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:34.090677    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:36:35.110707 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:35.110673    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:36:35.713243 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:35.713208    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:36:35.813480 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:35.813443    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:36:43.227136 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:43.227094    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:36:52.215575 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:52.215542    2576 generic.go:356] "Generic (PLEG): container finished" podID="270b4e59-f874-4227-9207-7fee83b78a28" containerID="c49f770156d6d69abed16e174e8f368d239196c0c9991572463e51b87c5d32ba" exitCode=0
Jul 30 04:36:52.216016 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:52.215614    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8" event={"ID":"270b4e59-f874-4227-9207-7fee83b78a28","Type":"ContainerDied","Data":"c49f770156d6d69abed16e174e8f368d239196c0c9991572463e51b87c5d32ba"}
Jul 30 04:36:52.216072 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:52.216040    2576 scope.go:117] "RemoveContainer" containerID="c49f770156d6d69abed16e174e8f368d239196c0c9991572463e51b87c5d32ba"
Jul 30 04:36:52.217115 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:52.217089    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:36:52.217187 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:52.217154    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:36:52.217224 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:52.217203    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:36:53.223711 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:53.223674    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8" event={"ID":"270b4e59-f874-4227-9207-7fee83b78a28","Type":"ContainerStarted","Data":"adc22403a33628304664f5ecb138494df4c3ea8337f180e4930b1e02fe08c5d0"}
Jul 30 04:36:53.292441 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:53.292407    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:36:53.292618 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:53.292599    2576 patch_prober.go:28] interesting pod/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Jul 30 04:36:53.292680 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:53.292657    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8" podUID="270b4e59-f874-4227-9207-7fee83b78a28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Jul 30 04:36:54.292066 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:54.292031    2576 patch_prober.go:28] interesting pod/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Jul 30 04:36:54.292437 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:54.292080    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8" podUID="270b4e59-f874-4227-9207-7fee83b78a28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Jul 30 04:36:55.291856 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:55.291828    2576 patch_prober.go:28] interesting pod/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Jul 30 04:36:55.292057 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:55.291877    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8" podUID="270b4e59-f874-4227-9207-7fee83b78a28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Jul 30 04:36:56.292042 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:56.292001    2576 patch_prober.go:28] interesting pod/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Jul 30 04:36:56.292458 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:56.292057    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8" podUID="270b4e59-f874-4227-9207-7fee83b78a28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Jul 30 04:36:57.295843 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:57.295808    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:36:57.296340 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:57.296198    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:36:57.296741 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:57.296724    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:36:58.932692 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:58.932659    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:36:59.565370 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:59.565335    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:36:59.832785 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:59.832697    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"]
Jul 30 04:36:59.836991 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:59.836963    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:36:59.839523 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:59.839496    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-1d8c4c-openshift-default-dockercfg-4rxh5\""
Jul 30 04:36:59.846179 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:59.846155    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"]
Jul 30 04:36:59.860979 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:59.860950    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:36:59.904358 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:59.904322    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d86441ab-8879-46b5-8bcf-7c659debb820-istio-podinfo\") pod \"e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj\" (UID: \"d86441ab-8879-46b5-8bcf-7c659debb820\") " pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:36:59.904536 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:59.904361    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d86441ab-8879-46b5-8bcf-7c659debb820-istio-token\") pod \"e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj\" (UID: \"d86441ab-8879-46b5-8bcf-7c659debb820\") " pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:36:59.904536 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:59.904428    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d86441ab-8879-46b5-8bcf-7c659debb820-workload-certs\") pod \"e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj\" (UID: \"d86441ab-8879-46b5-8bcf-7c659debb820\") " pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:36:59.904536 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:59.904483    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d86441ab-8879-46b5-8bcf-7c659debb820-credential-socket\") pod \"e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj\" (UID: \"d86441ab-8879-46b5-8bcf-7c659debb820\") " pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:36:59.904536 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:59.904524    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tppzj\" (UniqueName: \"kubernetes.io/projected/d86441ab-8879-46b5-8bcf-7c659debb820-kube-api-access-tppzj\") pod \"e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj\" (UID: \"d86441ab-8879-46b5-8bcf-7c659debb820\") " pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:36:59.904701 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:59.904565    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d86441ab-8879-46b5-8bcf-7c659debb820-istio-data\") pod \"e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj\" (UID: \"d86441ab-8879-46b5-8bcf-7c659debb820\") " pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:36:59.904701 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:59.904592    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d86441ab-8879-46b5-8bcf-7c659debb820-workload-socket\") pod \"e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj\" (UID: \"d86441ab-8879-46b5-8bcf-7c659debb820\") " pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:36:59.904701 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:59.904690    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d86441ab-8879-46b5-8bcf-7c659debb820-istiod-ca-cert\") pod \"e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj\" (UID: \"d86441ab-8879-46b5-8bcf-7c659debb820\") " pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:36:59.904803 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:36:59.904722    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d86441ab-8879-46b5-8bcf-7c659debb820-istio-envoy\") pod \"e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj\" (UID: \"d86441ab-8879-46b5-8bcf-7c659debb820\") " pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:37:00.005773 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:00.005735    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d86441ab-8879-46b5-8bcf-7c659debb820-istio-envoy\") pod \"e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj\" (UID: \"d86441ab-8879-46b5-8bcf-7c659debb820\") " pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:37:00.006195 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:00.005849    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d86441ab-8879-46b5-8bcf-7c659debb820-istio-podinfo\") pod \"e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj\" (UID: \"d86441ab-8879-46b5-8bcf-7c659debb820\") " pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:37:00.006195 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:00.005871    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d86441ab-8879-46b5-8bcf-7c659debb820-istio-token\") pod \"e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj\" (UID: \"d86441ab-8879-46b5-8bcf-7c659debb820\") " pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:37:00.006195 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:00.005893    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d86441ab-8879-46b5-8bcf-7c659debb820-workload-certs\") pod \"e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj\" (UID: \"d86441ab-8879-46b5-8bcf-7c659debb820\") " pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:37:00.006195 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:00.006049    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d86441ab-8879-46b5-8bcf-7c659debb820-credential-socket\") pod \"e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj\" (UID: \"d86441ab-8879-46b5-8bcf-7c659debb820\") " pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:37:00.006195 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:00.006086    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tppzj\" (UniqueName: \"kubernetes.io/projected/d86441ab-8879-46b5-8bcf-7c659debb820-kube-api-access-tppzj\") pod \"e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj\" (UID: \"d86441ab-8879-46b5-8bcf-7c659debb820\") " pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:37:00.006195 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:00.006131    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d86441ab-8879-46b5-8bcf-7c659debb820-istio-data\") pod \"e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj\" (UID: \"d86441ab-8879-46b5-8bcf-7c659debb820\") " pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:37:00.006195 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:00.006168    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d86441ab-8879-46b5-8bcf-7c659debb820-workload-socket\") pod \"e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj\" (UID: \"d86441ab-8879-46b5-8bcf-7c659debb820\") " pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:37:00.006566 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:00.006234    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d86441ab-8879-46b5-8bcf-7c659debb820-istiod-ca-cert\") pod \"e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj\" (UID: \"d86441ab-8879-46b5-8bcf-7c659debb820\") " pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:37:00.006566 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:00.006253    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d86441ab-8879-46b5-8bcf-7c659debb820-workload-certs\") pod \"e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj\" (UID: \"d86441ab-8879-46b5-8bcf-7c659debb820\") " pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:37:00.006566 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:00.006399    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d86441ab-8879-46b5-8bcf-7c659debb820-credential-socket\") pod \"e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj\" (UID: \"d86441ab-8879-46b5-8bcf-7c659debb820\") " pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:37:00.006566 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:00.006454    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d86441ab-8879-46b5-8bcf-7c659debb820-istio-data\") pod \"e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj\" (UID: \"d86441ab-8879-46b5-8bcf-7c659debb820\") " pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:37:00.006699 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:00.006629    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d86441ab-8879-46b5-8bcf-7c659debb820-workload-socket\") pod \"e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj\" (UID: \"d86441ab-8879-46b5-8bcf-7c659debb820\") " pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:37:00.006885 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:00.006868    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d86441ab-8879-46b5-8bcf-7c659debb820-istiod-ca-cert\") pod \"e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj\" (UID: \"d86441ab-8879-46b5-8bcf-7c659debb820\") " pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:37:00.007961 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:00.007940    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d86441ab-8879-46b5-8bcf-7c659debb820-istio-envoy\") pod \"e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj\" (UID: \"d86441ab-8879-46b5-8bcf-7c659debb820\") " pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:37:00.008182 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:00.008165    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d86441ab-8879-46b5-8bcf-7c659debb820-istio-podinfo\") pod \"e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj\" (UID: \"d86441ab-8879-46b5-8bcf-7c659debb820\") " pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:37:00.016731 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:00.016709    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tppzj\" (UniqueName: \"kubernetes.io/projected/d86441ab-8879-46b5-8bcf-7c659debb820-kube-api-access-tppzj\") pod \"e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj\" (UID: \"d86441ab-8879-46b5-8bcf-7c659debb820\") " pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:37:00.016857 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:00.016782    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d86441ab-8879-46b5-8bcf-7c659debb820-istio-token\") pod \"e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj\" (UID: \"d86441ab-8879-46b5-8bcf-7c659debb820\") " pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:37:00.149205 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:00.149106    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:37:00.235695 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:00.235668    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:37:00.305245 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:00.305204    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"]
Jul 30 04:37:00.307523 ip-10-0-143-117 kubenswrapper[2576]: W0730 04:37:00.307498    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd86441ab_8879_46b5_8bcf_7c659debb820.slice/crio-84412ef417f872f2ec7eeb4c62bd948d2df7f54a875c6c60e7ea398dc4c65e40 WatchSource:0}: Error finding container 84412ef417f872f2ec7eeb4c62bd948d2df7f54a875c6c60e7ea398dc4c65e40: Status 404 returned error can't find the container with id 84412ef417f872f2ec7eeb4c62bd948d2df7f54a875c6c60e7ea398dc4c65e40
Jul 30 04:37:00.309618 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:00.309587    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:37:00.309709 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:00.309661    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:37:00.309773 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:00.309709    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:37:01.274425 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:01.274388    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj" event={"ID":"d86441ab-8879-46b5-8bcf-7c659debb820","Type":"ContainerStarted","Data":"edb56b72549a837699ad8767f72fecfbf5ddae2eeb32d7c27e60d618ce5c66b0"}
Jul 30 04:37:01.274814 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:01.274454    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj" event={"ID":"d86441ab-8879-46b5-8bcf-7c659debb820","Type":"ContainerStarted","Data":"84412ef417f872f2ec7eeb4c62bd948d2df7f54a875c6c60e7ea398dc4c65e40"}
Jul 30 04:37:01.296969 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:01.296903    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj" podStartSLOduration=2.296888931 podStartE2EDuration="2.296888931s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:36:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:37:01.293679386 +0000 UTC m=+3726.911911638" watchObservedRunningTime="2026-07-30 04:37:01.296888931 +0000 UTC m=+3726.915121172"
Jul 30 04:37:02.149295 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:02.149264    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:37:02.154597 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:02.154574    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:37:02.280315 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:02.280284    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:37:02.281348 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:02.281330    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:37:03.170838 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:03.170802    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:37:03.270634 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:03.270588    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:37:04.356746 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:04.356712    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:37:04.619215 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:04.619131    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:37:04.799226 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:04.799194    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:37:04.911768 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:04.911667    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:37:12.736413 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:12.736382    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:37:21.398433 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:21.398403    2576 generic.go:356] "Generic (PLEG): container finished" podID="d86441ab-8879-46b5-8bcf-7c659debb820" containerID="edb56b72549a837699ad8767f72fecfbf5ddae2eeb32d7c27e60d618ce5c66b0" exitCode=0
Jul 30 04:37:21.398815 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:21.398474    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj" event={"ID":"d86441ab-8879-46b5-8bcf-7c659debb820","Type":"ContainerDied","Data":"edb56b72549a837699ad8767f72fecfbf5ddae2eeb32d7c27e60d618ce5c66b0"}
Jul 30 04:37:21.398879 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:21.398865    2576 scope.go:117] "RemoveContainer" containerID="edb56b72549a837699ad8767f72fecfbf5ddae2eeb32d7c27e60d618ce5c66b0"
Jul 30 04:37:21.400020 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:21.399991    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:37:21.400085 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:21.400061    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:37:21.400123 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:21.400095    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:37:22.406369 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:22.406338    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj" event={"ID":"d86441ab-8879-46b5-8bcf-7c659debb820","Type":"ContainerStarted","Data":"514a3b4486fa05fd7d78667d2eaf3bffdc9aee5e96ce25cac2f2c43be1c77d7f"}
Jul 30 04:37:23.150308 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:23.150262    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:37:23.150538 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:23.150399    2576 patch_prober.go:28] interesting pod/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Jul 30 04:37:23.150538 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:23.150478    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj" podUID="d86441ab-8879-46b5-8bcf-7c659debb820" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Jul 30 04:37:24.149695 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:24.149648    2576 patch_prober.go:28] interesting pod/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Jul 30 04:37:24.150223 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:24.149728    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj" podUID="d86441ab-8879-46b5-8bcf-7c659debb820" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Jul 30 04:37:25.150521 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:25.150485    2576 patch_prober.go:28] interesting pod/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Jul 30 04:37:25.151147 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:25.151106    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj" podUID="d86441ab-8879-46b5-8bcf-7c659debb820" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Jul 30 04:37:26.150254 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:26.150210    2576 patch_prober.go:28] interesting pod/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Jul 30 04:37:26.150466 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:26.150276    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj" podUID="d86441ab-8879-46b5-8bcf-7c659debb820" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Jul 30 04:37:27.153225 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:27.153193    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:37:27.153706 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:27.153602    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:37:27.154267 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:27.154251    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:37:28.129192 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:28.129163    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:37:34.892969 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:34.892919    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-8qlzb_9b4ba3f1-07c7-4335-97ad-bf7c9269f7c3/payload-processing/0.log"
Jul 30 04:37:41.949529 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:41.949496    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:37:45.624529 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:37:45.624500    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:38:20.025533 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:20.025497    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:38:29.964164 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:29.964131    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-8qlzb_9b4ba3f1-07c7-4335-97ad-bf7c9269f7c3/payload-processing/0.log"
Jul 30 04:38:30.343319 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:30.343289    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:38:30.628257 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:30.628182    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"]
Jul 30 04:38:30.631976 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:30.631956    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:38:30.634645 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:30.634627    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-410e1b-openshift-default-dockercfg-tkp28\""
Jul 30 04:38:30.642994 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:30.642952    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"]
Jul 30 04:38:30.647097 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:30.647044    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:38:30.778193 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:30.778159    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:38:30.796530 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:30.796502    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f3527cd-7d6d-4f75-9de1-fe794e198de0-workload-socket\") pod \"e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c\" (UID: \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\") " pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:38:30.796669 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:30.796532    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f3527cd-7d6d-4f75-9de1-fe794e198de0-istio-token\") pod \"e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c\" (UID: \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\") " pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:38:30.796669 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:30.796560    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f3527cd-7d6d-4f75-9de1-fe794e198de0-workload-certs\") pod \"e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c\" (UID: \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\") " pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:38:30.796669 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:30.796582    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f3527cd-7d6d-4f75-9de1-fe794e198de0-istio-data\") pod \"e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c\" (UID: \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\") " pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:38:30.796790 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:30.796711    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gw27q\" (UniqueName: \"kubernetes.io/projected/7f3527cd-7d6d-4f75-9de1-fe794e198de0-kube-api-access-gw27q\") pod \"e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c\" (UID: \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\") " pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:38:30.796790 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:30.796784    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f3527cd-7d6d-4f75-9de1-fe794e198de0-istio-envoy\") pod \"e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c\" (UID: \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\") " pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:38:30.796868 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:30.796801    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f3527cd-7d6d-4f75-9de1-fe794e198de0-istiod-ca-cert\") pod \"e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c\" (UID: \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\") " pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:38:30.796905 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:30.796867    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f3527cd-7d6d-4f75-9de1-fe794e198de0-credential-socket\") pod \"e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c\" (UID: \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\") " pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:38:30.796905 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:30.796894    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f3527cd-7d6d-4f75-9de1-fe794e198de0-istio-podinfo\") pod \"e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c\" (UID: \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\") " pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:38:30.897688 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:30.897603    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f3527cd-7d6d-4f75-9de1-fe794e198de0-credential-socket\") pod \"e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c\" (UID: \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\") " pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:38:30.897688 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:30.897643    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f3527cd-7d6d-4f75-9de1-fe794e198de0-istio-podinfo\") pod \"e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c\" (UID: \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\") " pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:38:30.897904 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:30.897707    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f3527cd-7d6d-4f75-9de1-fe794e198de0-workload-socket\") pod \"e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c\" (UID: \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\") " pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:38:30.897904 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:30.897733    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f3527cd-7d6d-4f75-9de1-fe794e198de0-istio-token\") pod \"e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c\" (UID: \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\") " pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:38:30.897904 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:30.897880    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f3527cd-7d6d-4f75-9de1-fe794e198de0-workload-certs\") pod \"e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c\" (UID: \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\") " pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:38:30.898113 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:30.897967    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f3527cd-7d6d-4f75-9de1-fe794e198de0-istio-data\") pod \"e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c\" (UID: \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\") " pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:38:30.898113 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:30.898009    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gw27q\" (UniqueName: \"kubernetes.io/projected/7f3527cd-7d6d-4f75-9de1-fe794e198de0-kube-api-access-gw27q\") pod \"e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c\" (UID: \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\") " pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:38:30.898113 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:30.898073    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f3527cd-7d6d-4f75-9de1-fe794e198de0-credential-socket\") pod \"e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c\" (UID: \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\") " pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:38:30.898113 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:30.898108    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f3527cd-7d6d-4f75-9de1-fe794e198de0-istio-envoy\") pod \"e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c\" (UID: \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\") " pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:38:30.898323 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:30.898131    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f3527cd-7d6d-4f75-9de1-fe794e198de0-workload-socket\") pod \"e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c\" (UID: \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\") " pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:38:30.898323 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:30.898135    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f3527cd-7d6d-4f75-9de1-fe794e198de0-istiod-ca-cert\") pod \"e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c\" (UID: \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\") " pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:38:30.898323 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:30.898236    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f3527cd-7d6d-4f75-9de1-fe794e198de0-workload-certs\") pod \"e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c\" (UID: \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\") " pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:38:30.898323 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:30.898292    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f3527cd-7d6d-4f75-9de1-fe794e198de0-istio-data\") pod \"e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c\" (UID: \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\") " pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:38:30.898759 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:30.898738    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f3527cd-7d6d-4f75-9de1-fe794e198de0-istiod-ca-cert\") pod \"e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c\" (UID: \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\") " pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:38:30.900281 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:30.900262    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f3527cd-7d6d-4f75-9de1-fe794e198de0-istio-envoy\") pod \"e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c\" (UID: \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\") " pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:38:30.900374 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:30.900263    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f3527cd-7d6d-4f75-9de1-fe794e198de0-istio-podinfo\") pod \"e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c\" (UID: \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\") " pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:38:30.906069 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:30.906044    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f3527cd-7d6d-4f75-9de1-fe794e198de0-istio-token\") pod \"e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c\" (UID: \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\") " pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:38:30.906298 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:30.906282    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gw27q\" (UniqueName: \"kubernetes.io/projected/7f3527cd-7d6d-4f75-9de1-fe794e198de0-kube-api-access-gw27q\") pod \"e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c\" (UID: \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\") " pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:38:30.943726 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:30.943706    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:38:31.083128 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:31.083103    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"]
Jul 30 04:38:31.083891 ip-10-0-143-117 kubenswrapper[2576]: W0730 04:38:31.083868    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f3527cd_7d6d_4f75_9de1_fe794e198de0.slice/crio-63edc8b8f51e4d6c0dea4efa9e716624ad433bbf6168453cf3120369c223097c WatchSource:0}: Error finding container 63edc8b8f51e4d6c0dea4efa9e716624ad433bbf6168453cf3120369c223097c: Status 404 returned error can't find the container with id 63edc8b8f51e4d6c0dea4efa9e716624ad433bbf6168453cf3120369c223097c
Jul 30 04:38:31.086205 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:31.086172    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:38:31.086299 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:31.086240    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:38:31.086299 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:31.086268    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:38:31.808633 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:31.808595    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c" event={"ID":"7f3527cd-7d6d-4f75-9de1-fe794e198de0","Type":"ContainerStarted","Data":"ebcbdecfbe878d583ab281da33badd306450fa6d8ae43cc9a32f37418ca3db3b"}
Jul 30 04:38:31.808818 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:31.808675    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c" event={"ID":"7f3527cd-7d6d-4f75-9de1-fe794e198de0","Type":"ContainerStarted","Data":"63edc8b8f51e4d6c0dea4efa9e716624ad433bbf6168453cf3120369c223097c"}
Jul 30 04:38:31.828493 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:31.828444    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c" podStartSLOduration=1.8284293360000001 podStartE2EDuration="1.828429336s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:38:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:38:31.827338686 +0000 UTC m=+3817.445570928" watchObservedRunningTime="2026-07-30 04:38:31.828429336 +0000 UTC m=+3817.446661625"
Jul 30 04:38:31.944827 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:31.944793    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:38:32.949026 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:32.948999    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:38:33.713236 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:33.713204    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:38:33.812338 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:33.812282    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:38:33.821606 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:33.821583    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:38:33.822307 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:33.822290    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:38:34.835101 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:34.835070    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:38:35.059689 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:35.059649    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:38:35.335058 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:35.335006    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:38:35.432125 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:35.432093    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:38:40.227388 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:40.227351    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:38:48.823194 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:48.823161    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Jul 30 04:38:48.823586 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:48.823222    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c" podUID="7f3527cd-7d6d-4f75-9de1-fe794e198de0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Jul 30 04:38:48.910331 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:48.910301    2576 generic.go:356] "Generic (PLEG): container finished" podID="7f3527cd-7d6d-4f75-9de1-fe794e198de0" containerID="ebcbdecfbe878d583ab281da33badd306450fa6d8ae43cc9a32f37418ca3db3b" exitCode=0
Jul 30 04:38:48.910753 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:48.910372    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c" event={"ID":"7f3527cd-7d6d-4f75-9de1-fe794e198de0","Type":"ContainerDied","Data":"ebcbdecfbe878d583ab281da33badd306450fa6d8ae43cc9a32f37418ca3db3b"}
Jul 30 04:38:48.910900 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:48.910884    2576 scope.go:117] "RemoveContainer" containerID="ebcbdecfbe878d583ab281da33badd306450fa6d8ae43cc9a32f37418ca3db3b"
Jul 30 04:38:48.912073 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:48.912044    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:38:48.912168 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:48.912113    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:38:48.912168 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:48.912142    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 04:38:49.918027 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:49.917994    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c" event={"ID":"7f3527cd-7d6d-4f75-9de1-fe794e198de0","Type":"ContainerStarted","Data":"e7ac5401d32c5110ad3d3963b00405f847e9af9fb44da19d67669c177a2ac0d8"}
Jul 30 04:38:49.944631 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:49.944552    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:38:49.945218 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:49.944906    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Jul 30 04:38:49.945218 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:49.944976    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c" podUID="7f3527cd-7d6d-4f75-9de1-fe794e198de0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Jul 30 04:38:50.944766 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:50.944731    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Jul 30 04:38:50.945459 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:50.945425    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c" podUID="7f3527cd-7d6d-4f75-9de1-fe794e198de0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Jul 30 04:38:51.944944 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:51.944882    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Jul 30 04:38:51.945561 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:51.944973    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c" podUID="7f3527cd-7d6d-4f75-9de1-fe794e198de0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Jul 30 04:38:52.944965 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:52.944900    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Jul 30 04:38:52.945504 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:52.944987    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c" podUID="7f3527cd-7d6d-4f75-9de1-fe794e198de0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Jul 30 04:38:53.948505 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:53.948477    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:38:53.948959 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:53.948918    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:38:53.949470 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:53.949451    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:38:58.641669 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:38:58.641639    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:39:15.439738 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:15.439697    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:39:15.507707 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:15.507679    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:39:15.567121 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:15.567087    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:39:15.627955 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:15.627897    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:39:15.979990 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:15.979960    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:39:20.624755 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:20.624719    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:39:21.673403 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:21.673366    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:39:21.773868 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:21.773827    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:39:21.928756 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:21.928661    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"]
Jul 30 04:39:21.928935 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:21.928891    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c" podUID="7f3527cd-7d6d-4f75-9de1-fe794e198de0" containerName="istio-proxy" containerID="cri-o://e7ac5401d32c5110ad3d3963b00405f847e9af9fb44da19d67669c177a2ac0d8" gracePeriod=30
Jul 30 04:39:21.945847 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:21.945808    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:39:22.152101 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:22.152071    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:39:23.950625 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:23.950585    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 30 04:39:23.951019 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:23.950705    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c" podUID="7f3527cd-7d6d-4f75-9de1-fe794e198de0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 04:39:27.143621 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:27.143586    2576 generic.go:356] "Generic (PLEG): container finished" podID="7f3527cd-7d6d-4f75-9de1-fe794e198de0" containerID="e7ac5401d32c5110ad3d3963b00405f847e9af9fb44da19d67669c177a2ac0d8" exitCode=0
Jul 30 04:39:27.144034 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:27.143662    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c" event={"ID":"7f3527cd-7d6d-4f75-9de1-fe794e198de0","Type":"ContainerDied","Data":"e7ac5401d32c5110ad3d3963b00405f847e9af9fb44da19d67669c177a2ac0d8"}
Jul 30 04:39:27.144034 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:27.143733    2576 scope.go:117] "RemoveContainer" containerID="ebcbdecfbe878d583ab281da33badd306450fa6d8ae43cc9a32f37418ca3db3b"
Jul 30 04:39:27.188559 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:27.188535    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:39:27.343946 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:27.343903    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f3527cd-7d6d-4f75-9de1-fe794e198de0-istio-data\") pod \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\" (UID: \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\") "
Jul 30 04:39:27.344097 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:27.343976    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f3527cd-7d6d-4f75-9de1-fe794e198de0-istio-podinfo\") pod \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\" (UID: \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\") "
Jul 30 04:39:27.344097 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:27.344023    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f3527cd-7d6d-4f75-9de1-fe794e198de0-istiod-ca-cert\") pod \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\" (UID: \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\") "
Jul 30 04:39:27.344097 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:27.344069    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f3527cd-7d6d-4f75-9de1-fe794e198de0-workload-certs\") pod \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\" (UID: \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\") "
Jul 30 04:39:27.344097 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:27.344093    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f3527cd-7d6d-4f75-9de1-fe794e198de0-credential-socket\") pod \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\" (UID: \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\") "
Jul 30 04:39:27.344336 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:27.344121    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f3527cd-7d6d-4f75-9de1-fe794e198de0-workload-socket\") pod \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\" (UID: \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\") "
Jul 30 04:39:27.344336 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:27.344152    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f3527cd-7d6d-4f75-9de1-fe794e198de0-istio-token\") pod \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\" (UID: \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\") "
Jul 30 04:39:27.344336 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:27.344211    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gw27q\" (UniqueName: \"kubernetes.io/projected/7f3527cd-7d6d-4f75-9de1-fe794e198de0-kube-api-access-gw27q\") pod \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\" (UID: \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\") "
Jul 30 04:39:27.344336 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:27.344276    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f3527cd-7d6d-4f75-9de1-fe794e198de0-istio-envoy\") pod \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\" (UID: \"7f3527cd-7d6d-4f75-9de1-fe794e198de0\") "
Jul 30 04:39:27.344554 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:27.344359    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f3527cd-7d6d-4f75-9de1-fe794e198de0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7f3527cd-7d6d-4f75-9de1-fe794e198de0" (UID: "7f3527cd-7d6d-4f75-9de1-fe794e198de0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:39:27.344554 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:27.344433    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f3527cd-7d6d-4f75-9de1-fe794e198de0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7f3527cd-7d6d-4f75-9de1-fe794e198de0" (UID: "7f3527cd-7d6d-4f75-9de1-fe794e198de0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:39:27.344554 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:27.344528    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f3527cd-7d6d-4f75-9de1-fe794e198de0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7f3527cd-7d6d-4f75-9de1-fe794e198de0" (UID: "7f3527cd-7d6d-4f75-9de1-fe794e198de0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:39:27.344791 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:27.344743    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f3527cd-7d6d-4f75-9de1-fe794e198de0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7f3527cd-7d6d-4f75-9de1-fe794e198de0" (UID: "7f3527cd-7d6d-4f75-9de1-fe794e198de0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:39:27.344917 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:27.344848    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f3527cd-7d6d-4f75-9de1-fe794e198de0-istiod-ca-cert\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:39:27.344917 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:27.344870    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f3527cd-7d6d-4f75-9de1-fe794e198de0-workload-certs\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:39:27.344917 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:27.344884    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f3527cd-7d6d-4f75-9de1-fe794e198de0-credential-socket\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:39:27.345118 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:27.344953    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f3527cd-7d6d-4f75-9de1-fe794e198de0-istio-data" (OuterVolumeSpecName: "istio-data") pod "7f3527cd-7d6d-4f75-9de1-fe794e198de0" (UID: "7f3527cd-7d6d-4f75-9de1-fe794e198de0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:39:27.346270 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:27.346247    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7f3527cd-7d6d-4f75-9de1-fe794e198de0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7f3527cd-7d6d-4f75-9de1-fe794e198de0" (UID: "7f3527cd-7d6d-4f75-9de1-fe794e198de0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 04:39:27.346385 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:27.346365    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f3527cd-7d6d-4f75-9de1-fe794e198de0-istio-token" (OuterVolumeSpecName: "istio-token") pod "7f3527cd-7d6d-4f75-9de1-fe794e198de0" (UID: "7f3527cd-7d6d-4f75-9de1-fe794e198de0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:39:27.346490 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:27.346469    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f3527cd-7d6d-4f75-9de1-fe794e198de0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7f3527cd-7d6d-4f75-9de1-fe794e198de0" (UID: "7f3527cd-7d6d-4f75-9de1-fe794e198de0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:39:27.346528 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:27.346489    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f3527cd-7d6d-4f75-9de1-fe794e198de0-kube-api-access-gw27q" (OuterVolumeSpecName: "kube-api-access-gw27q") pod "7f3527cd-7d6d-4f75-9de1-fe794e198de0" (UID: "7f3527cd-7d6d-4f75-9de1-fe794e198de0"). InnerVolumeSpecName "kube-api-access-gw27q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:39:27.445692 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:27.445653    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f3527cd-7d6d-4f75-9de1-fe794e198de0-istio-podinfo\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:39:27.445692 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:27.445685    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f3527cd-7d6d-4f75-9de1-fe794e198de0-workload-socket\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:39:27.445692 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:27.445695    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f3527cd-7d6d-4f75-9de1-fe794e198de0-istio-token\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:39:27.445955 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:27.445705    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gw27q\" (UniqueName: \"kubernetes.io/projected/7f3527cd-7d6d-4f75-9de1-fe794e198de0-kube-api-access-gw27q\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:39:27.445955 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:27.445714    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f3527cd-7d6d-4f75-9de1-fe794e198de0-istio-envoy\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:39:27.445955 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:27.445721    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f3527cd-7d6d-4f75-9de1-fe794e198de0-istio-data\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:39:28.151025 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:28.150627    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"
Jul 30 04:39:28.151025 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:28.150650    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c" event={"ID":"7f3527cd-7d6d-4f75-9de1-fe794e198de0","Type":"ContainerDied","Data":"63edc8b8f51e4d6c0dea4efa9e716624ad433bbf6168453cf3120369c223097c"}
Jul 30 04:39:28.151025 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:28.150699    2576 scope.go:117] "RemoveContainer" containerID="e7ac5401d32c5110ad3d3963b00405f847e9af9fb44da19d67669c177a2ac0d8"
Jul 30 04:39:28.173057 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:28.173031    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"]
Jul 30 04:39:28.175324 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:28.175303    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-410e1b-openshift-default-9bbc5bd46-8tq8c"]
Jul 30 04:39:29.051284 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:29.051252    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f3527cd-7d6d-4f75-9de1-fe794e198de0" path="/var/lib/kubelet/pods/7f3527cd-7d6d-4f75-9de1-fe794e198de0/volumes"
Jul 30 04:39:40.891599 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:40.891566    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:39:40.967005 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:40.966975    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:39:41.067020 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:41.066983    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:39:41.168646 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:41.168565    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:39:41.441090 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:41.441015    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:39:42.524817 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:42.524782    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:39:46.528190 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:46.528149    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:39:49.398117 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:49.398085    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:39:51.829312 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:39:51.829278    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:40:22.861387 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:22.861350    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:40:23.583550 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:23.583518    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:40:29.132058 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:29.132022    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:40:29.233016 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:29.232978    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:40:29.380111 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:29.380080    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"]
Jul 30 04:40:29.380299 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:29.380278    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8" podUID="270b4e59-f874-4227-9207-7fee83b78a28" containerName="istio-proxy" containerID="cri-o://adc22403a33628304664f5ecb138494df4c3ea8337f180e4930b1e02fe08c5d0" gracePeriod=30
Jul 30 04:40:29.400244 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:29.400175    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:40:29.620441 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:29.620405    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:40:31.445298 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:31.445265    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:40:31.546033 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:31.546004    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:40:34.529325 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:34.529296    2576 generic.go:356] "Generic (PLEG): container finished" podID="270b4e59-f874-4227-9207-7fee83b78a28" containerID="adc22403a33628304664f5ecb138494df4c3ea8337f180e4930b1e02fe08c5d0" exitCode=0
Jul 30 04:40:34.529701 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:34.529356    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8" event={"ID":"270b4e59-f874-4227-9207-7fee83b78a28","Type":"ContainerDied","Data":"adc22403a33628304664f5ecb138494df4c3ea8337f180e4930b1e02fe08c5d0"}
Jul 30 04:40:34.529701 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:34.529425    2576 scope.go:117] "RemoveContainer" containerID="c49f770156d6d69abed16e174e8f368d239196c0c9991572463e51b87c5d32ba"
Jul 30 04:40:34.640491 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:34.640463    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:40:34.792703 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:34.792681    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/270b4e59-f874-4227-9207-7fee83b78a28-workload-certs\") pod \"270b4e59-f874-4227-9207-7fee83b78a28\" (UID: \"270b4e59-f874-4227-9207-7fee83b78a28\") "
Jul 30 04:40:34.792837 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:34.792732    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dg8gm\" (UniqueName: \"kubernetes.io/projected/270b4e59-f874-4227-9207-7fee83b78a28-kube-api-access-dg8gm\") pod \"270b4e59-f874-4227-9207-7fee83b78a28\" (UID: \"270b4e59-f874-4227-9207-7fee83b78a28\") "
Jul 30 04:40:34.792837 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:34.792763    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/270b4e59-f874-4227-9207-7fee83b78a28-workload-socket\") pod \"270b4e59-f874-4227-9207-7fee83b78a28\" (UID: \"270b4e59-f874-4227-9207-7fee83b78a28\") "
Jul 30 04:40:34.792837 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:34.792792    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/270b4e59-f874-4227-9207-7fee83b78a28-istio-data\") pod \"270b4e59-f874-4227-9207-7fee83b78a28\" (UID: \"270b4e59-f874-4227-9207-7fee83b78a28\") "
Jul 30 04:40:34.792837 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:34.792821    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/270b4e59-f874-4227-9207-7fee83b78a28-istiod-ca-cert\") pod \"270b4e59-f874-4227-9207-7fee83b78a28\" (UID: \"270b4e59-f874-4227-9207-7fee83b78a28\") "
Jul 30 04:40:34.793082 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:34.792850    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/270b4e59-f874-4227-9207-7fee83b78a28-istio-token\") pod \"270b4e59-f874-4227-9207-7fee83b78a28\" (UID: \"270b4e59-f874-4227-9207-7fee83b78a28\") "
Jul 30 04:40:34.793082 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:34.792953    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/270b4e59-f874-4227-9207-7fee83b78a28-istio-podinfo\") pod \"270b4e59-f874-4227-9207-7fee83b78a28\" (UID: \"270b4e59-f874-4227-9207-7fee83b78a28\") "
Jul 30 04:40:34.793082 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:34.792988    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/270b4e59-f874-4227-9207-7fee83b78a28-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "270b4e59-f874-4227-9207-7fee83b78a28" (UID: "270b4e59-f874-4227-9207-7fee83b78a28"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:40:34.793082 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:34.793014    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/270b4e59-f874-4227-9207-7fee83b78a28-istio-envoy\") pod \"270b4e59-f874-4227-9207-7fee83b78a28\" (UID: \"270b4e59-f874-4227-9207-7fee83b78a28\") "
Jul 30 04:40:34.793082 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:34.793042    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/270b4e59-f874-4227-9207-7fee83b78a28-credential-socket\") pod \"270b4e59-f874-4227-9207-7fee83b78a28\" (UID: \"270b4e59-f874-4227-9207-7fee83b78a28\") "
Jul 30 04:40:34.793355 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:34.793110    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/270b4e59-f874-4227-9207-7fee83b78a28-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "270b4e59-f874-4227-9207-7fee83b78a28" (UID: "270b4e59-f874-4227-9207-7fee83b78a28"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:40:34.793355 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:34.793309    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/270b4e59-f874-4227-9207-7fee83b78a28-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "270b4e59-f874-4227-9207-7fee83b78a28" (UID: "270b4e59-f874-4227-9207-7fee83b78a28"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:40:34.793477 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:34.793389    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/270b4e59-f874-4227-9207-7fee83b78a28-credential-socket\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:40:34.793477 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:34.793405    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/270b4e59-f874-4227-9207-7fee83b78a28-workload-certs\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:40:34.793477 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:34.793417    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/270b4e59-f874-4227-9207-7fee83b78a28-workload-socket\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:40:34.793727 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:34.793250    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/270b4e59-f874-4227-9207-7fee83b78a28-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "270b4e59-f874-4227-9207-7fee83b78a28" (UID: "270b4e59-f874-4227-9207-7fee83b78a28"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:40:34.793727 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:34.793704    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/270b4e59-f874-4227-9207-7fee83b78a28-istio-data" (OuterVolumeSpecName: "istio-data") pod "270b4e59-f874-4227-9207-7fee83b78a28" (UID: "270b4e59-f874-4227-9207-7fee83b78a28"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:40:34.794962 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:34.794912    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/270b4e59-f874-4227-9207-7fee83b78a28-istio-token" (OuterVolumeSpecName: "istio-token") pod "270b4e59-f874-4227-9207-7fee83b78a28" (UID: "270b4e59-f874-4227-9207-7fee83b78a28"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:40:34.794962 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:34.794955    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/270b4e59-f874-4227-9207-7fee83b78a28-kube-api-access-dg8gm" (OuterVolumeSpecName: "kube-api-access-dg8gm") pod "270b4e59-f874-4227-9207-7fee83b78a28" (UID: "270b4e59-f874-4227-9207-7fee83b78a28"). InnerVolumeSpecName "kube-api-access-dg8gm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:40:34.795569 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:34.795550    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/270b4e59-f874-4227-9207-7fee83b78a28-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "270b4e59-f874-4227-9207-7fee83b78a28" (UID: "270b4e59-f874-4227-9207-7fee83b78a28"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 04:40:34.795634 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:34.795546    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/270b4e59-f874-4227-9207-7fee83b78a28-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "270b4e59-f874-4227-9207-7fee83b78a28" (UID: "270b4e59-f874-4227-9207-7fee83b78a28"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:40:34.894752 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:34.894723    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/270b4e59-f874-4227-9207-7fee83b78a28-istio-envoy\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:40:34.894752 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:34.894749    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dg8gm\" (UniqueName: \"kubernetes.io/projected/270b4e59-f874-4227-9207-7fee83b78a28-kube-api-access-dg8gm\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:40:34.894898 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:34.894763    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/270b4e59-f874-4227-9207-7fee83b78a28-istio-data\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:40:34.894898 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:34.894775    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/270b4e59-f874-4227-9207-7fee83b78a28-istiod-ca-cert\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:40:34.894898 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:34.894786    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/270b4e59-f874-4227-9207-7fee83b78a28-istio-token\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:40:34.894898 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:34.894796    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/270b4e59-f874-4227-9207-7fee83b78a28-istio-podinfo\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:40:35.173057 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:35.173023    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:40:35.238477 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:35.238443    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:40:35.293283 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:35.293254    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:40:35.394880 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:35.394795    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:40:35.536466 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:35.536436    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"
Jul 30 04:40:35.536466 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:35.536461    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8" event={"ID":"270b4e59-f874-4227-9207-7fee83b78a28","Type":"ContainerDied","Data":"b48467aed6de9d476b801887f02d1328a40e5b5789b34de03c99d31b16eb9876"}
Jul 30 04:40:35.537014 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:35.536525    2576 scope.go:117] "RemoveContainer" containerID="adc22403a33628304664f5ecb138494df4c3ea8337f180e4930b1e02fe08c5d0"
Jul 30 04:40:35.556512 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:35.556481    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"]
Jul 30 04:40:35.560996 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:35.560968    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-77b2c7-openshift-default-d68b4d6d6-d8dr8"]
Jul 30 04:40:35.771700 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:35.771669    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:40:37.052382 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:37.052350    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="270b4e59-f874-4227-9207-7fee83b78a28" path="/var/lib/kubelet/pods/270b4e59-f874-4227-9207-7fee83b78a28/volumes"
Jul 30 04:40:38.525016 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:38.524986    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:40:41.683659 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:41.683624    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:40:41.746535 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:41.746504    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:40:41.923155 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:41.923127    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"]
Jul 30 04:40:41.923379 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:41.923353    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj" podUID="d86441ab-8879-46b5-8bcf-7c659debb820" containerName="istio-proxy" containerID="cri-o://514a3b4486fa05fd7d78667d2eaf3bffdc9aee5e96ce25cac2f2c43be1c77d7f" gracePeriod=30
Jul 30 04:40:41.946302 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:41.946230    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:40:42.156074 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:42.156034    2576 patch_prober.go:28] interesting pod/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 30 04:40:42.156229 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:42.156118    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj" podUID="d86441ab-8879-46b5-8bcf-7c659debb820" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 04:40:42.240662 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:42.240586    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:40:47.173499 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.173466    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:40:47.217111 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.217044    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d86441ab-8879-46b5-8bcf-7c659debb820-workload-certs\") pod \"d86441ab-8879-46b5-8bcf-7c659debb820\" (UID: \"d86441ab-8879-46b5-8bcf-7c659debb820\") "
Jul 30 04:40:47.217111 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.217087    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d86441ab-8879-46b5-8bcf-7c659debb820-istiod-ca-cert\") pod \"d86441ab-8879-46b5-8bcf-7c659debb820\" (UID: \"d86441ab-8879-46b5-8bcf-7c659debb820\") "
Jul 30 04:40:47.217300 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.217121    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tppzj\" (UniqueName: \"kubernetes.io/projected/d86441ab-8879-46b5-8bcf-7c659debb820-kube-api-access-tppzj\") pod \"d86441ab-8879-46b5-8bcf-7c659debb820\" (UID: \"d86441ab-8879-46b5-8bcf-7c659debb820\") "
Jul 30 04:40:47.217300 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.217242    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d86441ab-8879-46b5-8bcf-7c659debb820-istio-data\") pod \"d86441ab-8879-46b5-8bcf-7c659debb820\" (UID: \"d86441ab-8879-46b5-8bcf-7c659debb820\") "
Jul 30 04:40:47.217410 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.217313    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d86441ab-8879-46b5-8bcf-7c659debb820-workload-socket\") pod \"d86441ab-8879-46b5-8bcf-7c659debb820\" (UID: \"d86441ab-8879-46b5-8bcf-7c659debb820\") "
Jul 30 04:40:47.217410 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.217318    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d86441ab-8879-46b5-8bcf-7c659debb820-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d86441ab-8879-46b5-8bcf-7c659debb820" (UID: "d86441ab-8879-46b5-8bcf-7c659debb820"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:40:47.217410 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.217347    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d86441ab-8879-46b5-8bcf-7c659debb820-istio-envoy\") pod \"d86441ab-8879-46b5-8bcf-7c659debb820\" (UID: \"d86441ab-8879-46b5-8bcf-7c659debb820\") "
Jul 30 04:40:47.217410 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.217395    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d86441ab-8879-46b5-8bcf-7c659debb820-istio-token\") pod \"d86441ab-8879-46b5-8bcf-7c659debb820\" (UID: \"d86441ab-8879-46b5-8bcf-7c659debb820\") "
Jul 30 04:40:47.217619 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.217433    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d86441ab-8879-46b5-8bcf-7c659debb820-istio-podinfo\") pod \"d86441ab-8879-46b5-8bcf-7c659debb820\" (UID: \"d86441ab-8879-46b5-8bcf-7c659debb820\") "
Jul 30 04:40:47.217619 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.217489    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d86441ab-8879-46b5-8bcf-7c659debb820-credential-socket\") pod \"d86441ab-8879-46b5-8bcf-7c659debb820\" (UID: \"d86441ab-8879-46b5-8bcf-7c659debb820\") "
Jul 30 04:40:47.217619 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.217495    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d86441ab-8879-46b5-8bcf-7c659debb820-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d86441ab-8879-46b5-8bcf-7c659debb820" (UID: "d86441ab-8879-46b5-8bcf-7c659debb820"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:40:47.217619 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.217577    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d86441ab-8879-46b5-8bcf-7c659debb820-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d86441ab-8879-46b5-8bcf-7c659debb820" (UID: "d86441ab-8879-46b5-8bcf-7c659debb820"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:40:47.217837 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.217806    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d86441ab-8879-46b5-8bcf-7c659debb820-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d86441ab-8879-46b5-8bcf-7c659debb820" (UID: "d86441ab-8879-46b5-8bcf-7c659debb820"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:40:47.217837 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.217827    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d86441ab-8879-46b5-8bcf-7c659debb820-workload-socket\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:40:47.217954 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.217848    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d86441ab-8879-46b5-8bcf-7c659debb820-workload-certs\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:40:47.217954 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.217863    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d86441ab-8879-46b5-8bcf-7c659debb820-istiod-ca-cert\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:40:47.218512 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.218483    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d86441ab-8879-46b5-8bcf-7c659debb820-istio-data" (OuterVolumeSpecName: "istio-data") pod "d86441ab-8879-46b5-8bcf-7c659debb820" (UID: "d86441ab-8879-46b5-8bcf-7c659debb820"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:40:47.219570 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.219545    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d86441ab-8879-46b5-8bcf-7c659debb820-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d86441ab-8879-46b5-8bcf-7c659debb820" (UID: "d86441ab-8879-46b5-8bcf-7c659debb820"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:40:47.219654 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.219622    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d86441ab-8879-46b5-8bcf-7c659debb820-istio-token" (OuterVolumeSpecName: "istio-token") pod "d86441ab-8879-46b5-8bcf-7c659debb820" (UID: "d86441ab-8879-46b5-8bcf-7c659debb820"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:40:47.219758 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.219738    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d86441ab-8879-46b5-8bcf-7c659debb820-kube-api-access-tppzj" (OuterVolumeSpecName: "kube-api-access-tppzj") pod "d86441ab-8879-46b5-8bcf-7c659debb820" (UID: "d86441ab-8879-46b5-8bcf-7c659debb820"). InnerVolumeSpecName "kube-api-access-tppzj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:40:47.219990 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.219968    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d86441ab-8879-46b5-8bcf-7c659debb820-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d86441ab-8879-46b5-8bcf-7c659debb820" (UID: "d86441ab-8879-46b5-8bcf-7c659debb820"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 04:40:47.322578 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.322550    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d86441ab-8879-46b5-8bcf-7c659debb820-credential-socket\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:40:47.322578 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.322578    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tppzj\" (UniqueName: \"kubernetes.io/projected/d86441ab-8879-46b5-8bcf-7c659debb820-kube-api-access-tppzj\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:40:47.322806 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.322593    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d86441ab-8879-46b5-8bcf-7c659debb820-istio-data\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:40:47.322806 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.322605    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d86441ab-8879-46b5-8bcf-7c659debb820-istio-envoy\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:40:47.322806 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.322615    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d86441ab-8879-46b5-8bcf-7c659debb820-istio-token\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:40:47.322806 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.322625    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d86441ab-8879-46b5-8bcf-7c659debb820-istio-podinfo\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:40:47.605906 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.605870    2576 generic.go:356] "Generic (PLEG): container finished" podID="d86441ab-8879-46b5-8bcf-7c659debb820" containerID="514a3b4486fa05fd7d78667d2eaf3bffdc9aee5e96ce25cac2f2c43be1c77d7f" exitCode=0
Jul 30 04:40:47.606114 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.605953    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"
Jul 30 04:40:47.606114 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.605957    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj" event={"ID":"d86441ab-8879-46b5-8bcf-7c659debb820","Type":"ContainerDied","Data":"514a3b4486fa05fd7d78667d2eaf3bffdc9aee5e96ce25cac2f2c43be1c77d7f"}
Jul 30 04:40:47.606114 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.606042    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj" event={"ID":"d86441ab-8879-46b5-8bcf-7c659debb820","Type":"ContainerDied","Data":"84412ef417f872f2ec7eeb4c62bd948d2df7f54a875c6c60e7ea398dc4c65e40"}
Jul 30 04:40:47.606114 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.606064    2576 scope.go:117] "RemoveContainer" containerID="514a3b4486fa05fd7d78667d2eaf3bffdc9aee5e96ce25cac2f2c43be1c77d7f"
Jul 30 04:40:47.616440 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.616421    2576 scope.go:117] "RemoveContainer" containerID="edb56b72549a837699ad8767f72fecfbf5ddae2eeb32d7c27e60d618ce5c66b0"
Jul 30 04:40:47.624476 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.624461    2576 scope.go:117] "RemoveContainer" containerID="514a3b4486fa05fd7d78667d2eaf3bffdc9aee5e96ce25cac2f2c43be1c77d7f"
Jul 30 04:40:47.624729 ip-10-0-143-117 kubenswrapper[2576]: E0730 04:40:47.624712    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"514a3b4486fa05fd7d78667d2eaf3bffdc9aee5e96ce25cac2f2c43be1c77d7f\": container with ID starting with 514a3b4486fa05fd7d78667d2eaf3bffdc9aee5e96ce25cac2f2c43be1c77d7f not found: ID does not exist" containerID="514a3b4486fa05fd7d78667d2eaf3bffdc9aee5e96ce25cac2f2c43be1c77d7f"
Jul 30 04:40:47.624765 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.624736    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"514a3b4486fa05fd7d78667d2eaf3bffdc9aee5e96ce25cac2f2c43be1c77d7f"} err="failed to get container status \"514a3b4486fa05fd7d78667d2eaf3bffdc9aee5e96ce25cac2f2c43be1c77d7f\": rpc error: code = NotFound desc = could not find container \"514a3b4486fa05fd7d78667d2eaf3bffdc9aee5e96ce25cac2f2c43be1c77d7f\": container with ID starting with 514a3b4486fa05fd7d78667d2eaf3bffdc9aee5e96ce25cac2f2c43be1c77d7f not found: ID does not exist"
Jul 30 04:40:47.624765 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.624753    2576 scope.go:117] "RemoveContainer" containerID="edb56b72549a837699ad8767f72fecfbf5ddae2eeb32d7c27e60d618ce5c66b0"
Jul 30 04:40:47.624992 ip-10-0-143-117 kubenswrapper[2576]: E0730 04:40:47.624973    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edb56b72549a837699ad8767f72fecfbf5ddae2eeb32d7c27e60d618ce5c66b0\": container with ID starting with edb56b72549a837699ad8767f72fecfbf5ddae2eeb32d7c27e60d618ce5c66b0 not found: ID does not exist" containerID="edb56b72549a837699ad8767f72fecfbf5ddae2eeb32d7c27e60d618ce5c66b0"
Jul 30 04:40:47.625044 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.625000    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edb56b72549a837699ad8767f72fecfbf5ddae2eeb32d7c27e60d618ce5c66b0"} err="failed to get container status \"edb56b72549a837699ad8767f72fecfbf5ddae2eeb32d7c27e60d618ce5c66b0\": rpc error: code = NotFound desc = could not find container \"edb56b72549a837699ad8767f72fecfbf5ddae2eeb32d7c27e60d618ce5c66b0\": container with ID starting with edb56b72549a837699ad8767f72fecfbf5ddae2eeb32d7c27e60d618ce5c66b0 not found: ID does not exist"
Jul 30 04:40:47.628626 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.628607    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"]
Jul 30 04:40:47.632623 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:47.632606    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-1d8c4c-openshift-default-78985d864b-5zjvj"]
Jul 30 04:40:49.051151 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:49.051119    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d86441ab-8879-46b5-8bcf-7c659debb820" path="/var/lib/kubelet/pods/d86441ab-8879-46b5-8bcf-7c659debb820/volumes"
Jul 30 04:40:52.782104 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:52.782068    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:40:52.815522 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:52.815492    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-19ef02-77955db588-b2gpj"]
Jul 30 04:40:52.815780 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:52.815752    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-19ef02-77955db588-b2gpj" podUID="b34a72cb-89b7-45d0-91fd-0e6add8db5b6" containerName="payload-pre-processing" containerID="cri-o://7944a37a71d4f12bcf200490c141ca45a1bc926c082b92b1c1fa4e59c5147d56" gracePeriod=30
Jul 30 04:40:52.852021 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:52.851992    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:40:52.949902 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:52.949871    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:40:53.010721 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:53.010670    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:40:53.089724 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:53.089702    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-19ef02-77955db588-b2gpj"
Jul 30 04:40:53.172426 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:53.172396    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jg6dh\" (UniqueName: \"kubernetes.io/projected/b34a72cb-89b7-45d0-91fd-0e6add8db5b6-kube-api-access-jg6dh\") pod \"b34a72cb-89b7-45d0-91fd-0e6add8db5b6\" (UID: \"b34a72cb-89b7-45d0-91fd-0e6add8db5b6\") "
Jul 30 04:40:53.172615 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:53.172474    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b34a72cb-89b7-45d0-91fd-0e6add8db5b6-plugins-config-volume\") pod \"b34a72cb-89b7-45d0-91fd-0e6add8db5b6\" (UID: \"b34a72cb-89b7-45d0-91fd-0e6add8db5b6\") "
Jul 30 04:40:53.172833 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:53.172811    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b34a72cb-89b7-45d0-91fd-0e6add8db5b6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b34a72cb-89b7-45d0-91fd-0e6add8db5b6" (UID: "b34a72cb-89b7-45d0-91fd-0e6add8db5b6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:40:53.174499 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:53.174479    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b34a72cb-89b7-45d0-91fd-0e6add8db5b6-kube-api-access-jg6dh" (OuterVolumeSpecName: "kube-api-access-jg6dh") pod "b34a72cb-89b7-45d0-91fd-0e6add8db5b6" (UID: "b34a72cb-89b7-45d0-91fd-0e6add8db5b6"). InnerVolumeSpecName "kube-api-access-jg6dh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:40:53.273131 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:53.273092    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b34a72cb-89b7-45d0-91fd-0e6add8db5b6-plugins-config-volume\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:40:53.273131 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:53.273125    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jg6dh\" (UniqueName: \"kubernetes.io/projected/b34a72cb-89b7-45d0-91fd-0e6add8db5b6-kube-api-access-jg6dh\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:40:53.301663 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:53.301641    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:40:53.639706 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:53.639673    2576 generic.go:356] "Generic (PLEG): container finished" podID="b34a72cb-89b7-45d0-91fd-0e6add8db5b6" containerID="7944a37a71d4f12bcf200490c141ca45a1bc926c082b92b1c1fa4e59c5147d56" exitCode=0
Jul 30 04:40:53.639894 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:53.639746    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-19ef02-77955db588-b2gpj"
Jul 30 04:40:53.639894 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:53.639752    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-19ef02-77955db588-b2gpj" event={"ID":"b34a72cb-89b7-45d0-91fd-0e6add8db5b6","Type":"ContainerDied","Data":"7944a37a71d4f12bcf200490c141ca45a1bc926c082b92b1c1fa4e59c5147d56"}
Jul 30 04:40:53.639894 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:53.639832    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-19ef02-77955db588-b2gpj" event={"ID":"b34a72cb-89b7-45d0-91fd-0e6add8db5b6","Type":"ContainerDied","Data":"b4d9cb6e96d122adbb8301aad9f0eda066c3cb2aec415e389385a45e1b7fd2a3"}
Jul 30 04:40:53.639894 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:53.639856    2576 scope.go:117] "RemoveContainer" containerID="7944a37a71d4f12bcf200490c141ca45a1bc926c082b92b1c1fa4e59c5147d56"
Jul 30 04:40:53.649565 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:53.649549    2576 scope.go:117] "RemoveContainer" containerID="7944a37a71d4f12bcf200490c141ca45a1bc926c082b92b1c1fa4e59c5147d56"
Jul 30 04:40:53.649834 ip-10-0-143-117 kubenswrapper[2576]: E0730 04:40:53.649814    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7944a37a71d4f12bcf200490c141ca45a1bc926c082b92b1c1fa4e59c5147d56\": container with ID starting with 7944a37a71d4f12bcf200490c141ca45a1bc926c082b92b1c1fa4e59c5147d56 not found: ID does not exist" containerID="7944a37a71d4f12bcf200490c141ca45a1bc926c082b92b1c1fa4e59c5147d56"
Jul 30 04:40:53.649900 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:53.649840    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7944a37a71d4f12bcf200490c141ca45a1bc926c082b92b1c1fa4e59c5147d56"} err="failed to get container status \"7944a37a71d4f12bcf200490c141ca45a1bc926c082b92b1c1fa4e59c5147d56\": rpc error: code = NotFound desc = could not find container \"7944a37a71d4f12bcf200490c141ca45a1bc926c082b92b1c1fa4e59c5147d56\": container with ID starting with 7944a37a71d4f12bcf200490c141ca45a1bc926c082b92b1c1fa4e59c5147d56 not found: ID does not exist"
Jul 30 04:40:53.662039 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:53.662020    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-19ef02-77955db588-b2gpj"]
Jul 30 04:40:53.666022 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:53.666002    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-19ef02-77955db588-b2gpj"]
Jul 30 04:40:55.051867 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:55.051834    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b34a72cb-89b7-45d0-91fd-0e6add8db5b6" path="/var/lib/kubelet/pods/b34a72cb-89b7-45d0-91fd-0e6add8db5b6/volumes"
Jul 30 04:40:55.830771 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:55.830683    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:40:59.195890 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:59.195846    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:40:59.297523 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:59.297491    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:40:59.429566 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:59.429531    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"]
Jul 30 04:40:59.429774 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:59.429745    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh" podUID="a596f883-2431-4345-aaba-d22c0f30ae8a" containerName="istio-proxy" containerID="cri-o://3ba8d8cbb1150f003dc9733ae9757b292863096e0872df578759dec66b89f177" gracePeriod=30
Jul 30 04:40:59.457380 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:59.457317    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:40:59.669656 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:40:59.669620    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:41:04.688937 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:04.688907    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:41:04.704136 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:04.704070    2576 generic.go:356] "Generic (PLEG): container finished" podID="a596f883-2431-4345-aaba-d22c0f30ae8a" containerID="3ba8d8cbb1150f003dc9733ae9757b292863096e0872df578759dec66b89f177" exitCode=0
Jul 30 04:41:04.704136 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:04.704119    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"
Jul 30 04:41:04.704330 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:04.704138    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh" event={"ID":"a596f883-2431-4345-aaba-d22c0f30ae8a","Type":"ContainerDied","Data":"3ba8d8cbb1150f003dc9733ae9757b292863096e0872df578759dec66b89f177"}
Jul 30 04:41:04.704330 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:04.704221    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh" event={"ID":"a596f883-2431-4345-aaba-d22c0f30ae8a","Type":"ContainerDied","Data":"840a7bb685a3049349f0e3a7e6b0f14d7af02dcbb0cdb9daf2f0751f772b1a9c"}
Jul 30 04:41:04.704330 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:04.704252    2576 scope.go:117] "RemoveContainer" containerID="3ba8d8cbb1150f003dc9733ae9757b292863096e0872df578759dec66b89f177"
Jul 30 04:41:04.714808 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:04.714781    2576 scope.go:117] "RemoveContainer" containerID="c880b496ff64ac85b1360e5c42da8d40fe9f6573a3c9316342c628ab44f81eae"
Jul 30 04:41:04.723088 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:04.723072    2576 scope.go:117] "RemoveContainer" containerID="3ba8d8cbb1150f003dc9733ae9757b292863096e0872df578759dec66b89f177"
Jul 30 04:41:04.723325 ip-10-0-143-117 kubenswrapper[2576]: E0730 04:41:04.723308    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ba8d8cbb1150f003dc9733ae9757b292863096e0872df578759dec66b89f177\": container with ID starting with 3ba8d8cbb1150f003dc9733ae9757b292863096e0872df578759dec66b89f177 not found: ID does not exist" containerID="3ba8d8cbb1150f003dc9733ae9757b292863096e0872df578759dec66b89f177"
Jul 30 04:41:04.723381 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:04.723334    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ba8d8cbb1150f003dc9733ae9757b292863096e0872df578759dec66b89f177"} err="failed to get container status \"3ba8d8cbb1150f003dc9733ae9757b292863096e0872df578759dec66b89f177\": rpc error: code = NotFound desc = could not find container \"3ba8d8cbb1150f003dc9733ae9757b292863096e0872df578759dec66b89f177\": container with ID starting with 3ba8d8cbb1150f003dc9733ae9757b292863096e0872df578759dec66b89f177 not found: ID does not exist"
Jul 30 04:41:04.723381 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:04.723349    2576 scope.go:117] "RemoveContainer" containerID="c880b496ff64ac85b1360e5c42da8d40fe9f6573a3c9316342c628ab44f81eae"
Jul 30 04:41:04.723560 ip-10-0-143-117 kubenswrapper[2576]: E0730 04:41:04.723544    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c880b496ff64ac85b1360e5c42da8d40fe9f6573a3c9316342c628ab44f81eae\": container with ID starting with c880b496ff64ac85b1360e5c42da8d40fe9f6573a3c9316342c628ab44f81eae not found: ID does not exist" containerID="c880b496ff64ac85b1360e5c42da8d40fe9f6573a3c9316342c628ab44f81eae"
Jul 30 04:41:04.723609 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:04.723563    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c880b496ff64ac85b1360e5c42da8d40fe9f6573a3c9316342c628ab44f81eae"} err="failed to get container status \"c880b496ff64ac85b1360e5c42da8d40fe9f6573a3c9316342c628ab44f81eae\": rpc error: code = NotFound desc = could not find container \"c880b496ff64ac85b1360e5c42da8d40fe9f6573a3c9316342c628ab44f81eae\": container with ID starting with c880b496ff64ac85b1360e5c42da8d40fe9f6573a3c9316342c628ab44f81eae not found: ID does not exist"
Jul 30 04:41:04.773502 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:04.773470    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a596f883-2431-4345-aaba-d22c0f30ae8a-istio-envoy\") pod \"a596f883-2431-4345-aaba-d22c0f30ae8a\" (UID: \"a596f883-2431-4345-aaba-d22c0f30ae8a\") "
Jul 30 04:41:04.773613 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:04.773512    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a596f883-2431-4345-aaba-d22c0f30ae8a-istio-podinfo\") pod \"a596f883-2431-4345-aaba-d22c0f30ae8a\" (UID: \"a596f883-2431-4345-aaba-d22c0f30ae8a\") "
Jul 30 04:41:04.773613 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:04.773553    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a596f883-2431-4345-aaba-d22c0f30ae8a-istiod-ca-cert\") pod \"a596f883-2431-4345-aaba-d22c0f30ae8a\" (UID: \"a596f883-2431-4345-aaba-d22c0f30ae8a\") "
Jul 30 04:41:04.773694 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:04.773629    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a596f883-2431-4345-aaba-d22c0f30ae8a-istio-token\") pod \"a596f883-2431-4345-aaba-d22c0f30ae8a\" (UID: \"a596f883-2431-4345-aaba-d22c0f30ae8a\") "
Jul 30 04:41:04.773694 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:04.773664    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a596f883-2431-4345-aaba-d22c0f30ae8a-workload-socket\") pod \"a596f883-2431-4345-aaba-d22c0f30ae8a\" (UID: \"a596f883-2431-4345-aaba-d22c0f30ae8a\") "
Jul 30 04:41:04.773694 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:04.773690    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a596f883-2431-4345-aaba-d22c0f30ae8a-workload-certs\") pod \"a596f883-2431-4345-aaba-d22c0f30ae8a\" (UID: \"a596f883-2431-4345-aaba-d22c0f30ae8a\") "
Jul 30 04:41:04.773837 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:04.773727    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a596f883-2431-4345-aaba-d22c0f30ae8a-credential-socket\") pod \"a596f883-2431-4345-aaba-d22c0f30ae8a\" (UID: \"a596f883-2431-4345-aaba-d22c0f30ae8a\") "
Jul 30 04:41:04.773837 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:04.773760    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qft77\" (UniqueName: \"kubernetes.io/projected/a596f883-2431-4345-aaba-d22c0f30ae8a-kube-api-access-qft77\") pod \"a596f883-2431-4345-aaba-d22c0f30ae8a\" (UID: \"a596f883-2431-4345-aaba-d22c0f30ae8a\") "
Jul 30 04:41:04.773837 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:04.773822    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a596f883-2431-4345-aaba-d22c0f30ae8a-istio-data\") pod \"a596f883-2431-4345-aaba-d22c0f30ae8a\" (UID: \"a596f883-2431-4345-aaba-d22c0f30ae8a\") "
Jul 30 04:41:04.774030 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:04.773956    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a596f883-2431-4345-aaba-d22c0f30ae8a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a596f883-2431-4345-aaba-d22c0f30ae8a" (UID: "a596f883-2431-4345-aaba-d22c0f30ae8a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:41:04.774163 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:04.774124    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a596f883-2431-4345-aaba-d22c0f30ae8a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a596f883-2431-4345-aaba-d22c0f30ae8a" (UID: "a596f883-2431-4345-aaba-d22c0f30ae8a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:41:04.774163 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:04.774135    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a596f883-2431-4345-aaba-d22c0f30ae8a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a596f883-2431-4345-aaba-d22c0f30ae8a" (UID: "a596f883-2431-4345-aaba-d22c0f30ae8a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:41:04.774163 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:04.774140    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a596f883-2431-4345-aaba-d22c0f30ae8a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a596f883-2431-4345-aaba-d22c0f30ae8a" (UID: "a596f883-2431-4345-aaba-d22c0f30ae8a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:41:04.774382 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:04.774198    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a596f883-2431-4345-aaba-d22c0f30ae8a-workload-socket\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:41:04.774849 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:04.774830    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a596f883-2431-4345-aaba-d22c0f30ae8a-istio-data" (OuterVolumeSpecName: "istio-data") pod "a596f883-2431-4345-aaba-d22c0f30ae8a" (UID: "a596f883-2431-4345-aaba-d22c0f30ae8a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:41:04.775692 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:04.775674    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a596f883-2431-4345-aaba-d22c0f30ae8a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a596f883-2431-4345-aaba-d22c0f30ae8a" (UID: "a596f883-2431-4345-aaba-d22c0f30ae8a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:41:04.775790 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:04.775772    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a596f883-2431-4345-aaba-d22c0f30ae8a-istio-token" (OuterVolumeSpecName: "istio-token") pod "a596f883-2431-4345-aaba-d22c0f30ae8a" (UID: "a596f883-2431-4345-aaba-d22c0f30ae8a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:41:04.776083 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:04.776062    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a596f883-2431-4345-aaba-d22c0f30ae8a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a596f883-2431-4345-aaba-d22c0f30ae8a" (UID: "a596f883-2431-4345-aaba-d22c0f30ae8a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 04:41:04.776177 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:04.776142    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a596f883-2431-4345-aaba-d22c0f30ae8a-kube-api-access-qft77" (OuterVolumeSpecName: "kube-api-access-qft77") pod "a596f883-2431-4345-aaba-d22c0f30ae8a" (UID: "a596f883-2431-4345-aaba-d22c0f30ae8a"). InnerVolumeSpecName "kube-api-access-qft77". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:41:04.875378 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:04.875346    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a596f883-2431-4345-aaba-d22c0f30ae8a-istio-token\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:41:04.875378 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:04.875372    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a596f883-2431-4345-aaba-d22c0f30ae8a-workload-certs\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:41:04.875378 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:04.875381    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a596f883-2431-4345-aaba-d22c0f30ae8a-credential-socket\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:41:04.875615 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:04.875394    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qft77\" (UniqueName: \"kubernetes.io/projected/a596f883-2431-4345-aaba-d22c0f30ae8a-kube-api-access-qft77\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:41:04.875615 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:04.875402    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a596f883-2431-4345-aaba-d22c0f30ae8a-istio-data\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:41:04.875615 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:04.875409    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a596f883-2431-4345-aaba-d22c0f30ae8a-istio-envoy\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:41:04.875615 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:04.875416    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a596f883-2431-4345-aaba-d22c0f30ae8a-istio-podinfo\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:41:04.875615 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:04.875423    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a596f883-2431-4345-aaba-d22c0f30ae8a-istiod-ca-cert\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:41:05.028881 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:05.028848    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"]
Jul 30 04:41:05.040595 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:05.040568    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-19ef02-openshift-default-c4868c5fb-cgzqh"]
Jul 30 04:41:05.052339 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:05.052312    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a596f883-2431-4345-aaba-d22c0f30ae8a" path="/var/lib/kubelet/pods/a596f883-2431-4345-aaba-d22c0f30ae8a/volumes"
Jul 30 04:41:10.244340 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:10.244307    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:41:10.269717 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:10.269685    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-fcf3e6-596fc7d866-lw8rm"]
Jul 30 04:41:10.269899 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:10.269873    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-fcf3e6-596fc7d866-lw8rm" podUID="aa7e6f7d-c522-48d9-aba9-ccaaad76d08d" containerName="payload-processing" containerID="cri-o://4f3c80cafd1e6983705f52df3ba4ea06c872f87523c2881462f1575ff3295cb0" gracePeriod=30
Jul 30 04:41:10.311567 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:10.311533    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:41:10.414328 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:10.414287    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:41:10.469025 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:10.468988    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:41:10.522767 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:10.522743    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-fcf3e6-596fc7d866-lw8rm"
Jul 30 04:41:10.631842 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:10.631808    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rgsc4\" (UniqueName: \"kubernetes.io/projected/aa7e6f7d-c522-48d9-aba9-ccaaad76d08d-kube-api-access-rgsc4\") pod \"aa7e6f7d-c522-48d9-aba9-ccaaad76d08d\" (UID: \"aa7e6f7d-c522-48d9-aba9-ccaaad76d08d\") "
Jul 30 04:41:10.632029 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:10.631897    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa7e6f7d-c522-48d9-aba9-ccaaad76d08d-plugins-config-volume\") pod \"aa7e6f7d-c522-48d9-aba9-ccaaad76d08d\" (UID: \"aa7e6f7d-c522-48d9-aba9-ccaaad76d08d\") "
Jul 30 04:41:10.632311 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:10.632291    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa7e6f7d-c522-48d9-aba9-ccaaad76d08d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "aa7e6f7d-c522-48d9-aba9-ccaaad76d08d" (UID: "aa7e6f7d-c522-48d9-aba9-ccaaad76d08d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:41:10.633899 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:10.633880    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa7e6f7d-c522-48d9-aba9-ccaaad76d08d-kube-api-access-rgsc4" (OuterVolumeSpecName: "kube-api-access-rgsc4") pod "aa7e6f7d-c522-48d9-aba9-ccaaad76d08d" (UID: "aa7e6f7d-c522-48d9-aba9-ccaaad76d08d"). InnerVolumeSpecName "kube-api-access-rgsc4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:41:10.733580 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:10.733550    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rgsc4\" (UniqueName: \"kubernetes.io/projected/aa7e6f7d-c522-48d9-aba9-ccaaad76d08d-kube-api-access-rgsc4\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:41:10.733580 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:10.733581    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa7e6f7d-c522-48d9-aba9-ccaaad76d08d-plugins-config-volume\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:41:10.744500 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:10.744467    2576 generic.go:356] "Generic (PLEG): container finished" podID="aa7e6f7d-c522-48d9-aba9-ccaaad76d08d" containerID="4f3c80cafd1e6983705f52df3ba4ea06c872f87523c2881462f1575ff3295cb0" exitCode=0
Jul 30 04:41:10.744655 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:10.744523    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-fcf3e6-596fc7d866-lw8rm"
Jul 30 04:41:10.744655 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:10.744555    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-fcf3e6-596fc7d866-lw8rm" event={"ID":"aa7e6f7d-c522-48d9-aba9-ccaaad76d08d","Type":"ContainerDied","Data":"4f3c80cafd1e6983705f52df3ba4ea06c872f87523c2881462f1575ff3295cb0"}
Jul 30 04:41:10.744655 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:10.744638    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-fcf3e6-596fc7d866-lw8rm" event={"ID":"aa7e6f7d-c522-48d9-aba9-ccaaad76d08d","Type":"ContainerDied","Data":"d004e047c6c1a8aff169a82ae27e764950408d75c792094b1698f8ba19dd6f03"}
Jul 30 04:41:10.744784 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:10.744665    2576 scope.go:117] "RemoveContainer" containerID="4f3c80cafd1e6983705f52df3ba4ea06c872f87523c2881462f1575ff3295cb0"
Jul 30 04:41:10.753497 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:10.753472    2576 scope.go:117] "RemoveContainer" containerID="4f3c80cafd1e6983705f52df3ba4ea06c872f87523c2881462f1575ff3295cb0"
Jul 30 04:41:10.753764 ip-10-0-143-117 kubenswrapper[2576]: E0730 04:41:10.753746    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f3c80cafd1e6983705f52df3ba4ea06c872f87523c2881462f1575ff3295cb0\": container with ID starting with 4f3c80cafd1e6983705f52df3ba4ea06c872f87523c2881462f1575ff3295cb0 not found: ID does not exist" containerID="4f3c80cafd1e6983705f52df3ba4ea06c872f87523c2881462f1575ff3295cb0"
Jul 30 04:41:10.753821 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:10.753775    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f3c80cafd1e6983705f52df3ba4ea06c872f87523c2881462f1575ff3295cb0"} err="failed to get container status \"4f3c80cafd1e6983705f52df3ba4ea06c872f87523c2881462f1575ff3295cb0\": rpc error: code = NotFound desc = could not find container \"4f3c80cafd1e6983705f52df3ba4ea06c872f87523c2881462f1575ff3295cb0\": container with ID starting with 4f3c80cafd1e6983705f52df3ba4ea06c872f87523c2881462f1575ff3295cb0 not found: ID does not exist"
Jul 30 04:41:10.768522 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:10.768492    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-fcf3e6-596fc7d866-lw8rm"]
Jul 30 04:41:10.774184 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:10.774158    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-fcf3e6-596fc7d866-lw8rm"]
Jul 30 04:41:10.790188 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:10.790164    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:41:11.051707 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:11.051675    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa7e6f7d-c522-48d9-aba9-ccaaad76d08d" path="/var/lib/kubelet/pods/aa7e6f7d-c522-48d9-aba9-ccaaad76d08d/volumes"
Jul 30 04:41:13.524937 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:13.524886    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2ml29"]
Jul 30 04:41:16.648785 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:16.648725    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:41:16.875313 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:16.875276    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"]
Jul 30 04:41:16.875604 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:16.875575    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd" podUID="c1352281-7174-49b9-b639-60c39cb4ccb0" containerName="istio-proxy" containerID="cri-o://b7291ba5bd852526c8fae3220f127d6476689ebc10ea243986180a1a166ffd80" gracePeriod=30
Jul 30 04:41:17.518818 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:17.518793    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7ddbfffd7d-pgt8d_d4249a5e-2ed9-44a7-ba88-793ebe1306c9/authorino/0.log"
Jul 30 04:41:17.935642 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:17.935608    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:41:22.129344 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.129320    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:41:22.233656 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.233575    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1352281-7174-49b9-b639-60c39cb4ccb0-istiod-ca-cert\") pod \"c1352281-7174-49b9-b639-60c39cb4ccb0\" (UID: \"c1352281-7174-49b9-b639-60c39cb4ccb0\") "
Jul 30 04:41:22.233656 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.233618    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1352281-7174-49b9-b639-60c39cb4ccb0-workload-certs\") pod \"c1352281-7174-49b9-b639-60c39cb4ccb0\" (UID: \"c1352281-7174-49b9-b639-60c39cb4ccb0\") "
Jul 30 04:41:22.233892 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.233663    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1352281-7174-49b9-b639-60c39cb4ccb0-istio-data\") pod \"c1352281-7174-49b9-b639-60c39cb4ccb0\" (UID: \"c1352281-7174-49b9-b639-60c39cb4ccb0\") "
Jul 30 04:41:22.233892 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.233698    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1352281-7174-49b9-b639-60c39cb4ccb0-credential-socket\") pod \"c1352281-7174-49b9-b639-60c39cb4ccb0\" (UID: \"c1352281-7174-49b9-b639-60c39cb4ccb0\") "
Jul 30 04:41:22.233892 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.233720    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1352281-7174-49b9-b639-60c39cb4ccb0-workload-socket\") pod \"c1352281-7174-49b9-b639-60c39cb4ccb0\" (UID: \"c1352281-7174-49b9-b639-60c39cb4ccb0\") "
Jul 30 04:41:22.233892 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.233772    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1352281-7174-49b9-b639-60c39cb4ccb0-istio-podinfo\") pod \"c1352281-7174-49b9-b639-60c39cb4ccb0\" (UID: \"c1352281-7174-49b9-b639-60c39cb4ccb0\") "
Jul 30 04:41:22.233892 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.233814    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1352281-7174-49b9-b639-60c39cb4ccb0-istio-envoy\") pod \"c1352281-7174-49b9-b639-60c39cb4ccb0\" (UID: \"c1352281-7174-49b9-b639-60c39cb4ccb0\") "
Jul 30 04:41:22.233892 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.233849    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7np6\" (UniqueName: \"kubernetes.io/projected/c1352281-7174-49b9-b639-60c39cb4ccb0-kube-api-access-k7np6\") pod \"c1352281-7174-49b9-b639-60c39cb4ccb0\" (UID: \"c1352281-7174-49b9-b639-60c39cb4ccb0\") "
Jul 30 04:41:22.233892 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.233878    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1352281-7174-49b9-b639-60c39cb4ccb0-istio-token\") pod \"c1352281-7174-49b9-b639-60c39cb4ccb0\" (UID: \"c1352281-7174-49b9-b639-60c39cb4ccb0\") "
Jul 30 04:41:22.234276 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.234000    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1352281-7174-49b9-b639-60c39cb4ccb0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c1352281-7174-49b9-b639-60c39cb4ccb0" (UID: "c1352281-7174-49b9-b639-60c39cb4ccb0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:41:22.234276 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.234024    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1352281-7174-49b9-b639-60c39cb4ccb0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c1352281-7174-49b9-b639-60c39cb4ccb0" (UID: "c1352281-7174-49b9-b639-60c39cb4ccb0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 04:41:22.234276 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.234063    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1352281-7174-49b9-b639-60c39cb4ccb0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c1352281-7174-49b9-b639-60c39cb4ccb0" (UID: "c1352281-7174-49b9-b639-60c39cb4ccb0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:41:22.234276 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.234102    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1352281-7174-49b9-b639-60c39cb4ccb0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c1352281-7174-49b9-b639-60c39cb4ccb0" (UID: "c1352281-7174-49b9-b639-60c39cb4ccb0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:41:22.234424 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.234336    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1352281-7174-49b9-b639-60c39cb4ccb0-istiod-ca-cert\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:41:22.234424 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.234357    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1352281-7174-49b9-b639-60c39cb4ccb0-workload-certs\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:41:22.234424 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.234369    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1352281-7174-49b9-b639-60c39cb4ccb0-credential-socket\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:41:22.234424 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.234383    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1352281-7174-49b9-b639-60c39cb4ccb0-workload-socket\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:41:22.234873 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.234850    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1352281-7174-49b9-b639-60c39cb4ccb0-istio-data" (OuterVolumeSpecName: "istio-data") pod "c1352281-7174-49b9-b639-60c39cb4ccb0" (UID: "c1352281-7174-49b9-b639-60c39cb4ccb0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:41:22.236133 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.236107    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1352281-7174-49b9-b639-60c39cb4ccb0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c1352281-7174-49b9-b639-60c39cb4ccb0" (UID: "c1352281-7174-49b9-b639-60c39cb4ccb0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 04:41:22.236246 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.236197    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1352281-7174-49b9-b639-60c39cb4ccb0-kube-api-access-k7np6" (OuterVolumeSpecName: "kube-api-access-k7np6") pod "c1352281-7174-49b9-b639-60c39cb4ccb0" (UID: "c1352281-7174-49b9-b639-60c39cb4ccb0"). InnerVolumeSpecName "kube-api-access-k7np6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:41:22.236433 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.236414    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1352281-7174-49b9-b639-60c39cb4ccb0-istio-token" (OuterVolumeSpecName: "istio-token") pod "c1352281-7174-49b9-b639-60c39cb4ccb0" (UID: "c1352281-7174-49b9-b639-60c39cb4ccb0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 04:41:22.236504 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.236413    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c1352281-7174-49b9-b639-60c39cb4ccb0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c1352281-7174-49b9-b639-60c39cb4ccb0" (UID: "c1352281-7174-49b9-b639-60c39cb4ccb0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 04:41:22.335246 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.335210    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1352281-7174-49b9-b639-60c39cb4ccb0-istio-data\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:41:22.335246 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.335242    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1352281-7174-49b9-b639-60c39cb4ccb0-istio-podinfo\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:41:22.335246 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.335251    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1352281-7174-49b9-b639-60c39cb4ccb0-istio-envoy\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:41:22.335416 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.335259    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7np6\" (UniqueName: \"kubernetes.io/projected/c1352281-7174-49b9-b639-60c39cb4ccb0-kube-api-access-k7np6\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:41:22.335416 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.335269    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1352281-7174-49b9-b639-60c39cb4ccb0-istio-token\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 30 04:41:22.483458 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.483415    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-d42fv_39e8666b-7a30-4644-8979-dc793338ba98/manager/0.log"
Jul 30 04:41:22.812721 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.812687    2576 generic.go:356] "Generic (PLEG): container finished" podID="c1352281-7174-49b9-b639-60c39cb4ccb0" containerID="b7291ba5bd852526c8fae3220f127d6476689ebc10ea243986180a1a166ffd80" exitCode=0
Jul 30 04:41:22.812892 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.812731    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"
Jul 30 04:41:22.812892 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.812771    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd" event={"ID":"c1352281-7174-49b9-b639-60c39cb4ccb0","Type":"ContainerDied","Data":"b7291ba5bd852526c8fae3220f127d6476689ebc10ea243986180a1a166ffd80"}
Jul 30 04:41:22.812892 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.812829    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd" event={"ID":"c1352281-7174-49b9-b639-60c39cb4ccb0","Type":"ContainerDied","Data":"28d270c07361ff16beb3be8c920b95fd8e2452a2ac6cd88d001d910680fc6b0b"}
Jul 30 04:41:22.812892 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.812852    2576 scope.go:117] "RemoveContainer" containerID="b7291ba5bd852526c8fae3220f127d6476689ebc10ea243986180a1a166ffd80"
Jul 30 04:41:22.822694 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.822678    2576 scope.go:117] "RemoveContainer" containerID="47b53757e44918f24fb13a76c2b53a72e9ff26867312aa6d50311e90daf5adde"
Jul 30 04:41:22.830569 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.830552    2576 scope.go:117] "RemoveContainer" containerID="b7291ba5bd852526c8fae3220f127d6476689ebc10ea243986180a1a166ffd80"
Jul 30 04:41:22.830805 ip-10-0-143-117 kubenswrapper[2576]: E0730 04:41:22.830788    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7291ba5bd852526c8fae3220f127d6476689ebc10ea243986180a1a166ffd80\": container with ID starting with b7291ba5bd852526c8fae3220f127d6476689ebc10ea243986180a1a166ffd80 not found: ID does not exist" containerID="b7291ba5bd852526c8fae3220f127d6476689ebc10ea243986180a1a166ffd80"
Jul 30 04:41:22.830848 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.830829    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7291ba5bd852526c8fae3220f127d6476689ebc10ea243986180a1a166ffd80"} err="failed to get container status \"b7291ba5bd852526c8fae3220f127d6476689ebc10ea243986180a1a166ffd80\": rpc error: code = NotFound desc = could not find container \"b7291ba5bd852526c8fae3220f127d6476689ebc10ea243986180a1a166ffd80\": container with ID starting with b7291ba5bd852526c8fae3220f127d6476689ebc10ea243986180a1a166ffd80 not found: ID does not exist"
Jul 30 04:41:22.830890 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.830847    2576 scope.go:117] "RemoveContainer" containerID="47b53757e44918f24fb13a76c2b53a72e9ff26867312aa6d50311e90daf5adde"
Jul 30 04:41:22.831080 ip-10-0-143-117 kubenswrapper[2576]: E0730 04:41:22.831066    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47b53757e44918f24fb13a76c2b53a72e9ff26867312aa6d50311e90daf5adde\": container with ID starting with 47b53757e44918f24fb13a76c2b53a72e9ff26867312aa6d50311e90daf5adde not found: ID does not exist" containerID="47b53757e44918f24fb13a76c2b53a72e9ff26867312aa6d50311e90daf5adde"
Jul 30 04:41:22.831122 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.831085    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47b53757e44918f24fb13a76c2b53a72e9ff26867312aa6d50311e90daf5adde"} err="failed to get container status \"47b53757e44918f24fb13a76c2b53a72e9ff26867312aa6d50311e90daf5adde\": rpc error: code = NotFound desc = could not find container \"47b53757e44918f24fb13a76c2b53a72e9ff26867312aa6d50311e90daf5adde\": container with ID starting with 47b53757e44918f24fb13a76c2b53a72e9ff26867312aa6d50311e90daf5adde not found: ID does not exist"
Jul 30 04:41:22.836545 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.836522    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"]
Jul 30 04:41:22.840162 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:22.840141    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-fcf3e6-openshift-default-76f8865664-lnsjd"]
Jul 30 04:41:23.050779 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:23.050745    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1352281-7174-49b9-b639-60c39cb4ccb0" path="/var/lib/kubelet/pods/c1352281-7174-49b9-b639-60c39cb4ccb0/volumes"
Jul 30 04:41:23.051710 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:23.051694    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5946c6c8fb-2h4sd_9bc8c99b-7801-4b28-a193-95d40194a98b/copy-manifests/0.log"
Jul 30 04:41:23.057392 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:23.057373    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5946c6c8fb-2h4sd_9bc8c99b-7801-4b28-a193-95d40194a98b/manager/0.log"
Jul 30 04:41:24.035602 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:24.035576    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-67vvd_a0530845-4a21-41c2-969e-daa4a0d1feba/postgres/0.log"
Jul 30 04:41:24.790901 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:24.790876    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqfb_0ceed716-436d-4616-a637-524eed35ff96/util/0.log"
Jul 30 04:41:24.796765 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:24.796743    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqfb_0ceed716-436d-4616-a637-524eed35ff96/pull/0.log"
Jul 30 04:41:24.802758 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:24.802743    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqfb_0ceed716-436d-4616-a637-524eed35ff96/extract/0.log"
Jul 30 04:41:24.910001 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:24.909980    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrdnb_f32f58e2-cb21-499f-b72c-4ce90eed5bba/extract/0.log"
Jul 30 04:41:24.915693 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:24.915673    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrdnb_f32f58e2-cb21-499f-b72c-4ce90eed5bba/util/0.log"
Jul 30 04:41:24.921313 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:24.921298    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrdnb_f32f58e2-cb21-499f-b72c-4ce90eed5bba/pull/0.log"
Jul 30 04:41:25.025422 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:25.025400    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r4s7n_9ada5a4e-7a8c-413b-a838-2d98660edf99/util/0.log"
Jul 30 04:41:25.031155 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:25.031135    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r4s7n_9ada5a4e-7a8c-413b-a838-2d98660edf99/pull/0.log"
Jul 30 04:41:25.036740 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:25.036726    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r4s7n_9ada5a4e-7a8c-413b-a838-2d98660edf99/extract/0.log"
Jul 30 04:41:25.135734 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:25.135663    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cst2q_0ceb213f-4e8b-4567-865d-5f370ebc38ea/extract/0.log"
Jul 30 04:41:25.140861 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:25.140842    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cst2q_0ceb213f-4e8b-4567-865d-5f370ebc38ea/util/0.log"
Jul 30 04:41:25.146151 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:25.146122    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cst2q_0ceb213f-4e8b-4567-865d-5f370ebc38ea/pull/0.log"
Jul 30 04:41:25.263108 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:25.263070    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7ddbfffd7d-pgt8d_d4249a5e-2ed9-44a7-ba88-793ebe1306c9/authorino/0.log"
Jul 30 04:41:25.371635 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:25.371614    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-8pv87_2492ddbc-0304-4f0e-a209-3fb9e789899e/manager/0.log"
Jul 30 04:41:25.477467 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:25.477401    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-x5rnh_6e3c0ae4-6549-4214-a3cd-101ec3deaabf/manager/0.log"
Jul 30 04:41:25.581736 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:25.581684    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-vk9f7_6e2fe08c-820d-4da0-91ca-752de17b3bf2/kuadrant-console-plugin/0.log"
Jul 30 04:41:25.691733 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:25.691701    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-txf8l_9d1726a3-a5af-49f0-b216-79753b6dc67b/registry-server/0.log"
Jul 30 04:41:25.914413 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:25.914386    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-2ml29_e2d0eef0-b840-4496-aa36-e6a3fa885d70/limitador/0.log"
Jul 30 04:41:26.354391 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:26.354361    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl_244fd030-1879-462b-8307-7c9de84f4666/istio-proxy/0.log"
Jul 30 04:41:26.682249 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:26.682175    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-fb6bd797-dh9vd_fc92711f-ae9e-46fa-b2e5-49c9f392b846/kube-auth-proxy/0.log"
Jul 30 04:41:26.794578 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:26.794546    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-zvgfm_60518678-0996-4b4f-a74c-1221fecbedcc/istio-proxy/1.log"
Jul 30 04:41:27.014382 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:27.014303    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-8qlzb_9b4ba3f1-07c7-4335-97ad-bf7c9269f7c3/payload-processing/0.log"
Jul 30 04:41:27.441731 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:27.441705    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-qqrhk_abcac6f7-ad4c-4b47-8b18-348d98bc3c87/main/0.log"
Jul 30 04:41:27.651289 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:27.651261    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-qwnzr_58bb4966-2e49-491b-abde-744efcb0d35f/main/0.log"
Jul 30 04:41:27.953517 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:27.953490    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-lw2dk_a5f48781-5841-4e0c-946a-af5beac71c94/main/0.log"
Jul 30 04:41:33.024662 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.024620    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ckfp2/must-gather-f8wjd"]
Jul 30 04:41:33.025184 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.025166    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a596f883-2431-4345-aaba-d22c0f30ae8a" containerName="istio-proxy"
Jul 30 04:41:33.025288 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.025187    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a596f883-2431-4345-aaba-d22c0f30ae8a" containerName="istio-proxy"
Jul 30 04:41:33.025288 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.025199    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1352281-7174-49b9-b639-60c39cb4ccb0" containerName="istio-proxy"
Jul 30 04:41:33.025288 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.025207    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1352281-7174-49b9-b639-60c39cb4ccb0" containerName="istio-proxy"
Jul 30 04:41:33.025288 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.025225    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a596f883-2431-4345-aaba-d22c0f30ae8a" containerName="istio-proxy"
Jul 30 04:41:33.025288 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.025233    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a596f883-2431-4345-aaba-d22c0f30ae8a" containerName="istio-proxy"
Jul 30 04:41:33.025288 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.025247    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1352281-7174-49b9-b639-60c39cb4ccb0" containerName="istio-proxy"
Jul 30 04:41:33.025288 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.025255    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1352281-7174-49b9-b639-60c39cb4ccb0" containerName="istio-proxy"
Jul 30 04:41:33.025288 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.025265    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d86441ab-8879-46b5-8bcf-7c659debb820" containerName="istio-proxy"
Jul 30 04:41:33.025288 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.025273    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d86441ab-8879-46b5-8bcf-7c659debb820" containerName="istio-proxy"
Jul 30 04:41:33.025288 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.025283    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b34a72cb-89b7-45d0-91fd-0e6add8db5b6" containerName="payload-pre-processing"
Jul 30 04:41:33.025288 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.025293    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b34a72cb-89b7-45d0-91fd-0e6add8db5b6" containerName="payload-pre-processing"
Jul 30 04:41:33.025864 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.025301    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="270b4e59-f874-4227-9207-7fee83b78a28" containerName="istio-proxy"
Jul 30 04:41:33.025864 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.025310    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="270b4e59-f874-4227-9207-7fee83b78a28" containerName="istio-proxy"
Jul 30 04:41:33.025864 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.025323    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="270b4e59-f874-4227-9207-7fee83b78a28" containerName="istio-proxy"
Jul 30 04:41:33.025864 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.025332    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="270b4e59-f874-4227-9207-7fee83b78a28" containerName="istio-proxy"
Jul 30 04:41:33.025864 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.025357    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f3527cd-7d6d-4f75-9de1-fe794e198de0" containerName="istio-proxy"
Jul 30 04:41:33.025864 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.025365    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f3527cd-7d6d-4f75-9de1-fe794e198de0" containerName="istio-proxy"
Jul 30 04:41:33.025864 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.025377    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa7e6f7d-c522-48d9-aba9-ccaaad76d08d" containerName="payload-processing"
Jul 30 04:41:33.025864 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.025387    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa7e6f7d-c522-48d9-aba9-ccaaad76d08d" containerName="payload-processing"
Jul 30 04:41:33.025864 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.025398    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f3527cd-7d6d-4f75-9de1-fe794e198de0" containerName="istio-proxy"
Jul 30 04:41:33.025864 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.025406    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f3527cd-7d6d-4f75-9de1-fe794e198de0" containerName="istio-proxy"
Jul 30 04:41:33.025864 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.025503    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b34a72cb-89b7-45d0-91fd-0e6add8db5b6" containerName="payload-pre-processing"
Jul 30 04:41:33.025864 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.025516    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1352281-7174-49b9-b639-60c39cb4ccb0" containerName="istio-proxy"
Jul 30 04:41:33.025864 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.025525    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="270b4e59-f874-4227-9207-7fee83b78a28" containerName="istio-proxy"
Jul 30 04:41:33.025864 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.025534    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f3527cd-7d6d-4f75-9de1-fe794e198de0" containerName="istio-proxy"
Jul 30 04:41:33.025864 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.025544    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d86441ab-8879-46b5-8bcf-7c659debb820" containerName="istio-proxy"
Jul 30 04:41:33.025864 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.025553    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d86441ab-8879-46b5-8bcf-7c659debb820" containerName="istio-proxy"
Jul 30 04:41:33.025864 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.025564    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a596f883-2431-4345-aaba-d22c0f30ae8a" containerName="istio-proxy"
Jul 30 04:41:33.025864 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.025573    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa7e6f7d-c522-48d9-aba9-ccaaad76d08d" containerName="payload-processing"
Jul 30 04:41:33.025864 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.025582    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a596f883-2431-4345-aaba-d22c0f30ae8a" containerName="istio-proxy"
Jul 30 04:41:33.025864 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.025679    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d86441ab-8879-46b5-8bcf-7c659debb820" containerName="istio-proxy"
Jul 30 04:41:33.025864 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.025689    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d86441ab-8879-46b5-8bcf-7c659debb820" containerName="istio-proxy"
Jul 30 04:41:33.025864 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.025765    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1352281-7174-49b9-b639-60c39cb4ccb0" containerName="istio-proxy"
Jul 30 04:41:33.025864 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.025774    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="270b4e59-f874-4227-9207-7fee83b78a28" containerName="istio-proxy"
Jul 30 04:41:33.025864 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.025784    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f3527cd-7d6d-4f75-9de1-fe794e198de0" containerName="istio-proxy"
Jul 30 04:41:33.030340 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.030322    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ckfp2/must-gather-f8wjd"
Jul 30 04:41:33.034205 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.034184    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ckfp2\"/\"kube-root-ca.crt\""
Jul 30 04:41:33.034205 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.034193    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ckfp2\"/\"default-dockercfg-pr8bt\""
Jul 30 04:41:33.034364 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.034241    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ckfp2\"/\"openshift-service-ca.crt\""
Jul 30 04:41:33.039697 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.039677    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ckfp2/must-gather-f8wjd"]
Jul 30 04:41:33.043179 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.043153    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:41:33.119229 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.119196    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/dc1e1d55-ddab-40c9-830d-0a00fed42eb0-must-gather-output\") pod \"must-gather-f8wjd\" (UID: \"dc1e1d55-ddab-40c9-830d-0a00fed42eb0\") " pod="openshift-must-gather-ckfp2/must-gather-f8wjd"
Jul 30 04:41:33.119392 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.119301    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkcw2\" (UniqueName: \"kubernetes.io/projected/dc1e1d55-ddab-40c9-830d-0a00fed42eb0-kube-api-access-bkcw2\") pod \"must-gather-f8wjd\" (UID: \"dc1e1d55-ddab-40c9-830d-0a00fed42eb0\") " pod="openshift-must-gather-ckfp2/must-gather-f8wjd"
Jul 30 04:41:33.141686 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.141665    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:41:33.219861 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.219838    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkcw2\" (UniqueName: \"kubernetes.io/projected/dc1e1d55-ddab-40c9-830d-0a00fed42eb0-kube-api-access-bkcw2\") pod \"must-gather-f8wjd\" (UID: \"dc1e1d55-ddab-40c9-830d-0a00fed42eb0\") " pod="openshift-must-gather-ckfp2/must-gather-f8wjd"
Jul 30 04:41:33.220041 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.219876    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/dc1e1d55-ddab-40c9-830d-0a00fed42eb0-must-gather-output\") pod \"must-gather-f8wjd\" (UID: \"dc1e1d55-ddab-40c9-830d-0a00fed42eb0\") " pod="openshift-must-gather-ckfp2/must-gather-f8wjd"
Jul 30 04:41:33.220167 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.220153    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/dc1e1d55-ddab-40c9-830d-0a00fed42eb0-must-gather-output\") pod \"must-gather-f8wjd\" (UID: \"dc1e1d55-ddab-40c9-830d-0a00fed42eb0\") " pod="openshift-must-gather-ckfp2/must-gather-f8wjd"
Jul 30 04:41:33.227830 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.227810    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkcw2\" (UniqueName: \"kubernetes.io/projected/dc1e1d55-ddab-40c9-830d-0a00fed42eb0-kube-api-access-bkcw2\") pod \"must-gather-f8wjd\" (UID: \"dc1e1d55-ddab-40c9-830d-0a00fed42eb0\") " pod="openshift-must-gather-ckfp2/must-gather-f8wjd"
Jul 30 04:41:33.340528 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.340466    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ckfp2/must-gather-f8wjd"
Jul 30 04:41:33.472982 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.472959    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ckfp2/must-gather-f8wjd"]
Jul 30 04:41:33.474704 ip-10-0-143-117 kubenswrapper[2576]: W0730 04:41:33.474666    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddc1e1d55_ddab_40c9_830d_0a00fed42eb0.slice/crio-9de186726576b6e714ff1de02a21eb57c7fbf56e69df7cf96e6c9b2a22200951 WatchSource:0}: Error finding container 9de186726576b6e714ff1de02a21eb57c7fbf56e69df7cf96e6c9b2a22200951: Status 404 returned error can't find the container with id 9de186726576b6e714ff1de02a21eb57c7fbf56e69df7cf96e6c9b2a22200951
Jul 30 04:41:33.476581 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.476562    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 04:41:33.874583 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:33.874552    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ckfp2/must-gather-f8wjd" event={"ID":"dc1e1d55-ddab-40c9-830d-0a00fed42eb0","Type":"ContainerStarted","Data":"9de186726576b6e714ff1de02a21eb57c7fbf56e69df7cf96e6c9b2a22200951"}
Jul 30 04:41:34.884903 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:34.884858    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ckfp2/must-gather-f8wjd" event={"ID":"dc1e1d55-ddab-40c9-830d-0a00fed42eb0","Type":"ContainerStarted","Data":"de29419253cda4ac210e4e7c02fa069b7fbcfcdd6b0218a4160c24b8ca24e644"}
Jul 30 04:41:34.885384 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:34.884916    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ckfp2/must-gather-f8wjd" event={"ID":"dc1e1d55-ddab-40c9-830d-0a00fed42eb0","Type":"ContainerStarted","Data":"80edfed2c9a10a25128a6f78ae232e9b42adecc66346e4dc258757294ff1c0e4"}
Jul 30 04:41:34.901179 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:34.901127    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ckfp2/must-gather-f8wjd" podStartSLOduration=0.958907603 podStartE2EDuration="1.901108341s" totalImagesPullingTime="942.200738ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 04:41:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:41:34.900961123 +0000 UTC m=+4000.519193358" watchObservedRunningTime="2026-07-30 04:41:34.901108341 +0000 UTC m=+4000.519340582"
Jul 30 04:41:35.852220 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:35.852181    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-pzlrv_1c2622ca-8052-489f-8d23-999618145c5a/global-pull-secret-syncer/0.log"
Jul 30 04:41:35.940939 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:35.940886    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log"
Jul 30 04:41:36.100039 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:36.100008    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-117.ec2.internal_78f929edfbd95057c37ce421449a9742/haproxy/0.log"
Jul 30 04:41:40.110971 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:40.110889    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqfb_0ceed716-436d-4616-a637-524eed35ff96/extract/0.log"
Jul 30 04:41:40.131314 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:40.131285    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqfb_0ceed716-436d-4616-a637-524eed35ff96/util/0.log"
Jul 30 04:41:40.154021 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:40.153972    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqfb_0ceed716-436d-4616-a637-524eed35ff96/pull/0.log"
Jul 30 04:41:40.187946 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:40.187903    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrdnb_f32f58e2-cb21-499f-b72c-4ce90eed5bba/extract/0.log"
Jul 30 04:41:40.208058 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:40.208032    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrdnb_f32f58e2-cb21-499f-b72c-4ce90eed5bba/util/0.log"
Jul 30 04:41:40.232147 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:40.232119    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xrdnb_f32f58e2-cb21-499f-b72c-4ce90eed5bba/pull/0.log"
Jul 30 04:41:40.274811 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:40.274784    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r4s7n_9ada5a4e-7a8c-413b-a838-2d98660edf99/extract/0.log"
Jul 30 04:41:40.296407 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:40.296383    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r4s7n_9ada5a4e-7a8c-413b-a838-2d98660edf99/util/0.log"
Jul 30 04:41:40.314954 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:40.314912    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r4s7n_9ada5a4e-7a8c-413b-a838-2d98660edf99/pull/0.log"
Jul 30 04:41:40.341720 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:40.341695    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cst2q_0ceb213f-4e8b-4567-865d-5f370ebc38ea/extract/0.log"
Jul 30 04:41:40.376616 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:40.376540    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cst2q_0ceb213f-4e8b-4567-865d-5f370ebc38ea/util/0.log"
Jul 30 04:41:40.398563 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:40.398532    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cst2q_0ceb213f-4e8b-4567-865d-5f370ebc38ea/pull/0.log"
Jul 30 04:41:40.871312 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:40.870001    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7ddbfffd7d-pgt8d_d4249a5e-2ed9-44a7-ba88-793ebe1306c9/authorino/0.log"
Jul 30 04:41:40.909538 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:40.908014    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-8pv87_2492ddbc-0304-4f0e-a209-3fb9e789899e/manager/0.log"
Jul 30 04:41:40.936947 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:40.936085    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-x5rnh_6e3c0ae4-6549-4214-a3cd-101ec3deaabf/manager/0.log"
Jul 30 04:41:40.971496 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:40.971464    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-vk9f7_6e2fe08c-820d-4da0-91ca-752de17b3bf2/kuadrant-console-plugin/0.log"
Jul 30 04:41:41.024950 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:41.024218    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-txf8l_9d1726a3-a5af-49f0-b216-79753b6dc67b/registry-server/0.log"
Jul 30 04:41:41.184245 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:41.184167    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-2ml29_e2d0eef0-b840-4496-aa36-e6a3fa885d70/limitador/0.log"
Jul 30 04:41:42.947123 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:42.947076    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_da7311f9-4fbf-44cf-acda-e1cf0159f191/alertmanager/0.log"
Jul 30 04:41:42.992660 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:42.992629    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_da7311f9-4fbf-44cf-acda-e1cf0159f191/config-reloader/0.log"
Jul 30 04:41:43.024966 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:43.024936    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_da7311f9-4fbf-44cf-acda-e1cf0159f191/kube-rbac-proxy-web/0.log"
Jul 30 04:41:43.049971 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:43.049916    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_da7311f9-4fbf-44cf-acda-e1cf0159f191/kube-rbac-proxy/0.log"
Jul 30 04:41:43.073337 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:43.073306    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_da7311f9-4fbf-44cf-acda-e1cf0159f191/kube-rbac-proxy-metric/0.log"
Jul 30 04:41:43.095295 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:43.095270    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_da7311f9-4fbf-44cf-acda-e1cf0159f191/prom-label-proxy/0.log"
Jul 30 04:41:43.118553 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:43.118458    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_da7311f9-4fbf-44cf-acda-e1cf0159f191/init-config-reloader/0.log"
Jul 30 04:41:43.325756 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:43.325696    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mxscm_00de1d13-af77-4a4d-b18a-d94bd36f78b7/node-exporter/0.log"
Jul 30 04:41:43.367141 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:43.367116    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mxscm_00de1d13-af77-4a4d-b18a-d94bd36f78b7/kube-rbac-proxy/0.log"
Jul 30 04:41:43.393313 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:43.393238    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mxscm_00de1d13-af77-4a4d-b18a-d94bd36f78b7/init-textfile/0.log"
Jul 30 04:41:46.002114 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:46.002083    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ckfp2/perf-node-gather-daemonset-lkkw6"]
Jul 30 04:41:46.011196 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:46.011168    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ckfp2/perf-node-gather-daemonset-lkkw6"
Jul 30 04:41:46.016556 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:46.016528    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ckfp2/perf-node-gather-daemonset-lkkw6"]
Jul 30 04:41:46.083778 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:46.083745    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t5kt\" (UniqueName: \"kubernetes.io/projected/2dc23132-9479-4f55-a734-7f4ce9d97a91-kube-api-access-4t5kt\") pod \"perf-node-gather-daemonset-lkkw6\" (UID: \"2dc23132-9479-4f55-a734-7f4ce9d97a91\") " pod="openshift-must-gather-ckfp2/perf-node-gather-daemonset-lkkw6"
Jul 30 04:41:46.083985 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:46.083811    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2dc23132-9479-4f55-a734-7f4ce9d97a91-proc\") pod \"perf-node-gather-daemonset-lkkw6\" (UID: \"2dc23132-9479-4f55-a734-7f4ce9d97a91\") " pod="openshift-must-gather-ckfp2/perf-node-gather-daemonset-lkkw6"
Jul 30 04:41:46.083985 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:46.083869    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2dc23132-9479-4f55-a734-7f4ce9d97a91-podres\") pod \"perf-node-gather-daemonset-lkkw6\" (UID: \"2dc23132-9479-4f55-a734-7f4ce9d97a91\") " pod="openshift-must-gather-ckfp2/perf-node-gather-daemonset-lkkw6"
Jul 30 04:41:46.083985 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:46.083973    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2dc23132-9479-4f55-a734-7f4ce9d97a91-sys\") pod \"perf-node-gather-daemonset-lkkw6\" (UID: \"2dc23132-9479-4f55-a734-7f4ce9d97a91\") " pod="openshift-must-gather-ckfp2/perf-node-gather-daemonset-lkkw6"
Jul 30 04:41:46.084117 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:46.083993    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2dc23132-9479-4f55-a734-7f4ce9d97a91-lib-modules\") pod \"perf-node-gather-daemonset-lkkw6\" (UID: \"2dc23132-9479-4f55-a734-7f4ce9d97a91\") " pod="openshift-must-gather-ckfp2/perf-node-gather-daemonset-lkkw6"
Jul 30 04:41:46.184673 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:46.184631    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2dc23132-9479-4f55-a734-7f4ce9d97a91-sys\") pod \"perf-node-gather-daemonset-lkkw6\" (UID: \"2dc23132-9479-4f55-a734-7f4ce9d97a91\") " pod="openshift-must-gather-ckfp2/perf-node-gather-daemonset-lkkw6"
Jul 30 04:41:46.184853 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:46.184678    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2dc23132-9479-4f55-a734-7f4ce9d97a91-lib-modules\") pod \"perf-node-gather-daemonset-lkkw6\" (UID: \"2dc23132-9479-4f55-a734-7f4ce9d97a91\") " pod="openshift-must-gather-ckfp2/perf-node-gather-daemonset-lkkw6"
Jul 30 04:41:46.184853 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:46.184720    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t5kt\" (UniqueName: \"kubernetes.io/projected/2dc23132-9479-4f55-a734-7f4ce9d97a91-kube-api-access-4t5kt\") pod \"perf-node-gather-daemonset-lkkw6\" (UID: \"2dc23132-9479-4f55-a734-7f4ce9d97a91\") " pod="openshift-must-gather-ckfp2/perf-node-gather-daemonset-lkkw6"
Jul 30 04:41:46.184853 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:46.184770    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2dc23132-9479-4f55-a734-7f4ce9d97a91-proc\") pod \"perf-node-gather-daemonset-lkkw6\" (UID: \"2dc23132-9479-4f55-a734-7f4ce9d97a91\") " pod="openshift-must-gather-ckfp2/perf-node-gather-daemonset-lkkw6"
Jul 30 04:41:46.184853 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:46.184777    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2dc23132-9479-4f55-a734-7f4ce9d97a91-sys\") pod \"perf-node-gather-daemonset-lkkw6\" (UID: \"2dc23132-9479-4f55-a734-7f4ce9d97a91\") " pod="openshift-must-gather-ckfp2/perf-node-gather-daemonset-lkkw6"
Jul 30 04:41:46.184853 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:46.184823    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2dc23132-9479-4f55-a734-7f4ce9d97a91-podres\") pod \"perf-node-gather-daemonset-lkkw6\" (UID: \"2dc23132-9479-4f55-a734-7f4ce9d97a91\") " pod="openshift-must-gather-ckfp2/perf-node-gather-daemonset-lkkw6"
Jul 30 04:41:46.185162 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:46.185135    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2dc23132-9479-4f55-a734-7f4ce9d97a91-lib-modules\") pod \"perf-node-gather-daemonset-lkkw6\" (UID: \"2dc23132-9479-4f55-a734-7f4ce9d97a91\") " pod="openshift-must-gather-ckfp2/perf-node-gather-daemonset-lkkw6"
Jul 30 04:41:46.185354 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:46.185162    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2dc23132-9479-4f55-a734-7f4ce9d97a91-proc\") pod \"perf-node-gather-daemonset-lkkw6\" (UID: \"2dc23132-9479-4f55-a734-7f4ce9d97a91\") " pod="openshift-must-gather-ckfp2/perf-node-gather-daemonset-lkkw6"
Jul 30 04:41:46.185429 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:46.185243    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2dc23132-9479-4f55-a734-7f4ce9d97a91-podres\") pod \"perf-node-gather-daemonset-lkkw6\" (UID: \"2dc23132-9479-4f55-a734-7f4ce9d97a91\") " pod="openshift-must-gather-ckfp2/perf-node-gather-daemonset-lkkw6"
Jul 30 04:41:46.193594 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:46.193562    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t5kt\" (UniqueName: \"kubernetes.io/projected/2dc23132-9479-4f55-a734-7f4ce9d97a91-kube-api-access-4t5kt\") pod \"perf-node-gather-daemonset-lkkw6\" (UID: \"2dc23132-9479-4f55-a734-7f4ce9d97a91\") " pod="openshift-must-gather-ckfp2/perf-node-gather-daemonset-lkkw6"
Jul 30 04:41:46.323228 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:46.323195    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ckfp2/perf-node-gather-daemonset-lkkw6"
Jul 30 04:41:46.360366 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:46.360325    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-594f54c687-9m247_9e2e2686-79f6-4374-8e8b-6374c28d8f82/console/0.log"
Jul 30 04:41:46.503205 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:46.502477    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ckfp2/perf-node-gather-daemonset-lkkw6"]
Jul 30 04:41:46.503800 ip-10-0-143-117 kubenswrapper[2576]: W0730 04:41:46.503595    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2dc23132_9479_4f55_a734_7f4ce9d97a91.slice/crio-0e93d136b51c47feeebb9274878eb09bc2d7e8658279822eb130f9567ac0fbec WatchSource:0}: Error finding container 0e93d136b51c47feeebb9274878eb09bc2d7e8658279822eb130f9567ac0fbec: Status 404 returned error can't find the container with id 0e93d136b51c47feeebb9274878eb09bc2d7e8658279822eb130f9567ac0fbec
Jul 30 04:41:46.983994 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:46.983892    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ckfp2/perf-node-gather-daemonset-lkkw6" event={"ID":"2dc23132-9479-4f55-a734-7f4ce9d97a91","Type":"ContainerStarted","Data":"61a9d4ef0e339068be8e28d18eff1d0010d878cc46cac6e2dfb94c90806c2082"}
Jul 30 04:41:46.984171 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:46.984000    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ckfp2/perf-node-gather-daemonset-lkkw6" event={"ID":"2dc23132-9479-4f55-a734-7f4ce9d97a91","Type":"ContainerStarted","Data":"0e93d136b51c47feeebb9274878eb09bc2d7e8658279822eb130f9567ac0fbec"}
Jul 30 04:41:46.984858 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:46.984836    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-ckfp2/perf-node-gather-daemonset-lkkw6"
Jul 30 04:41:48.166455 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:48.166428    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rg4t8_1f75cd36-d556-4abe-88d6-e83b63fb3799/dns/0.log"
Jul 30 04:41:48.195404 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:48.195349    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rg4t8_1f75cd36-d556-4abe-88d6-e83b63fb3799/kube-rbac-proxy/0.log"
Jul 30 04:41:48.257365 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:48.257341    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mfgtq_49852e08-c471-4251-9b75-5e3e7fcc189b/dns-node-resolver/0.log"
Jul 30 04:41:48.866686 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:48.866661    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tl4qv_b90028ac-a40a-4d9c-816a-8fec579a4b93/node-ca/0.log"
Jul 30 04:41:49.765865 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:49.765831    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl_244fd030-1879-462b-8307-7c9de84f4666/istio-proxy/0.log"
Jul 30 04:41:50.079949 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:50.079907    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-fb6bd797-dh9vd_fc92711f-ae9e-46fa-b2e5-49c9f392b846/kube-auth-proxy/0.log"
Jul 30 04:41:50.106676 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:50.106644    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-zvgfm_60518678-0996-4b4f-a74c-1221fecbedcc/istio-proxy/0.log"
Jul 30 04:41:50.142108 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:50.142080    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-zvgfm_60518678-0996-4b4f-a74c-1221fecbedcc/istio-proxy/1.log"
Jul 30 04:41:50.211771 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:50.211738    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-8qlzb_9b4ba3f1-07c7-4335-97ad-bf7c9269f7c3/payload-processing/0.log"
Jul 30 04:41:51.108133 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:51.108109    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qkjrz_2de4f068-3e70-41d6-8c64-5b6620269083/serve-healthcheck-canary/0.log"
Jul 30 04:41:51.655193 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:51.655166    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wvzzm_30a35c99-75dd-4268-8446-9caec1370557/kube-rbac-proxy/0.log"
Jul 30 04:41:51.676882 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:51.676850    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wvzzm_30a35c99-75dd-4268-8446-9caec1370557/exporter/0.log"
Jul 30 04:41:51.696621 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:51.696575    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wvzzm_30a35c99-75dd-4268-8446-9caec1370557/extractor/0.log"
Jul 30 04:41:53.305208 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:53.305173    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jdph_c50df0e7-6b07-47da-9c3c-d4328dcdcfa5/konnectivity-agent/0.log" limit=1024
Jul 30 04:41:53.818860 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:53.818836    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-d42fv_39e8666b-7a30-4644-8979-dc793338ba98/manager/0.log"
Jul 30 04:41:54.008948 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:54.008909    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-ckfp2/perf-node-gather-daemonset-lkkw6"
Jul 30 04:41:54.032560 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:54.032499    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ckfp2/perf-node-gather-daemonset-lkkw6" podStartSLOduration=9.03247982 podStartE2EDuration="9.03247982s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 04:41:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 04:41:47.026444513 +0000 UTC m=+4012.644676996" watchObservedRunningTime="2026-07-30 04:41:54.03247982 +0000 UTC m=+4019.650712061"
Jul 30 04:41:54.641260 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:54.641230    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5946c6c8fb-2h4sd_9bc8c99b-7801-4b28-a193-95d40194a98b/manager/0.log"
Jul 30 04:41:54.660855 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:54.660831    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5946c6c8fb-2h4sd_9bc8c99b-7801-4b28-a193-95d40194a98b/copy-manifests/0.log"
Jul 30 04:41:56.193043 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:56.193010    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl_244fd030-1879-462b-8307-7c9de84f4666/istio-proxy/0.log"
Jul 30 04:41:56.483122 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:56.483093    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-fb6bd797-dh9vd_fc92711f-ae9e-46fa-b2e5-49c9f392b846/kube-auth-proxy/0.log"
Jul 30 04:41:56.499828 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:56.499791    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-zvgfm_60518678-0996-4b4f-a74c-1221fecbedcc/istio-proxy/0.log"
Jul 30 04:41:56.537080 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:56.537044    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-zvgfm_60518678-0996-4b4f-a74c-1221fecbedcc/istio-proxy/1.log"
Jul 30 04:41:56.585889 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:56.585852    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-8qlzb_9b4ba3f1-07c7-4335-97ad-bf7c9269f7c3/payload-processing/0.log"
Jul 30 04:41:57.909707 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:57.909681    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzcl_244fd030-1879-462b-8307-7c9de84f4666/istio-proxy/0.log"
Jul 30 04:41:58.206014 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:58.205917    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-fb6bd797-dh9vd_fc92711f-ae9e-46fa-b2e5-49c9f392b846/kube-auth-proxy/0.log"
Jul 30 04:41:58.227136 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:58.227107    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-zvgfm_60518678-0996-4b4f-a74c-1221fecbedcc/istio-proxy/0.log"
Jul 30 04:41:58.265320 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:58.265274    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-zvgfm_60518678-0996-4b4f-a74c-1221fecbedcc/istio-proxy/1.log"
Jul 30 04:41:58.322856 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:41:58.322815    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-8qlzb_9b4ba3f1-07c7-4335-97ad-bf7c9269f7c3/payload-processing/0.log"
Jul 30 04:42:02.375588 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:42:02.375562    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7n5z9_04cb8a74-98ed-4fe2-9c4f-189783dce4da/kube-multus-additional-cni-plugins/0.log"
Jul 30 04:42:02.393599 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:42:02.393569    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7n5z9_04cb8a74-98ed-4fe2-9c4f-189783dce4da/egress-router-binary-copy/0.log"
Jul 30 04:42:02.411855 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:42:02.411831    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7n5z9_04cb8a74-98ed-4fe2-9c4f-189783dce4da/cni-plugins/0.log"
Jul 30 04:42:02.430720 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:42:02.430697    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7n5z9_04cb8a74-98ed-4fe2-9c4f-189783dce4da/bond-cni-plugin/0.log"
Jul 30 04:42:02.448580 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:42:02.448557    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7n5z9_04cb8a74-98ed-4fe2-9c4f-189783dce4da/routeoverride-cni/0.log"
Jul 30 04:42:02.466700 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:42:02.466679    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7n5z9_04cb8a74-98ed-4fe2-9c4f-189783dce4da/whereabouts-cni-bincopy/0.log"
Jul 30 04:42:02.490111 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:42:02.490076    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7n5z9_04cb8a74-98ed-4fe2-9c4f-189783dce4da/whereabouts-cni/0.log"
Jul 30 04:42:02.672005 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:42:02.671914    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-th2nr_2e09e50f-f869-4451-9504-a0909a29ae2a/kube-multus/0.log"
Jul 30 04:42:02.800942 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:42:02.800848    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rqrtn_890cb2cc-a9ca-4b13-bef5-f2282e486612/network-metrics-daemon/0.log"
Jul 30 04:42:02.816573 ip-10-0-143-117 kubenswrapper[2576]: I0730 04:42:02.816549    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rqrtn_890cb2cc-a9ca-4b13-bef5-f2282e486612/kube-rbac-proxy/0.log"