Jul 26 18:11:54.166038 ip-10-0-134-7 systemd[1]: Starting Kubernetes Kubelet...
Jul 26 18:11:54.604093 ip-10-0-134-7 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 26 18:11:54.604093 ip-10-0-134-7 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 26 18:11:54.604093 ip-10-0-134-7 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 26 18:11:54.604093 ip-10-0-134-7 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 26 18:11:54.604093 ip-10-0-134-7 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 26 18:11:54.605159 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.605075    2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 26 18:11:54.608714 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608698    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 18:11:54.608714 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608713    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 18:11:54.608799 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608717    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 18:11:54.608799 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608721    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 18:11:54.608799 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608725    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 18:11:54.608799 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608727    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 18:11:54.608799 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608731    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 18:11:54.608799 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608734    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 18:11:54.608799 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608737    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 18:11:54.608799 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608741    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 18:11:54.608799 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608754    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 18:11:54.608799 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608757    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 18:11:54.608799 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608760    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 18:11:54.608799 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608763    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 18:11:54.608799 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608766    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 18:11:54.608799 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608768    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 18:11:54.608799 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608771    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 18:11:54.608799 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608774    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 18:11:54.608799 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608776    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 18:11:54.608799 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608779    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 18:11:54.608799 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608782    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 18:11:54.609244 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608785    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 18:11:54.609244 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608787    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 18:11:54.609244 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608790    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 18:11:54.609244 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608793    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 18:11:54.609244 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608797    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 18:11:54.609244 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608801    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 18:11:54.609244 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608804    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 18:11:54.609244 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608807    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 18:11:54.609244 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608810    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 18:11:54.609244 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608814    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 18:11:54.609244 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608817    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 18:11:54.609244 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608819    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 18:11:54.609244 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608822    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 18:11:54.609244 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608825    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 18:11:54.609244 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608827    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 18:11:54.609244 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608830    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 18:11:54.609244 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608833    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 18:11:54.609244 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608835    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 18:11:54.609244 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608838    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 18:11:54.609714 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608841    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 18:11:54.609714 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608843    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 18:11:54.609714 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608846    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 18:11:54.609714 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608848    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 18:11:54.609714 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608851    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 18:11:54.609714 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608853    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 18:11:54.609714 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608856    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 18:11:54.609714 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608858    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 18:11:54.609714 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608862    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 18:11:54.609714 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608864    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 18:11:54.609714 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608867    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 18:11:54.609714 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608869    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 18:11:54.609714 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608872    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 18:11:54.609714 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608875    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 18:11:54.609714 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608878    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 18:11:54.609714 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608881    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 18:11:54.609714 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608884    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 18:11:54.609714 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608887    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 18:11:54.609714 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608889    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 18:11:54.609714 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608892    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 18:11:54.610246 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608894    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 18:11:54.610246 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608897    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 18:11:54.610246 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608900    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 18:11:54.610246 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608902    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 18:11:54.610246 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608905    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 18:11:54.610246 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608908    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 18:11:54.610246 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608910    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 18:11:54.610246 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608913    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 18:11:54.610246 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608916    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 18:11:54.610246 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608919    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 18:11:54.610246 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608921    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 18:11:54.610246 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608924    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 18:11:54.610246 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608926    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 18:11:54.610246 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608930    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 18:11:54.610246 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608934    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 18:11:54.610246 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608937    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 18:11:54.610246 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608940    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 18:11:54.610246 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608942    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 18:11:54.610246 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608945    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 18:11:54.610246 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608948    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 18:11:54.610810 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608951    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 18:11:54.610810 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608953    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 18:11:54.610810 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608956    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 18:11:54.610810 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608958    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 18:11:54.610810 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608961    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 18:11:54.610810 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608963    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 18:11:54.610810 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608967    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 18:11:54.610810 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608971    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 18:11:54.610810 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608973    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 18:11:54.610810 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608976    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 18:11:54.610810 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608979    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 18:11:54.610810 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608981    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 18:11:54.610810 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608984    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 18:11:54.610810 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608987    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 18:11:54.610810 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608989    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 18:11:54.610810 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608992    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 18:11:54.610810 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.608996    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 18:11:54.610810 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.609000    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 18:11:54.610810 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.609003    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 18:11:54.611418 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611406    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 18:11:54.611418 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611417    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 18:11:54.611468 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611422    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 18:11:54.611468 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611427    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 18:11:54.611468 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611431    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 18:11:54.611468 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611433    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 18:11:54.611468 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611450    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 18:11:54.611468 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611453    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 18:11:54.611468 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611457    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 18:11:54.611468 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611460    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 18:11:54.611468 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611464    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 18:11:54.611468 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611467    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 18:11:54.611468 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611471    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 18:11:54.611760 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611474    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 18:11:54.611760 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611478    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 18:11:54.611760 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611480    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 18:11:54.611760 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611484    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 18:11:54.611760 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611488    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 18:11:54.611760 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611490    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 18:11:54.611760 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611494    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 18:11:54.611760 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611496    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 18:11:54.611760 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611499    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 18:11:54.611760 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611502    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 18:11:54.611760 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611505    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 18:11:54.611760 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611507    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 18:11:54.611760 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611510    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 18:11:54.611760 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611512    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 18:11:54.611760 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611515    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 18:11:54.611760 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611517    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 18:11:54.611760 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611520    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 18:11:54.611760 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611523    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 18:11:54.611760 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611525    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 18:11:54.612219 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611528    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 18:11:54.612219 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611530    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 18:11:54.612219 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611533    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 18:11:54.612219 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611536    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 18:11:54.612219 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611538    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 18:11:54.612219 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611541    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 18:11:54.612219 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611544    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 18:11:54.612219 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611546    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 18:11:54.612219 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611549    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 18:11:54.612219 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611552    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 18:11:54.612219 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611555    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 18:11:54.612219 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611558    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 18:11:54.612219 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611561    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 18:11:54.612219 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611563    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 18:11:54.612219 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611566    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 18:11:54.612219 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611569    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 18:11:54.612219 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611572    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 18:11:54.612219 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611574    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 18:11:54.612219 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611577    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 18:11:54.612219 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611579    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 18:11:54.612719 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611582    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 18:11:54.612719 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611584    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 18:11:54.612719 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611587    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 18:11:54.612719 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611589    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 18:11:54.612719 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611592    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 18:11:54.612719 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611594    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 18:11:54.612719 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611597    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 18:11:54.612719 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611599    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 18:11:54.612719 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611602    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 18:11:54.612719 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611604    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 18:11:54.612719 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611607    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 18:11:54.612719 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611609    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 18:11:54.612719 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611612    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 18:11:54.612719 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611615    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 18:11:54.612719 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611617    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 18:11:54.612719 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611619    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 18:11:54.612719 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611623    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 18:11:54.612719 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611626    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 18:11:54.612719 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611628    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 18:11:54.612719 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611631    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 18:11:54.613232 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611633    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 18:11:54.613232 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611636    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 18:11:54.613232 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611639    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 18:11:54.613232 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611641    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 18:11:54.613232 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611644    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 18:11:54.613232 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611646    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 18:11:54.613232 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611649    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 18:11:54.613232 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611652    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 18:11:54.613232 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611654    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 18:11:54.613232 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611657    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 18:11:54.613232 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611660    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 18:11:54.613232 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611662    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 18:11:54.613232 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611665    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 18:11:54.613232 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611668    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 18:11:54.613232 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611670    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 18:11:54.613232 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611673    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 18:11:54.613232 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611675    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 18:11:54.613232 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611678    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 18:11:54.613232 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611680    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 18:11:54.613232 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611683    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 18:11:54.613726 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611685    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 18:11:54.613726 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611688    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 18:11:54.613726 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611690    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 18:11:54.613726 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611693    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 18:11:54.613726 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611695    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 18:11:54.613726 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611698    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 18:11:54.613726 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.611700    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 18:11:54.613726 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612070    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 18:11:54.613726 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612075    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 18:11:54.613726 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612079    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 18:11:54.613726 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612082    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 18:11:54.613726 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612085    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 18:11:54.613726 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612088    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 18:11:54.613726 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612091    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 18:11:54.613726 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612094    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 18:11:54.613726 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612097    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 18:11:54.613726 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612101    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 18:11:54.613726 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612105    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 18:11:54.613726 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612109    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 18:11:54.613726 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612112    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 18:11:54.614223 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612115    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 18:11:54.614223 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612117    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 18:11:54.614223 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612120    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 18:11:54.614223 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612122    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 18:11:54.614223 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612125    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 18:11:54.614223 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612128    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 18:11:54.614223 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612130    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 18:11:54.614223 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612133    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 18:11:54.614223 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612135    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 18:11:54.614223 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612138    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 18:11:54.614223 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612141    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 18:11:54.614223 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612143    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 18:11:54.614223 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612146    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 18:11:54.614223 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612148    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 18:11:54.614223 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612151    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 18:11:54.614223 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612153    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 18:11:54.614223 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612156    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 18:11:54.614223 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612159    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 18:11:54.614223 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612161    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 18:11:54.614223 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612164    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 18:11:54.614712 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612166    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 18:11:54.614712 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612169    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 18:11:54.614712 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612172    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 18:11:54.614712 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612175    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 18:11:54.614712 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612177    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 18:11:54.614712 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612180    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 18:11:54.614712 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612182    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 18:11:54.614712 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612185    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 18:11:54.614712 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612187    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 18:11:54.614712 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612190    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 18:11:54.614712 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612192    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 18:11:54.614712 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612196    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 18:11:54.614712 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612198    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 18:11:54.614712 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612201    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 18:11:54.614712 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612203    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 18:11:54.614712 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612206    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 18:11:54.614712 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612209    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 18:11:54.614712 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612211    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 18:11:54.614712 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612214    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 18:11:54.614712 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612216    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 18:11:54.615234 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612219    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 18:11:54.615234 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612221    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 18:11:54.615234 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612224    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 18:11:54.615234 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612227    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 18:11:54.615234 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612229    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 18:11:54.615234 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612232    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 18:11:54.615234 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612235    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 18:11:54.615234 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612238    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 18:11:54.615234 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612241    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 18:11:54.615234 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612244    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 18:11:54.615234 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612246    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 18:11:54.615234 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612249    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 18:11:54.615234 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612251    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 18:11:54.615234 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612254    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 18:11:54.615234 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612257    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 18:11:54.615234 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612262    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 18:11:54.615234 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612265    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 18:11:54.615234 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612268    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 18:11:54.615234 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612271    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 18:11:54.615694 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612274    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 18:11:54.615694 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612277    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 18:11:54.615694 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612280    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 18:11:54.615694 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612283    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 18:11:54.615694 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612287    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 18:11:54.615694 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612290    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 18:11:54.615694 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612293    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 18:11:54.615694 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612295    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 18:11:54.615694 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612298    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 18:11:54.615694 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612301    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 18:11:54.615694 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612303    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 18:11:54.615694 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612306    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 18:11:54.615694 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612308    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 18:11:54.615694 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612312    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 18:11:54.615694 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612314    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 18:11:54.615694 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612317    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 18:11:54.615694 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612320    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 18:11:54.615694 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612323    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 18:11:54.615694 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612325    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 18:11:54.615694 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612328    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 18:11:54.616188 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612330    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 18:11:54.616188 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612333    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 18:11:54.616188 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612336    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 18:11:54.616188 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612338    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 18:11:54.616188 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612341    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 18:11:54.616188 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612343    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 18:11:54.616188 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.612347    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 18:11:54.616188 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612412    2576 flags.go:64] FLAG: --address="0.0.0.0"
Jul 26 18:11:54.616188 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612423    2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 26 18:11:54.616188 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612430    2576 flags.go:64] FLAG: --anonymous-auth="true"
Jul 26 18:11:54.616188 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612434    2576 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 26 18:11:54.616188 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612439    2576 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 26 18:11:54.616188 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612442    2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 26 18:11:54.616188 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612446    2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 26 18:11:54.616188 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612450    2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 26 18:11:54.616188 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612453    2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 26 18:11:54.616188 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612456    2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 26 18:11:54.616188 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612460    2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 26 18:11:54.616188 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612463    2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 26 18:11:54.616188 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612466    2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 26 18:11:54.616188 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612469    2576 flags.go:64] FLAG: --cgroup-root=""
Jul 26 18:11:54.616188 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612472    2576 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 26 18:11:54.616730 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612475    2576 flags.go:64] FLAG: --client-ca-file=""
Jul 26 18:11:54.616730 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612478    2576 flags.go:64] FLAG: --cloud-provider="external"
Jul 26 18:11:54.616730 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612481    2576 flags.go:64] FLAG: --cluster-dns="[]"
Jul 26 18:11:54.616730 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612485    2576 flags.go:64] FLAG: --cluster-domain=""
Jul 26 18:11:54.616730 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612488    2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 26 18:11:54.616730 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612492    2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 26 18:11:54.616730 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612495    2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 26 18:11:54.616730 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612498    2576 flags.go:64] FLAG: --container-log-max-files="5"
Jul 26 18:11:54.616730 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612502    2576 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 26 18:11:54.616730 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612505    2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 26 18:11:54.616730 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612508    2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 26 18:11:54.616730 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612511    2576 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 26 18:11:54.616730 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612515    2576 flags.go:64] FLAG: --contention-profiling="false"
Jul 26 18:11:54.616730 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612518    2576 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 26 18:11:54.616730 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612521    2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 26 18:11:54.616730 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612524    2576 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 26 18:11:54.616730 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612528    2576 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 26 18:11:54.616730 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612532    2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 26 18:11:54.616730 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612536    2576 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 26 18:11:54.616730 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612539    2576 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 26 18:11:54.616730 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612543    2576 flags.go:64] FLAG: --enable-load-reader="false"
Jul 26 18:11:54.616730 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612546    2576 flags.go:64] FLAG: --enable-server="true"
Jul 26 18:11:54.616730 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612549    2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 26 18:11:54.616730 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612554    2576 flags.go:64] FLAG: --event-burst="100"
Jul 26 18:11:54.616730 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612557    2576 flags.go:64] FLAG: --event-qps="50"
Jul 26 18:11:54.617360 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612560    2576 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 26 18:11:54.617360 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612562    2576 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 26 18:11:54.617360 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612565    2576 flags.go:64] FLAG: --eviction-hard=""
Jul 26 18:11:54.617360 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612569    2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 26 18:11:54.617360 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612572    2576 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 26 18:11:54.617360 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612575    2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 26 18:11:54.617360 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612578    2576 flags.go:64] FLAG: --eviction-soft=""
Jul 26 18:11:54.617360 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612581    2576 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 26 18:11:54.617360 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612586    2576 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 26 18:11:54.617360 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612589    2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 26 18:11:54.617360 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612592    2576 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 26 18:11:54.617360 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612595    2576 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 26 18:11:54.617360 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612598    2576 flags.go:64] FLAG: --fail-swap-on="true"
Jul 26 18:11:54.617360 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612601    2576 flags.go:64] FLAG: --feature-gates=""
Jul 26 18:11:54.617360 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612605    2576 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 26 18:11:54.617360 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612608    2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 26 18:11:54.617360 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612611    2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 26 18:11:54.617360 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612614    2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 26 18:11:54.617360 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612617    2576 flags.go:64] FLAG: --healthz-port="10248"
Jul 26 18:11:54.617360 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612620    2576 flags.go:64] FLAG: --help="false"
Jul 26 18:11:54.617360 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612623    2576 flags.go:64] FLAG: --hostname-override="ip-10-0-134-7.ec2.internal"
Jul 26 18:11:54.617360 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612626    2576 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 26 18:11:54.617360 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612629    2576 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 26 18:11:54.617360 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612635    2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 26 18:11:54.617984 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612638    2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 26 18:11:54.617984 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612641    2576 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 26 18:11:54.617984 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612644    2576 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 26 18:11:54.617984 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612647    2576 flags.go:64] FLAG: --image-service-endpoint=""
Jul 26 18:11:54.617984 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612650    2576 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 26 18:11:54.617984 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612653    2576 flags.go:64] FLAG: --kube-api-burst="100"
Jul 26 18:11:54.617984 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612656    2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 26 18:11:54.617984 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612660    2576 flags.go:64] FLAG: --kube-api-qps="50"
Jul 26 18:11:54.617984 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612662    2576 flags.go:64] FLAG: --kube-reserved=""
Jul 26 18:11:54.617984 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612665    2576 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 26 18:11:54.617984 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612668    2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 26 18:11:54.617984 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612671    2576 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 26 18:11:54.617984 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612674    2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 26 18:11:54.617984 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612677    2576 flags.go:64] FLAG: --lock-file=""
Jul 26 18:11:54.617984 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612680    2576 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 26 18:11:54.617984 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612683    2576 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 26 18:11:54.617984 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612687    2576 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 26 18:11:54.617984 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612692    2576 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 26 18:11:54.617984 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612695    2576 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 26 18:11:54.617984 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612698    2576 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 26 18:11:54.617984 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612701    2576 flags.go:64] FLAG: --logging-format="text"
Jul 26 18:11:54.617984 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612704    2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 26 18:11:54.617984 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612707    2576 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 26 18:11:54.617984 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612710    2576 flags.go:64] FLAG: --manifest-url=""
Jul 26 18:11:54.618567 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612713    2576 flags.go:64] FLAG: --manifest-url-header=""
Jul 26 18:11:54.618567 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612717    2576 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 26 18:11:54.618567 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612720    2576 flags.go:64] FLAG: --max-open-files="1000000"
Jul 26 18:11:54.618567 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612725    2576 flags.go:64] FLAG: --max-pods="110"
Jul 26 18:11:54.618567 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612728    2576 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 26 18:11:54.618567 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612731    2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 26 18:11:54.618567 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612734    2576 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 26 18:11:54.618567 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612738    2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 26 18:11:54.618567 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612741    2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 26 18:11:54.618567 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612756    2576 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 26 18:11:54.618567 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612760    2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 26 18:11:54.618567 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612766    2576 flags.go:64] FLAG: --node-status-max-images="50"
Jul 26 18:11:54.618567 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612769    2576 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 26 18:11:54.618567 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612773    2576 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 26 18:11:54.618567 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612776    2576 flags.go:64] FLAG: --pod-cidr=""
Jul 26 18:11:54.618567 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612779    2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 26 18:11:54.618567 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612784    2576 flags.go:64] FLAG: --pod-manifest-path=""
Jul 26 18:11:54.618567 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612787    2576 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 26 18:11:54.618567 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612790    2576 flags.go:64] FLAG: --pods-per-core="0"
Jul 26 18:11:54.618567 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612793    2576 flags.go:64] FLAG: --port="10250"
Jul 26 18:11:54.618567 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612796    2576 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 26 18:11:54.618567 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612799    2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-01d1f3ef7cce0510b"
Jul 26 18:11:54.618567 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612802    2576 flags.go:64] FLAG: --qos-reserved=""
Jul 26 18:11:54.618567 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612805    2576 flags.go:64] FLAG: --read-only-port="10255"
Jul 26 18:11:54.619218 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612810    2576 flags.go:64] FLAG: --register-node="true"
Jul 26 18:11:54.619218 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612813    2576 flags.go:64] FLAG: --register-with-taints=""
Jul 26 18:11:54.619218 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612816    2576 flags.go:64] FLAG: --registry-burst="10"
Jul 26 18:11:54.619218 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612819    2576 flags.go:64] FLAG: --registry-qps="5"
Jul 26 18:11:54.619218 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612822    2576 flags.go:64] FLAG: --reserved-cpus=""
Jul 26 18:11:54.619218 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612825    2576 flags.go:64] FLAG: --reserved-memory=""
Jul 26 18:11:54.619218 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612828    2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 26 18:11:54.619218 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612831    2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 26 18:11:54.619218 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612834    2576 flags.go:64] FLAG: --rotate-certificates="false"
Jul 26 18:11:54.619218 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612837    2576 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 26 18:11:54.619218 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612840    2576 flags.go:64] FLAG: --runonce="false"
Jul 26 18:11:54.619218 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612843    2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 26 18:11:54.619218 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612846    2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 26 18:11:54.619218 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612849    2576 flags.go:64] FLAG: --seccomp-default="false"
Jul 26 18:11:54.619218 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612852    2576 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 26 18:11:54.619218 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612854    2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 26 18:11:54.619218 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612859    2576 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 26 18:11:54.619218 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612862    2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 26 18:11:54.619218 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612865    2576 flags.go:64] FLAG: --storage-driver-password="root"
Jul 26 18:11:54.619218 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612868    2576 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 26 18:11:54.619218 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612871    2576 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 26 18:11:54.619218 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612874    2576 flags.go:64] FLAG: --storage-driver-user="root"
Jul 26 18:11:54.619218 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612877    2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 26 18:11:54.619218 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612880    2576 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 26 18:11:54.619218 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612884    2576 flags.go:64] FLAG: --system-cgroups=""
Jul 26 18:11:54.619870 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612886    2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 26 18:11:54.619870 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612893    2576 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 26 18:11:54.619870 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612895    2576 flags.go:64] FLAG: --tls-cert-file=""
Jul 26 18:11:54.619870 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612898    2576 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 26 18:11:54.619870 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612903    2576 flags.go:64] FLAG: --tls-min-version=""
Jul 26 18:11:54.619870 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612905    2576 flags.go:64] FLAG: --tls-private-key-file=""
Jul 26 18:11:54.619870 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612908    2576 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 26 18:11:54.619870 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612913    2576 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 26 18:11:54.619870 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612916    2576 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 26 18:11:54.619870 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612919    2576 flags.go:64] FLAG: --v="2"
Jul 26 18:11:54.619870 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612923    2576 flags.go:64] FLAG: --version="false"
Jul 26 18:11:54.619870 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612927    2576 flags.go:64] FLAG: --vmodule=""
Jul 26 18:11:54.619870 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612931    2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 26 18:11:54.619870 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.612934    2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 26 18:11:54.619870 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613045    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 18:11:54.619870 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613049    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 18:11:54.619870 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613052    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 18:11:54.619870 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613055    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 18:11:54.619870 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613058    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 18:11:54.619870 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613060    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 18:11:54.619870 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613063    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 18:11:54.619870 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613066    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 18:11:54.619870 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613069    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 18:11:54.620450 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613072    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 18:11:54.620450 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613075    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 18:11:54.620450 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613078    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 18:11:54.620450 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613080    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 18:11:54.620450 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613083    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 18:11:54.620450 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613085    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 18:11:54.620450 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613088    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 18:11:54.620450 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613093    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 18:11:54.620450 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613095    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 18:11:54.620450 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613098    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 18:11:54.620450 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613101    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 18:11:54.620450 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613103    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 18:11:54.620450 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613106    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 18:11:54.620450 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613109    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 18:11:54.620450 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613111    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 18:11:54.620450 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613114    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 18:11:54.620450 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613118    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 18:11:54.620450 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613121    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 18:11:54.620450 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613123    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 18:11:54.620450 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613126    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 18:11:54.620953 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613129    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 18:11:54.620953 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613131    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 18:11:54.620953 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613134    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 18:11:54.620953 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613136    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 18:11:54.620953 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613139    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 18:11:54.620953 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613142    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 18:11:54.620953 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613144    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 18:11:54.620953 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613147    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 18:11:54.620953 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613149    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 18:11:54.620953 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613152    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 18:11:54.620953 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613155    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 18:11:54.620953 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613157    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 18:11:54.620953 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613161    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 18:11:54.620953 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613164    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 18:11:54.620953 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613166    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 18:11:54.620953 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613169    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 18:11:54.620953 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613172    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 18:11:54.620953 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613175    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 18:11:54.620953 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613177    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 18:11:54.620953 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613180    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 18:11:54.621449 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613183    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 18:11:54.621449 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613186    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 18:11:54.621449 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613188    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 18:11:54.621449 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613190    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 18:11:54.621449 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613193    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 18:11:54.621449 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613196    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 18:11:54.621449 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613198    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 18:11:54.621449 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613201    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 18:11:54.621449 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613204    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 18:11:54.621449 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613207    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 18:11:54.621449 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613209    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 18:11:54.621449 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613212    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 18:11:54.621449 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613215    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 18:11:54.621449 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613219    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 18:11:54.621449 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613223    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 18:11:54.621449 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613227    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 18:11:54.621449 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613230    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 18:11:54.621449 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613233    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 18:11:54.621449 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613236    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 18:11:54.621962 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613239    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 18:11:54.621962 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613242    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 18:11:54.621962 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613244    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 18:11:54.621962 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613248    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 18:11:54.621962 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613251    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 18:11:54.621962 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613255    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 18:11:54.621962 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613258    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 18:11:54.621962 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613261    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 18:11:54.621962 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613264    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 18:11:54.621962 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613266    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 18:11:54.621962 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613269    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 18:11:54.621962 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613271    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 18:11:54.621962 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613274    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 18:11:54.621962 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613277    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 18:11:54.621962 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613280    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 18:11:54.621962 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613282    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 18:11:54.621962 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613285    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 18:11:54.621962 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613287    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 18:11:54.621962 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613290    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 18:11:54.621962 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613292    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 18:11:54.622728 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613295    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 18:11:54.622728 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613298    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 18:11:54.622728 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613308    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 18:11:54.622728 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613311    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 18:11:54.622728 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613313    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 18:11:54.622728 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613316    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 18:11:54.622728 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613318    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 18:11:54.622728 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613321    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 18:11:54.622728 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613324    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 18:11:54.622728 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613326    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 18:11:54.622728 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.613329    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 18:11:54.622728 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.613983    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 26 18:11:54.622728 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.620901    2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 26 18:11:54.622728 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.620915    2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 26 18:11:54.622728 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.621717    2576 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 26 18:11:54.623386 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.621723    2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 26 18:11:54.623386 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622036    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 18:11:54.623386 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622045    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 18:11:54.623386 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622051    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 18:11:54.623386 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622056    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 18:11:54.623386 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622062    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 18:11:54.623386 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622072    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 18:11:54.623386 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622076    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 18:11:54.623386 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622081    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 18:11:54.623386 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622085    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 18:11:54.623386 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622089    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 18:11:54.623386 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622093    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 18:11:54.623386 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622097    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 18:11:54.623386 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622102    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 18:11:54.623386 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622106    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 18:11:54.623386 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622110    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 18:11:54.623386 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622114    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 18:11:54.623386 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622120    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 18:11:54.623386 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622124    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 18:11:54.624086 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622133    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 18:11:54.624086 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622137    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 18:11:54.624086 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622141    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 18:11:54.624086 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622146    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 18:11:54.624086 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622149    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 18:11:54.624086 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622154    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 18:11:54.624086 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622158    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 18:11:54.624086 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622162    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 18:11:54.624086 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622166    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 18:11:54.624086 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622170    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 18:11:54.624086 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622175    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 18:11:54.624086 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622181    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 18:11:54.624086 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622186    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 18:11:54.624086 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622195    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 18:11:54.624086 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622199    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 18:11:54.624086 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622207    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 18:11:54.624086 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622214    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 18:11:54.624086 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622218    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 18:11:54.624086 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622224    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 18:11:54.624086 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622229    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 18:11:54.624648 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622233    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 18:11:54.624648 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622239    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 18:11:54.624648 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622243    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 18:11:54.624648 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622248    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 18:11:54.624648 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622254    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 18:11:54.624648 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622259    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 18:11:54.624648 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622270    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 18:11:54.624648 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622274    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 18:11:54.624648 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622279    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 18:11:54.624648 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622283    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 18:11:54.624648 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622287    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 18:11:54.624648 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622292    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 18:11:54.624648 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622296    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 18:11:54.624648 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622301    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 18:11:54.624648 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622305    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 18:11:54.624648 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622310    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 18:11:54.624648 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622314    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 18:11:54.624648 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622318    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 18:11:54.624648 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622327    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 18:11:54.624648 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622331    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 18:11:54.625164 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622335    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 18:11:54.625164 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622340    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 18:11:54.625164 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622344    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 18:11:54.625164 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622350    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 18:11:54.625164 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622355    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 18:11:54.625164 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622360    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 18:11:54.625164 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622364    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 18:11:54.625164 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622369    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 18:11:54.625164 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622373    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 18:11:54.625164 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622377    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 18:11:54.625164 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622390    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 18:11:54.625164 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622396    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 18:11:54.625164 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622401    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 18:11:54.625164 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622406    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 18:11:54.625164 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622411    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 18:11:54.625164 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622415    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 18:11:54.625164 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622420    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 18:11:54.625164 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622425    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 18:11:54.625164 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622429    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 18:11:54.625164 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622433    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 18:11:54.625879 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622438    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 18:11:54.625879 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622444    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 18:11:54.625879 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622456    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 18:11:54.625879 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622462    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 18:11:54.625879 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622467    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 18:11:54.625879 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622472    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 18:11:54.625879 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622477    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 18:11:54.625879 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622482    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 18:11:54.625879 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622486    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 18:11:54.625879 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622491    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 18:11:54.625879 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622495    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 18:11:54.625879 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622500    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 18:11:54.625879 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622505    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 18:11:54.625879 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622510    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 18:11:54.625879 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622520    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 18:11:54.625879 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622526    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 18:11:54.625879 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622530    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 18:11:54.625879 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622536    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 18:11:54.625879 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622540    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 18:11:54.626624 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622545    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 18:11:54.626624 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622549    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 18:11:54.626624 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.622558    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 26 18:11:54.626624 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622963    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 18:11:54.626624 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.622974    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 18:11:54.626624 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623142    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 18:11:54.626624 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623155    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 18:11:54.626624 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623162    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 18:11:54.626624 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623167    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 18:11:54.626624 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623172    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 18:11:54.626624 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623176    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 18:11:54.626624 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623181    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 18:11:54.626624 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623185    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 18:11:54.626624 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623190    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 18:11:54.626624 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623194    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 18:11:54.627129 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623200    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 18:11:54.627129 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623205    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 18:11:54.627129 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623209    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 18:11:54.627129 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623213    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 18:11:54.627129 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623219    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 18:11:54.627129 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623224    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 18:11:54.627129 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623232    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 18:11:54.627129 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623241    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 18:11:54.627129 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623249    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 18:11:54.627129 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623253    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 18:11:54.627129 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623258    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 18:11:54.627129 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623263    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 18:11:54.627129 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623268    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 18:11:54.627129 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623272    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 18:11:54.627129 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623277    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 18:11:54.627129 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623282    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 18:11:54.627129 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623288    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 18:11:54.627129 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623293    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 18:11:54.627129 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623298    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 18:11:54.627758 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623302    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 18:11:54.627758 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623307    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 18:11:54.627758 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623312    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 18:11:54.627758 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623317    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 18:11:54.627758 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623321    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 18:11:54.627758 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623326    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 18:11:54.627758 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623330    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 18:11:54.627758 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623335    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 18:11:54.627758 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623339    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 18:11:54.627758 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623343    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 18:11:54.627758 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623348    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 18:11:54.627758 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623352    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 18:11:54.627758 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623357    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 18:11:54.627758 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623361    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 18:11:54.627758 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623365    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 18:11:54.627758 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623370    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 18:11:54.627758 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623374    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 18:11:54.627758 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623379    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 18:11:54.627758 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623383    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 18:11:54.627758 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623387    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 18:11:54.628259 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623392    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 18:11:54.628259 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623396    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 18:11:54.628259 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623400    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 18:11:54.628259 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623404    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 18:11:54.628259 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623409    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 18:11:54.628259 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623413    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 18:11:54.628259 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623418    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 18:11:54.628259 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623422    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 18:11:54.628259 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623426    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 18:11:54.628259 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623431    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 18:11:54.628259 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623435    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 18:11:54.628259 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623440    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 18:11:54.628259 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623444    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 18:11:54.628259 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623448    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 18:11:54.628259 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623453    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 18:11:54.628259 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623458    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 18:11:54.628259 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623462    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 18:11:54.628259 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623467    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 18:11:54.628259 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623471    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 18:11:54.628259 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623476    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 18:11:54.628865 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623480    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 18:11:54.628865 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623487    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 18:11:54.628865 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623492    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 18:11:54.628865 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623496    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 18:11:54.628865 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623501    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 18:11:54.628865 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623506    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 18:11:54.628865 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623510    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 18:11:54.628865 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623515    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 18:11:54.628865 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623520    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 18:11:54.628865 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623524    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 18:11:54.628865 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623528    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 18:11:54.628865 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623533    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 18:11:54.628865 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623538    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 18:11:54.628865 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623543    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 18:11:54.628865 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623547    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 18:11:54.628865 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623551    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 18:11:54.628865 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623555    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 18:11:54.628865 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623559    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 18:11:54.628865 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623564    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 18:11:54.629341 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623568    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 18:11:54.629341 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623572    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 18:11:54.629341 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623577    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 18:11:54.629341 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623581    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 18:11:54.629341 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623585    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 18:11:54.629341 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623590    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 18:11:54.629341 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623594    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 18:11:54.629341 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623598    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 18:11:54.629341 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:54.623603    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 18:11:54.629341 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.623611    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 26 18:11:54.629341 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.623791    2576 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 26 18:11:54.629341 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.628644    2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 26 18:11:54.630453 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.630439    2576 server.go:1019] "Starting client certificate rotation"
Jul 26 18:11:54.630565 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.630547    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 26 18:11:54.630598 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.630580    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 26 18:11:54.633060 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.633043    2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 26 18:11:54.634290 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.634264    2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 26 18:11:54.645390 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.645371    2576 log.go:25] "Validated CRI v1 runtime API"
Jul 26 18:11:54.649248 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.649232    2576 log.go:25] "Validated CRI v1 image API"
Jul 26 18:11:54.650086 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.650071    2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 26 18:11:54.654762 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.654726    2576 fs.go:135] Filesystem UUIDs: map[5b0f1fa1-f3e4-4e87-90d1-5beec4679b0d:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a116069c-c267-4618-89f3-7dff93943bbc:/dev/nvme0n1p4]
Jul 26 18:11:54.654837 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.654762    2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 26 18:11:54.660000 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.659888    2576 manager.go:216] Machine: {Timestamp:2026-07-26 18:11:54.658426023 +0000 UTC m=+0.365833073 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100799 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22e792f450a242ce6347077391f3ff SystemUUID:ec22e792-f450-a242-ce63-47077391f3ff BootID:c8c8c663-05b2-43f0-92db-6cc0ec0db56f Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d3:c4:d2:0e:b9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d3:c4:d2:0e:b9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:72:da:db:24:73:dc Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 26 18:11:54.660000 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.659989    2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 26 18:11:54.660159 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.660139    2576 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 26 18:11:54.663572 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.663544    2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 26 18:11:54.663704 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.663576    2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-7.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 26 18:11:54.664465 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.664454    2576 topology_manager.go:138] "Creating topology manager with none policy"
Jul 26 18:11:54.664498 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.664466    2576 container_manager_linux.go:309] "Creating device plugin manager"
Jul 26 18:11:54.664498 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.664474    2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 26 18:11:54.665285 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.665275    2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 26 18:11:54.666545 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.666536    2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 26 18:11:54.668363 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.668352    2576 state_mem.go:36] "Initialized new in-memory state store"
Jul 26 18:11:54.668470 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.668461    2576 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 26 18:11:54.669193 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.669184    2576 kubelet.go:486] "Attempting to sync node with API server"
Jul 26 18:11:54.669229 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.669196    2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 26 18:11:54.669229 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.669212    2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 26 18:11:54.669229 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.669221    2576 kubelet.go:398] "Adding apiserver pod source"
Jul 26 18:11:54.669324 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.669231    2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 26 18:11:54.669594 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.669582    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 26 18:11:54.669634 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.669601    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 26 18:11:54.673582 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.673397    2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 26 18:11:54.675850 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.675834    2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 26 18:11:54.675913 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.675865    2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 26 18:11:54.677942 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.677929    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 26 18:11:54.677997 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.677945    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 26 18:11:54.677997 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.677951    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 26 18:11:54.677997 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.677957    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 26 18:11:54.677997 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.677965    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 26 18:11:54.677997 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.677971    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 26 18:11:54.677997 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.677979    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 26 18:11:54.677997 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.677985    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 26 18:11:54.677997 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.677991    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 26 18:11:54.677997 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.677996    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 26 18:11:54.678241 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.678004    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 26 18:11:54.678241 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.678009    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 26 18:11:54.678945 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.678935    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 26 18:11:54.678945 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.678945    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 26 18:11:54.682155 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.682142    2576 server.go:1268] "Started kubelet"
Jul 26 18:11:54.682298 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.682264    2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 26 18:11:54.682775 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.682244    2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 26 18:11:54.682775 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.682535    2576 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 26 18:11:54.682892 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.682777    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 26 18:11:54.683040 ip-10-0-134-7 systemd[1]: Started Kubernetes Kubelet.
Jul 26 18:11:54.684238 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.684225    2576 server.go:310] "Adding debug handlers to kubelet server"
Jul 26 18:11:54.685413 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.685136    2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-7.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 26 18:11:54.685413 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:54.685197    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 26 18:11:54.685413 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:54.685285    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-7.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 26 18:11:54.685529 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.685513    2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 26 18:11:54.690967 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:54.690938    2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 26 18:11:54.697703 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.697677    2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 26 18:11:54.697703 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.697686    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 26 18:11:54.697859 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:54.696528    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-7.ec2.internal.18c5e974aee700f9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-7.ec2.internal,UID:ip-10-0-134-7.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-7.ec2.internal,},FirstTimestamp:2026-07-26 18:11:54.682122489 +0000 UTC m=+0.389529541,LastTimestamp:2026-07-26 18:11:54.682122489 +0000 UTC m=+0.389529541,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-7.ec2.internal,}"
Jul 26 18:11:54.698019 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:54.697998    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-7.ec2.internal\" not found"
Jul 26 18:11:54.698419 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.698399    2576 factory.go:55] Registering systemd factory
Jul 26 18:11:54.698419 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.698421    2576 factory.go:223] Registration of the systemd container factory successfully
Jul 26 18:11:54.698550 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.698499    2576 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 26 18:11:54.698550 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.698514    2576 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 26 18:11:54.698550 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.698522    2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 26 18:11:54.698689 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.698616    2576 reconstruct.go:97] "Volume reconstruction finished"
Jul 26 18:11:54.698689 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.698624    2576 reconciler.go:29] "Reconciler: start to sync state"
Jul 26 18:11:54.698787 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.698689    2576 factory.go:153] Registering CRI-O factory
Jul 26 18:11:54.698787 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.698699    2576 factory.go:223] Registration of the crio container factory successfully
Jul 26 18:11:54.698787 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.698735    2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 26 18:11:54.698787 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.698769    2576 factory.go:103] Registering Raw factory
Jul 26 18:11:54.698787 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.698782    2576 manager.go:1196] Started watching for new ooms in manager
Jul 26 18:11:54.699174 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.699158    2576 manager.go:349] Starting recovery of all containers
Jul 26 18:11:54.709017 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.708999    2576 manager.go:354] Recovery completed
Jul 26 18:11:54.710557 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:54.710527    2576 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-7.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 26 18:11:54.710730 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:54.710684    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 26 18:11:54.711525 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:54.711504    2576 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 26 18:11:54.715176 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.715164    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 18:11:54.717010 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.716990    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-7.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 18:11:54.717076 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.717028    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-7.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 18:11:54.717076 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.717042    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-7.ec2.internal" event="NodeHasSufficientPID"
Jul 26 18:11:54.717555 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.717541    2576 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 26 18:11:54.717555 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.717551    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 18:11:54.717677 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.717552    2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 26 18:11:54.717677 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.717581    2576 state_mem.go:36] "Initialized new in-memory state store"
Jul 26 18:11:54.718303 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.718288    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-7.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 18:11:54.718364 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.718315    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-7.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 18:11:54.718364 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.718327    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-7.ec2.internal" event="NodeHasSufficientPID"
Jul 26 18:11:54.719384 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.719371    2576 policy_none.go:49] "None policy: Start"
Jul 26 18:11:54.719448 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.719387    2576 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 26 18:11:54.719448 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.719397    2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 26 18:11:54.720410 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.720400    2576 policy_none.go:47] "Start"
Jul 26 18:11:54.724904 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.724737    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6fn7v"
Jul 26 18:11:54.730026 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:54.729940    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-7.ec2.internal.18c5e974b0fb5d33  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-7.ec2.internal,UID:ip-10-0-134-7.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-7.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-7.ec2.internal,},FirstTimestamp:2026-07-26 18:11:54.717011251 +0000 UTC m=+0.424418303,LastTimestamp:2026-07-26 18:11:54.717011251 +0000 UTC m=+0.424418303,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-7.ec2.internal,}"
Jul 26 18:11:54.732205 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.732191    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6fn7v"
Jul 26 18:11:54.758326 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.758313    2576 manager.go:337] "Starting Device Plugin manager"
Jul 26 18:11:54.773458 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:54.758342    2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 26 18:11:54.773458 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.758353    2576 server.go:87] "Starting device plugin registration server"
Jul 26 18:11:54.773458 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.758578    2576 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 26 18:11:54.773458 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.758590    2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 26 18:11:54.773458 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.758755    2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 26 18:11:54.773458 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.758832    2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 26 18:11:54.773458 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.758841    2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 26 18:11:54.773458 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:54.759397    2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 26 18:11:54.773458 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:54.759456    2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-7.ec2.internal\" not found"
Jul 26 18:11:54.841585 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.841552    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 26 18:11:54.842727 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.842703    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 26 18:11:54.842831 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.842764    2576 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 26 18:11:54.842831 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.842794    2576 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 26 18:11:54.842923 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:54.842836    2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 26 18:11:54.845469 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.845451    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 18:11:54.859585 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.859541    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 18:11:54.862900 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.862884    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-7.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 18:11:54.862983 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.862917    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-7.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 18:11:54.862983 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.862932    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-7.ec2.internal" event="NodeHasSufficientPID"
Jul 26 18:11:54.862983 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.862962    2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-7.ec2.internal"
Jul 26 18:11:54.874127 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.874112    2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-7.ec2.internal"
Jul 26 18:11:54.874180 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:54.874134    2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-7.ec2.internal\": node \"ip-10-0-134-7.ec2.internal\" not found"
Jul 26 18:11:54.890220 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:54.890200    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-7.ec2.internal\" not found"
Jul 26 18:11:54.943726 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.943693    2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-134-7.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-7.ec2.internal"]
Jul 26 18:11:54.943894 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.943774    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 18:11:54.945381 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.945359    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-7.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 18:11:54.945459 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.945403    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-7.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 18:11:54.945459 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.945413    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-7.ec2.internal" event="NodeHasSufficientPID"
Jul 26 18:11:54.946563 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.946550    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 18:11:54.946668 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.946653    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-7.ec2.internal"
Jul 26 18:11:54.946720 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.946681    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 18:11:54.947270 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.947250    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-7.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 18:11:54.947270 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.947270    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-7.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 18:11:54.947395 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.947283    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-7.ec2.internal" event="NodeHasSufficientPID"
Jul 26 18:11:54.947395 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.947289    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-7.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 18:11:54.947395 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.947308    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-7.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 18:11:54.947395 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.947318    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-7.ec2.internal" event="NodeHasSufficientPID"
Jul 26 18:11:54.948509 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.948497    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-7.ec2.internal"
Jul 26 18:11:54.948556 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.948522    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 18:11:54.949150 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.949132    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-7.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 18:11:54.949246 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.949161    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-7.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 18:11:54.949246 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:54.949173    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-7.ec2.internal" event="NodeHasSufficientPID"
Jul 26 18:11:54.975356 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:54.975336    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-7.ec2.internal\" not found" node="ip-10-0-134-7.ec2.internal"
Jul 26 18:11:54.979528 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:54.979514    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-7.ec2.internal\" not found" node="ip-10-0-134-7.ec2.internal"
Jul 26 18:11:54.990271 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:54.990255    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-7.ec2.internal\" not found"
Jul 26 18:11:55.090725 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:55.090705    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-7.ec2.internal\" not found"
Jul 26 18:11:55.100368 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:55.100348    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/75dc32c0f7305fc42c8ffda6c7e0c513-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-7.ec2.internal\" (UID: \"75dc32c0f7305fc42c8ffda6c7e0c513\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-7.ec2.internal"
Jul 26 18:11:55.100432 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:55.100372    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4c3d4ae719d8eb92a6e5cea24a51f094-config\") pod \"kube-apiserver-proxy-ip-10-0-134-7.ec2.internal\" (UID: \"4c3d4ae719d8eb92a6e5cea24a51f094\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-7.ec2.internal"
Jul 26 18:11:55.100432 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:55.100390    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/75dc32c0f7305fc42c8ffda6c7e0c513-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-7.ec2.internal\" (UID: \"75dc32c0f7305fc42c8ffda6c7e0c513\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-7.ec2.internal"
Jul 26 18:11:55.191373 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:55.191285    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-7.ec2.internal\" not found"
Jul 26 18:11:55.200681 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:55.200657    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4c3d4ae719d8eb92a6e5cea24a51f094-config\") pod \"kube-apiserver-proxy-ip-10-0-134-7.ec2.internal\" (UID: \"4c3d4ae719d8eb92a6e5cea24a51f094\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-7.ec2.internal"
Jul 26 18:11:55.200739 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:55.200687    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/75dc32c0f7305fc42c8ffda6c7e0c513-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-7.ec2.internal\" (UID: \"75dc32c0f7305fc42c8ffda6c7e0c513\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-7.ec2.internal"
Jul 26 18:11:55.200739 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:55.200706    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/75dc32c0f7305fc42c8ffda6c7e0c513-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-7.ec2.internal\" (UID: \"75dc32c0f7305fc42c8ffda6c7e0c513\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-7.ec2.internal"
Jul 26 18:11:55.200739 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:55.200733    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/75dc32c0f7305fc42c8ffda6c7e0c513-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-7.ec2.internal\" (UID: \"75dc32c0f7305fc42c8ffda6c7e0c513\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-7.ec2.internal"
Jul 26 18:11:55.200862 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:55.200762    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4c3d4ae719d8eb92a6e5cea24a51f094-config\") pod \"kube-apiserver-proxy-ip-10-0-134-7.ec2.internal\" (UID: \"4c3d4ae719d8eb92a6e5cea24a51f094\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-7.ec2.internal"
Jul 26 18:11:55.200862 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:55.200786    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/75dc32c0f7305fc42c8ffda6c7e0c513-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-7.ec2.internal\" (UID: \"75dc32c0f7305fc42c8ffda6c7e0c513\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-7.ec2.internal"
Jul 26 18:11:55.277862 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:55.277823    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-7.ec2.internal"
Jul 26 18:11:55.283804 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:55.283784    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-7.ec2.internal"
Jul 26 18:11:55.291997 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:55.291970    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-7.ec2.internal\" not found"
Jul 26 18:11:55.392575 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:55.392524    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-7.ec2.internal\" not found"
Jul 26 18:11:55.496666 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:55.496606    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-7.ec2.internal\" not found"
Jul 26 18:11:55.602088 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:55.602059    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-7.ec2.internal\" not found"
Jul 26 18:11:55.630631 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:55.630603    2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 26 18:11:55.631123 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:55.630775    2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 26 18:11:55.690863 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:55.690835    2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 18:11:55.692783 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:55.692749    2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 18:11:55.702188 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:55.702161    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-7.ec2.internal\" not found"
Jul 26 18:11:55.736922 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:55.736876    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-25 18:06:55 +0000 UTC" deadline="2028-04-24 23:59:33.487266512 +0000 UTC"
Jul 26 18:11:55.736922 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:55.736919    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15317h47m37.750352402s"
Jul 26 18:11:55.802290 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:55.802219    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-7.ec2.internal\" not found"
Jul 26 18:11:55.847590 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:55.847553    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75dc32c0f7305fc42c8ffda6c7e0c513.slice/crio-bf3acaadeab5cd7d0cf66af4ddf0d4acc7f9b71925608eb4ea492d2bb96837da WatchSource:0}: Error finding container bf3acaadeab5cd7d0cf66af4ddf0d4acc7f9b71925608eb4ea492d2bb96837da: Status 404 returned error can't find the container with id bf3acaadeab5cd7d0cf66af4ddf0d4acc7f9b71925608eb4ea492d2bb96837da
Jul 26 18:11:55.847838 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:55.847816    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c3d4ae719d8eb92a6e5cea24a51f094.slice/crio-4b82f60c201536d7ce4acced6479cd872cede5b90068aba326fa5bfb82dccbe3 WatchSource:0}: Error finding container 4b82f60c201536d7ce4acced6479cd872cede5b90068aba326fa5bfb82dccbe3: Status 404 returned error can't find the container with id 4b82f60c201536d7ce4acced6479cd872cede5b90068aba326fa5bfb82dccbe3
Jul 26 18:11:55.853045 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:55.853024    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 18:11:55.856905 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:55.856876    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 18:11:55.857850 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:55.857818    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-7.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 18:11:55.857957 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:55.857862    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-7.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 18:11:55.857957 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:55.857878    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-7.ec2.internal" event="NodeHasSufficientPID"
Jul 26 18:11:55.902866 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:55.902840    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-7.ec2.internal\" not found"
Jul 26 18:11:56.005955 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:56.005916    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-7.ec2.internal\" not found"
Jul 26 18:11:56.029018 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.028998    2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 18:11:56.099023 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.098929    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-7.ec2.internal"
Jul 26 18:11:56.108463 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.108434    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 26 18:11:56.109589 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.109571    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-7.ec2.internal"
Jul 26 18:11:56.121736 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.121715    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 26 18:11:56.636525 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.636491    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 18:11:56.671144 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.671110    2576 apiserver.go:52] "Watching apiserver"
Jul 26 18:11:56.680062 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.680029    2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 26 18:11:56.681617 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.681579    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-6rvcp","openshift-ovn-kubernetes/ovnkube-node-5wfmw","kube-system/global-pull-secret-syncer-fsm57","kube-system/konnectivity-agent-6vnjh","openshift-cluster-node-tuning-operator/tuned-hp9vz","openshift-image-registry/node-ca-k9khs","openshift-multus/multus-7z456","openshift-multus/multus-additional-cni-plugins-rb2p7","openshift-multus/network-metrics-daemon-fxp88","openshift-network-diagnostics/network-check-target-rrn49","kube-system/kube-apiserver-proxy-ip-10-0-134-7.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zfbtf","openshift-dns/node-resolver-hz9mx","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-7.ec2.internal"]
Jul 26 18:11:56.683497 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.683472    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.685310 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.685287    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hz9mx"
Jul 26 18:11:56.686596 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.686573    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 26 18:11:56.686596 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.686593    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 26 18:11:56.686825 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.686618    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 26 18:11:56.686825 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.686686    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-sbqfq\""
Jul 26 18:11:56.687283 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.687020    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fsm57"
Jul 26 18:11:56.687283 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.687106    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 26 18:11:56.687283 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:56.687099    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fsm57" podUID="54acf4cd-cd75-4990-9ccf-274ef789daa5"
Jul 26 18:11:56.690794 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.688486    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 26 18:11:56.690794 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.688703    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7xpsm\""
Jul 26 18:11:56.690794 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.689424    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 26 18:11:56.690794 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.689991    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6vnjh"
Jul 26 18:11:56.690794 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.690347    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.691267 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.691198    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-k9khs"
Jul 26 18:11:56.692563 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.692477    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 26 18:11:56.692753 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.692732    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hxm7s\""
Jul 26 18:11:56.693101 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.693008    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 26 18:11:56.693286 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.693256    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-scfts\""
Jul 26 18:11:56.693498 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.693481    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 26 18:11:56.693663 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.693639    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 26 18:11:56.693969 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.693953    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 26 18:11:56.694120 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.694100    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 26 18:11:56.694194 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.694176    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxp88"
Jul 26 18:11:56.694255 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.694233    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 26 18:11:56.694311 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:56.694256    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxp88" podUID="97230a04-09c6-4445-b713-98d115f6b1aa"
Jul 26 18:11:56.694311 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.694181    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-m5fkb\""
Jul 26 18:11:56.695197 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.694997    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6rvcp"
Jul 26 18:11:56.696849 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.696812    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.697794 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.697764    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 26 18:11:56.699207 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.698026    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gp8gb\""
Jul 26 18:11:56.699207 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.698218    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 26 18:11:56.700634 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.700369    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 26 18:11:56.700634 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.700481    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rrn49"
Jul 26 18:11:56.700634 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:56.700561    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rrn49" podUID="8a2c5810-fcc5-4224-8d19-5fe5cc832d5a"
Jul 26 18:11:56.701946 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.701233    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 26 18:11:56.701946 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.701499    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-t6z9t\""
Jul 26 18:11:56.701946 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.701529    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 26 18:11:56.701946 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.701708    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 26 18:11:56.701946 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.701722    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 26 18:11:56.702459 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.702439    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 26 18:11:56.702877 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.702855    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 26 18:11:56.702999 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.702875    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 26 18:11:56.709728 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.709708    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rb2p7"
Jul 26 18:11:56.710447 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.710421    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97230a04-09c6-4445-b713-98d115f6b1aa-metrics-certs\") pod \"network-metrics-daemon-fxp88\" (UID: \"97230a04-09c6-4445-b713-98d115f6b1aa\") " pod="openshift-multus/network-metrics-daemon-fxp88"
Jul 26 18:11:56.710544 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.710460    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-systemd-units\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.710544 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.710487    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/04f08429-551f-4765-8985-6dc6d0256dab-host-slash\") pod \"iptables-alerter-6rvcp\" (UID: \"04f08429-551f-4765-8985-6dc6d0256dab\") " pod="openshift-network-operator/iptables-alerter-6rvcp"
Jul 26 18:11:56.710544 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.710515    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/261aba58-ddd5-4f9e-af39-7475dec519f9-host\") pod \"node-ca-k9khs\" (UID: \"261aba58-ddd5-4f9e-af39-7475dec519f9\") " pod="openshift-image-registry/node-ca-k9khs"
Jul 26 18:11:56.710544 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.710541    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-etc-openvswitch\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.710789 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.710564    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/94b1d70f-c671-4fe3-bc2c-05b9426a496f-ovnkube-config\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.710789 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.710588    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/94b1d70f-c671-4fe3-bc2c-05b9426a496f-ovnkube-script-lib\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.710789 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.710612    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-system-cni-dir\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.710789 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.710637    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d94bk\" (UniqueName: \"kubernetes.io/projected/04f08429-551f-4765-8985-6dc6d0256dab-kube-api-access-d94bk\") pod \"iptables-alerter-6rvcp\" (UID: \"04f08429-551f-4765-8985-6dc6d0256dab\") " pod="openshift-network-operator/iptables-alerter-6rvcp"
Jul 26 18:11:56.711050 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.710845    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/54acf4cd-cd75-4990-9ccf-274ef789daa5-kubelet-config\") pod \"global-pull-secret-syncer-fsm57\" (UID: \"54acf4cd-cd75-4990-9ccf-274ef789daa5\") " pod="kube-system/global-pull-secret-syncer-fsm57"
Jul 26 18:11:56.711050 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.710875    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49e6be1b-9cd1-4a86-a30b-1c414e573659-tmp\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.711050 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.710932    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-var-lib-openvswitch\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.711050 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.710965    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.711050 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.710990    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-multus-socket-dir-parent\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.711050 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.711014    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-host-var-lib-kubelet\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.711050 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.711038    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e895b2b-026e-4228-9fc7-fbe90089f5a9-tmp-dir\") pod \"node-resolver-hz9mx\" (UID: \"7e895b2b-026e-4228-9fc7-fbe90089f5a9\") " pod="openshift-dns/node-resolver-hz9mx"
Jul 26 18:11:56.711406 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.711063    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/49e6be1b-9cd1-4a86-a30b-1c414e573659-etc-sysctl-conf\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.711406 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.711140    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bm45\" (UniqueName: \"kubernetes.io/projected/49e6be1b-9cd1-4a86-a30b-1c414e573659-kube-api-access-4bm45\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.711406 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.711163    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-host-run-netns\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.711406 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.711219    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-host-var-lib-cni-bin\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.711406 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.711268    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-host-cni-bin\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.711406 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.711319    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/94b1d70f-c671-4fe3-bc2c-05b9426a496f-ovn-node-metrics-cert\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.711406 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.711343    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-cnibin\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.711406 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.711370    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fb2kx\" (UniqueName: \"kubernetes.io/projected/5c253be6-2414-4ed1-b8b0-d0758f838ab9-kube-api-access-fb2kx\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.711406 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.711395    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/04f08429-551f-4765-8985-6dc6d0256dab-iptables-alerter-script\") pod \"iptables-alerter-6rvcp\" (UID: \"04f08429-551f-4765-8985-6dc6d0256dab\") " pod="openshift-network-operator/iptables-alerter-6rvcp"
Jul 26 18:11:56.711923 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.711418    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/49e6be1b-9cd1-4a86-a30b-1c414e573659-run\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.711923 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.711458    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/49e6be1b-9cd1-4a86-a30b-1c414e573659-host\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.711923 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.711519    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-host-slash\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.711923 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.711559    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-run-openvswitch\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.711923 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.711586    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-multus-conf-dir\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.711923 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.711659    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/54acf4cd-cd75-4990-9ccf-274ef789daa5-dbus\") pod \"global-pull-secret-syncer-fsm57\" (UID: \"54acf4cd-cd75-4990-9ccf-274ef789daa5\") " pod="kube-system/global-pull-secret-syncer-fsm57"
Jul 26 18:11:56.711923 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.711689    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/261aba58-ddd5-4f9e-af39-7475dec519f9-serviceca\") pod \"node-ca-k9khs\" (UID: \"261aba58-ddd5-4f9e-af39-7475dec519f9\") " pod="openshift-image-registry/node-ca-k9khs"
Jul 26 18:11:56.711923 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.711714    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/94b1d70f-c671-4fe3-bc2c-05b9426a496f-env-overrides\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.711923 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.711736    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-etc-kubernetes\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.711923 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.711783    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/65820b5a-b897-45fc-ae26-e286b5444375-konnectivity-ca\") pod \"konnectivity-agent-6vnjh\" (UID: \"65820b5a-b897-45fc-ae26-e286b5444375\") " pod="kube-system/konnectivity-agent-6vnjh"
Jul 26 18:11:56.711923 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.711813    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/54acf4cd-cd75-4990-9ccf-274ef789daa5-original-pull-secret\") pod \"global-pull-secret-syncer-fsm57\" (UID: \"54acf4cd-cd75-4990-9ccf-274ef789daa5\") " pod="kube-system/global-pull-secret-syncer-fsm57"
Jul 26 18:11:56.711923 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.711843    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/49e6be1b-9cd1-4a86-a30b-1c414e573659-etc-systemd\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.711923 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.711867    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-log-socket\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.711923 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.711889    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-925kz\" (UniqueName: \"kubernetes.io/projected/94b1d70f-c671-4fe3-bc2c-05b9426a496f-kube-api-access-925kz\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.712651 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.711942    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5c253be6-2414-4ed1-b8b0-d0758f838ab9-multus-daemon-config\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.712651 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.711966    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7e895b2b-026e-4228-9fc7-fbe90089f5a9-hosts-file\") pod \"node-resolver-hz9mx\" (UID: \"7e895b2b-026e-4228-9fc7-fbe90089f5a9\") " pod="openshift-dns/node-resolver-hz9mx"
Jul 26 18:11:56.712651 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.711989    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/49e6be1b-9cd1-4a86-a30b-1c414e573659-etc-modprobe-d\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.712651 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.712013    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/49e6be1b-9cd1-4a86-a30b-1c414e573659-etc-sysconfig\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.712651 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.712036    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/49e6be1b-9cd1-4a86-a30b-1c414e573659-lib-modules\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.712651 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.712082    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-run-systemd\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.712651 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.712105    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-node-log\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.712651 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.712128    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-host-kubelet\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.712651 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.712156    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-host-cni-netd\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.712651 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.712201    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29pz7\" (UniqueName: \"kubernetes.io/projected/7e895b2b-026e-4228-9fc7-fbe90089f5a9-kube-api-access-29pz7\") pod \"node-resolver-hz9mx\" (UID: \"7e895b2b-026e-4228-9fc7-fbe90089f5a9\") " pod="openshift-dns/node-resolver-hz9mx"
Jul 26 18:11:56.712651 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.712226    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/49e6be1b-9cd1-4a86-a30b-1c414e573659-etc-sysctl-d\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.712651 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.712255    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-host-var-lib-cni-multus\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.712651 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.712281    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-host-run-multus-certs\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.712651 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.712306    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/65820b5a-b897-45fc-ae26-e286b5444375-agent-certs\") pod \"konnectivity-agent-6vnjh\" (UID: \"65820b5a-b897-45fc-ae26-e286b5444375\") " pod="kube-system/konnectivity-agent-6vnjh"
Jul 26 18:11:56.712651 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.712383    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/49e6be1b-9cd1-4a86-a30b-1c414e573659-sys\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.712651 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.712433    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-host-run-k8s-cni-cncf-io\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.712651 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.712476    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qmwr\" (UniqueName: \"kubernetes.io/projected/261aba58-ddd5-4f9e-af39-7475dec519f9-kube-api-access-6qmwr\") pod \"node-ca-k9khs\" (UID: \"261aba58-ddd5-4f9e-af39-7475dec519f9\") " pod="openshift-image-registry/node-ca-k9khs"
Jul 26 18:11:56.713474 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.712505    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-run-ovn\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.713474 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.712529    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5c253be6-2414-4ed1-b8b0-d0758f838ab9-cni-binary-copy\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.713474 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.712555    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qzgk\" (UniqueName: \"kubernetes.io/projected/8a2c5810-fcc5-4224-8d19-5fe5cc832d5a-kube-api-access-7qzgk\") pod \"network-check-target-rrn49\" (UID: \"8a2c5810-fcc5-4224-8d19-5fe5cc832d5a\") " pod="openshift-network-diagnostics/network-check-target-rrn49"
Jul 26 18:11:56.713474 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.712582    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-multus-cni-dir\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.713474 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.712607    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-os-release\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.713474 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.712631    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/49e6be1b-9cd1-4a86-a30b-1c414e573659-etc-kubernetes\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.713474 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.712656    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/49e6be1b-9cd1-4a86-a30b-1c414e573659-var-lib-kubelet\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.713474 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.712683    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/49e6be1b-9cd1-4a86-a30b-1c414e573659-etc-tuned\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.713474 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.712722    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpr97\" (UniqueName: \"kubernetes.io/projected/97230a04-09c6-4445-b713-98d115f6b1aa-kube-api-access-dpr97\") pod \"network-metrics-daemon-fxp88\" (UID: \"97230a04-09c6-4445-b713-98d115f6b1aa\") " pod="openshift-multus/network-metrics-daemon-fxp88"
Jul 26 18:11:56.713474 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.712746    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-host-run-ovn-kubernetes\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.713474 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.712760    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-cgjb9\""
Jul 26 18:11:56.713474 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.712770    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-host-run-netns\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.713474 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.712843    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-hostroot\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.713474 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.713039    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 26 18:11:56.713474 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.713050    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 26 18:11:56.713474 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.713338    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zfbtf"
Jul 26 18:11:56.718709 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.717498    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 26 18:11:56.718709 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.717713    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 26 18:11:56.718709 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.717950    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 26 18:11:56.718709 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.718244    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-q97xj\""
Jul 26 18:11:56.718709 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.718445    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 26 18:11:56.741446 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.741421    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-l99g4"
Jul 26 18:11:56.751967 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.751942    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-l99g4"
Jul 26 18:11:56.799690 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.799660    2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 26 18:11:56.813256 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.813231    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/94b1d70f-c671-4fe3-bc2c-05b9426a496f-env-overrides\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.813256 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.813261    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-etc-kubernetes\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.813439 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.813277    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/65820b5a-b897-45fc-ae26-e286b5444375-konnectivity-ca\") pod \"konnectivity-agent-6vnjh\" (UID: \"65820b5a-b897-45fc-ae26-e286b5444375\") " pod="kube-system/konnectivity-agent-6vnjh"
Jul 26 18:11:56.813439 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.813306    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/63e8eeac-36f8-46a7-add2-5e222dac8d5f-system-cni-dir\") pod \"multus-additional-cni-plugins-rb2p7\" (UID: \"63e8eeac-36f8-46a7-add2-5e222dac8d5f\") " pod="openshift-multus/multus-additional-cni-plugins-rb2p7"
Jul 26 18:11:56.813439 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.813335    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/63e8eeac-36f8-46a7-add2-5e222dac8d5f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rb2p7\" (UID: \"63e8eeac-36f8-46a7-add2-5e222dac8d5f\") " pod="openshift-multus/multus-additional-cni-plugins-rb2p7"
Jul 26 18:11:56.813439 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.813358    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/54acf4cd-cd75-4990-9ccf-274ef789daa5-original-pull-secret\") pod \"global-pull-secret-syncer-fsm57\" (UID: \"54acf4cd-cd75-4990-9ccf-274ef789daa5\") " pod="kube-system/global-pull-secret-syncer-fsm57"
Jul 26 18:11:56.813439 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.813376    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/49e6be1b-9cd1-4a86-a30b-1c414e573659-etc-systemd\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.813439 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.813398    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-log-socket\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.813439 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.813422    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-925kz\" (UniqueName: \"kubernetes.io/projected/94b1d70f-c671-4fe3-bc2c-05b9426a496f-kube-api-access-925kz\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.813717 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.813446    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5c253be6-2414-4ed1-b8b0-d0758f838ab9-multus-daemon-config\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.813717 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.813465    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7e895b2b-026e-4228-9fc7-fbe90089f5a9-hosts-file\") pod \"node-resolver-hz9mx\" (UID: \"7e895b2b-026e-4228-9fc7-fbe90089f5a9\") " pod="openshift-dns/node-resolver-hz9mx"
Jul 26 18:11:56.813717 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.813481    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/63e8eeac-36f8-46a7-add2-5e222dac8d5f-cnibin\") pod \"multus-additional-cni-plugins-rb2p7\" (UID: \"63e8eeac-36f8-46a7-add2-5e222dac8d5f\") " pod="openshift-multus/multus-additional-cni-plugins-rb2p7"
Jul 26 18:11:56.813717 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.813496    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/49e6be1b-9cd1-4a86-a30b-1c414e573659-etc-modprobe-d\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.813717 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.813515    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/49e6be1b-9cd1-4a86-a30b-1c414e573659-etc-sysconfig\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.813717 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.813540    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/49e6be1b-9cd1-4a86-a30b-1c414e573659-lib-modules\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.813717 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.813565    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-run-systemd\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.813717 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.813590    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-node-log\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.813717 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.813615    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-host-kubelet\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.813717 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.813654    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-host-cni-netd\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.813717 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.813679    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29pz7\" (UniqueName: \"kubernetes.io/projected/7e895b2b-026e-4228-9fc7-fbe90089f5a9-kube-api-access-29pz7\") pod \"node-resolver-hz9mx\" (UID: \"7e895b2b-026e-4228-9fc7-fbe90089f5a9\") " pod="openshift-dns/node-resolver-hz9mx"
Jul 26 18:11:56.813717 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.813707    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/63e8eeac-36f8-46a7-add2-5e222dac8d5f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rb2p7\" (UID: \"63e8eeac-36f8-46a7-add2-5e222dac8d5f\") " pod="openshift-multus/multus-additional-cni-plugins-rb2p7"
Jul 26 18:11:56.814248 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.813733    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f7378af1-e3bd-48fa-a2d6-1b5db464dbb3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zfbtf\" (UID: \"f7378af1-e3bd-48fa-a2d6-1b5db464dbb3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zfbtf"
Jul 26 18:11:56.814248 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.813758    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/49e6be1b-9cd1-4a86-a30b-1c414e573659-etc-sysctl-d\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.814248 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.813786    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-host-var-lib-cni-multus\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.814248 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.813827    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-host-run-multus-certs\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.814248 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.813852    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/65820b5a-b897-45fc-ae26-e286b5444375-agent-certs\") pod \"konnectivity-agent-6vnjh\" (UID: \"65820b5a-b897-45fc-ae26-e286b5444375\") " pod="kube-system/konnectivity-agent-6vnjh"
Jul 26 18:11:56.814248 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.813859    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7e895b2b-026e-4228-9fc7-fbe90089f5a9-hosts-file\") pod \"node-resolver-hz9mx\" (UID: \"7e895b2b-026e-4228-9fc7-fbe90089f5a9\") " pod="openshift-dns/node-resolver-hz9mx"
Jul 26 18:11:56.814248 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.813874    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/49e6be1b-9cd1-4a86-a30b-1c414e573659-sys\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.814248 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.813891    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-host-run-k8s-cni-cncf-io\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.814248 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.813909    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/63e8eeac-36f8-46a7-add2-5e222dac8d5f-cni-binary-copy\") pod \"multus-additional-cni-plugins-rb2p7\" (UID: \"63e8eeac-36f8-46a7-add2-5e222dac8d5f\") " pod="openshift-multus/multus-additional-cni-plugins-rb2p7"
Jul 26 18:11:56.814248 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.813925    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qmwr\" (UniqueName: \"kubernetes.io/projected/261aba58-ddd5-4f9e-af39-7475dec519f9-kube-api-access-6qmwr\") pod \"node-ca-k9khs\" (UID: \"261aba58-ddd5-4f9e-af39-7475dec519f9\") " pod="openshift-image-registry/node-ca-k9khs"
Jul 26 18:11:56.814248 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.813952    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-run-ovn\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.814248 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.813977    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5c253be6-2414-4ed1-b8b0-d0758f838ab9-cni-binary-copy\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.814248 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814004    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qzgk\" (UniqueName: \"kubernetes.io/projected/8a2c5810-fcc5-4224-8d19-5fe5cc832d5a-kube-api-access-7qzgk\") pod \"network-check-target-rrn49\" (UID: \"8a2c5810-fcc5-4224-8d19-5fe5cc832d5a\") " pod="openshift-network-diagnostics/network-check-target-rrn49"
Jul 26 18:11:56.814248 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814031    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f7378af1-e3bd-48fa-a2d6-1b5db464dbb3-registration-dir\") pod \"aws-ebs-csi-driver-node-zfbtf\" (UID: \"f7378af1-e3bd-48fa-a2d6-1b5db464dbb3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zfbtf"
Jul 26 18:11:56.814248 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814057    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-multus-cni-dir\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.814248 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814095    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-os-release\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.814248 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814121    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/49e6be1b-9cd1-4a86-a30b-1c414e573659-etc-kubernetes\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.815265 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814145    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/49e6be1b-9cd1-4a86-a30b-1c414e573659-var-lib-kubelet\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.815265 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814169    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/49e6be1b-9cd1-4a86-a30b-1c414e573659-etc-tuned\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.815265 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814194    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpr97\" (UniqueName: \"kubernetes.io/projected/97230a04-09c6-4445-b713-98d115f6b1aa-kube-api-access-dpr97\") pod \"network-metrics-daemon-fxp88\" (UID: \"97230a04-09c6-4445-b713-98d115f6b1aa\") " pod="openshift-multus/network-metrics-daemon-fxp88"
Jul 26 18:11:56.815265 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814200    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-host-var-lib-cni-multus\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.815265 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814221    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-host-run-ovn-kubernetes\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.815265 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814253    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-host-run-netns\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.815265 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814274    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-hostroot\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.815265 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814295    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97230a04-09c6-4445-b713-98d115f6b1aa-metrics-certs\") pod \"network-metrics-daemon-fxp88\" (UID: \"97230a04-09c6-4445-b713-98d115f6b1aa\") " pod="openshift-multus/network-metrics-daemon-fxp88"
Jul 26 18:11:56.815265 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814318    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-systemd-units\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.815265 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814318    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/94b1d70f-c671-4fe3-bc2c-05b9426a496f-env-overrides\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.815265 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814343    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/04f08429-551f-4765-8985-6dc6d0256dab-host-slash\") pod \"iptables-alerter-6rvcp\" (UID: \"04f08429-551f-4765-8985-6dc6d0256dab\") " pod="openshift-network-operator/iptables-alerter-6rvcp"
Jul 26 18:11:56.815265 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814341    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/49e6be1b-9cd1-4a86-a30b-1c414e573659-etc-modprobe-d\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.815265 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814388    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/04f08429-551f-4765-8985-6dc6d0256dab-host-slash\") pod \"iptables-alerter-6rvcp\" (UID: \"04f08429-551f-4765-8985-6dc6d0256dab\") " pod="openshift-network-operator/iptables-alerter-6rvcp"
Jul 26 18:11:56.815265 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814393    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/261aba58-ddd5-4f9e-af39-7475dec519f9-host\") pod \"node-ca-k9khs\" (UID: \"261aba58-ddd5-4f9e-af39-7475dec519f9\") " pod="openshift-image-registry/node-ca-k9khs"
Jul 26 18:11:56.815265 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814400    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/49e6be1b-9cd1-4a86-a30b-1c414e573659-etc-sysconfig\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.815265 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814428    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-etc-openvswitch\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.815265 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814459    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-host-run-multus-certs\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.815265 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814459    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/94b1d70f-c671-4fe3-bc2c-05b9426a496f-ovnkube-config\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.816201 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814475    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/49e6be1b-9cd1-4a86-a30b-1c414e573659-lib-modules\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.816201 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814506    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/94b1d70f-c671-4fe3-bc2c-05b9426a496f-ovnkube-script-lib\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.816201 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814533    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-system-cni-dir\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.816201 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814558    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d94bk\" (UniqueName: \"kubernetes.io/projected/04f08429-551f-4765-8985-6dc6d0256dab-kube-api-access-d94bk\") pod \"iptables-alerter-6rvcp\" (UID: \"04f08429-551f-4765-8985-6dc6d0256dab\") " pod="openshift-network-operator/iptables-alerter-6rvcp"
Jul 26 18:11:56.816201 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814585    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/54acf4cd-cd75-4990-9ccf-274ef789daa5-kubelet-config\") pod \"global-pull-secret-syncer-fsm57\" (UID: \"54acf4cd-cd75-4990-9ccf-274ef789daa5\") " pod="kube-system/global-pull-secret-syncer-fsm57"
Jul 26 18:11:56.816201 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814610    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49e6be1b-9cd1-4a86-a30b-1c414e573659-tmp\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.816201 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814634    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-var-lib-openvswitch\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.816201 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814663    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.816201 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814697    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-multus-socket-dir-parent\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.816201 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814725    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-host-var-lib-kubelet\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.816201 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814750    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e895b2b-026e-4228-9fc7-fbe90089f5a9-tmp-dir\") pod \"node-resolver-hz9mx\" (UID: \"7e895b2b-026e-4228-9fc7-fbe90089f5a9\") " pod="openshift-dns/node-resolver-hz9mx"
Jul 26 18:11:56.816201 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814781    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/63e8eeac-36f8-46a7-add2-5e222dac8d5f-os-release\") pod \"multus-additional-cni-plugins-rb2p7\" (UID: \"63e8eeac-36f8-46a7-add2-5e222dac8d5f\") " pod="openshift-multus/multus-additional-cni-plugins-rb2p7"
Jul 26 18:11:56.816201 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814809    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/49e6be1b-9cd1-4a86-a30b-1c414e573659-etc-sysctl-conf\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.816201 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814803    2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 26 18:11:56.816201 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814861    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bm45\" (UniqueName: \"kubernetes.io/projected/49e6be1b-9cd1-4a86-a30b-1c414e573659-kube-api-access-4bm45\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.816201 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814886    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-host-run-netns\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.816201 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814911    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-host-var-lib-cni-bin\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.816928 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814935    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqchl\" (UniqueName: \"kubernetes.io/projected/63e8eeac-36f8-46a7-add2-5e222dac8d5f-kube-api-access-kqchl\") pod \"multus-additional-cni-plugins-rb2p7\" (UID: \"63e8eeac-36f8-46a7-add2-5e222dac8d5f\") " pod="openshift-multus/multus-additional-cni-plugins-rb2p7"
Jul 26 18:11:56.816928 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814958    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f7378af1-e3bd-48fa-a2d6-1b5db464dbb3-socket-dir\") pod \"aws-ebs-csi-driver-node-zfbtf\" (UID: \"f7378af1-e3bd-48fa-a2d6-1b5db464dbb3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zfbtf"
Jul 26 18:11:56.816928 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814983    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f7378af1-e3bd-48fa-a2d6-1b5db464dbb3-device-dir\") pod \"aws-ebs-csi-driver-node-zfbtf\" (UID: \"f7378af1-e3bd-48fa-a2d6-1b5db464dbb3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zfbtf"
Jul 26 18:11:56.816928 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.814991    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/94b1d70f-c671-4fe3-bc2c-05b9426a496f-ovnkube-config\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.816928 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.815011    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f7378af1-e3bd-48fa-a2d6-1b5db464dbb3-sys-fs\") pod \"aws-ebs-csi-driver-node-zfbtf\" (UID: \"f7378af1-e3bd-48fa-a2d6-1b5db464dbb3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zfbtf"
Jul 26 18:11:56.816928 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.815059    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-etc-kubernetes\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.816928 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.815080    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-run-systemd\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.816928 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.815129    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-host-kubelet\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.816928 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.815159    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.816928 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.815200    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-os-release\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.816928 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.815258    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/49e6be1b-9cd1-4a86-a30b-1c414e573659-sys\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.816928 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.815263    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-multus-socket-dir-parent\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.816928 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.815317    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-host-run-k8s-cni-cncf-io\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.816928 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.815323    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-host-run-netns\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.816928 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.815367    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-host-var-lib-cni-bin\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.816928 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.815466    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-run-ovn\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.816928 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.815621    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/65820b5a-b897-45fc-ae26-e286b5444375-konnectivity-ca\") pod \"konnectivity-agent-6vnjh\" (UID: \"65820b5a-b897-45fc-ae26-e286b5444375\") " pod="kube-system/konnectivity-agent-6vnjh"
Jul 26 18:11:56.817453 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.815658    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-host-cni-bin\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.817453 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.815678    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/94b1d70f-c671-4fe3-bc2c-05b9426a496f-ovn-node-metrics-cert\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.817453 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.816471    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-cnibin\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.817453 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.816674    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fb2kx\" (UniqueName: \"kubernetes.io/projected/5c253be6-2414-4ed1-b8b0-d0758f838ab9-kube-api-access-fb2kx\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.817453 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.816718    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/04f08429-551f-4765-8985-6dc6d0256dab-iptables-alerter-script\") pod \"iptables-alerter-6rvcp\" (UID: \"04f08429-551f-4765-8985-6dc6d0256dab\") " pod="openshift-network-operator/iptables-alerter-6rvcp"
Jul 26 18:11:56.817453 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.816756    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f7378af1-e3bd-48fa-a2d6-1b5db464dbb3-etc-selinux\") pod \"aws-ebs-csi-driver-node-zfbtf\" (UID: \"f7378af1-e3bd-48fa-a2d6-1b5db464dbb3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zfbtf"
Jul 26 18:11:56.817453 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.816807    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjp4l\" (UniqueName: \"kubernetes.io/projected/f7378af1-e3bd-48fa-a2d6-1b5db464dbb3-kube-api-access-qjp4l\") pod \"aws-ebs-csi-driver-node-zfbtf\" (UID: \"f7378af1-e3bd-48fa-a2d6-1b5db464dbb3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zfbtf"
Jul 26 18:11:56.817453 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.816850    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/49e6be1b-9cd1-4a86-a30b-1c414e573659-run\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.817453 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.816884    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/49e6be1b-9cd1-4a86-a30b-1c414e573659-host\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.817453 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.816916    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-host-slash\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.817453 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.816949    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-run-openvswitch\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.817453 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.816983    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-multus-conf-dir\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.817453 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.817013    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/63e8eeac-36f8-46a7-add2-5e222dac8d5f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rb2p7\" (UID: \"63e8eeac-36f8-46a7-add2-5e222dac8d5f\") " pod="openshift-multus/multus-additional-cni-plugins-rb2p7"
Jul 26 18:11:56.817453 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.817049    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/54acf4cd-cd75-4990-9ccf-274ef789daa5-dbus\") pod \"global-pull-secret-syncer-fsm57\" (UID: \"54acf4cd-cd75-4990-9ccf-274ef789daa5\") " pod="kube-system/global-pull-secret-syncer-fsm57"
Jul 26 18:11:56.817453 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.817083    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/261aba58-ddd5-4f9e-af39-7475dec519f9-serviceca\") pod \"node-ca-k9khs\" (UID: \"261aba58-ddd5-4f9e-af39-7475dec519f9\") " pod="openshift-image-registry/node-ca-k9khs"
Jul 26 18:11:56.817453 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.817376    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5c253be6-2414-4ed1-b8b0-d0758f838ab9-cni-binary-copy\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.817945 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.817586    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/261aba58-ddd5-4f9e-af39-7475dec519f9-serviceca\") pod \"node-ca-k9khs\" (UID: \"261aba58-ddd5-4f9e-af39-7475dec519f9\") " pod="openshift-image-registry/node-ca-k9khs"
Jul 26 18:11:56.817945 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:56.817726    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 18:11:56.817945 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:56.817807    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54acf4cd-cd75-4990-9ccf-274ef789daa5-original-pull-secret podName:54acf4cd-cd75-4990-9ccf-274ef789daa5 nodeName:}" failed. No retries permitted until 2026-07-26 18:11:57.317777444 +0000 UTC m=+3.025184504 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/54acf4cd-cd75-4990-9ccf-274ef789daa5-original-pull-secret") pod "global-pull-secret-syncer-fsm57" (UID: "54acf4cd-cd75-4990-9ccf-274ef789daa5") : object "kube-system"/"original-pull-secret" not registered
Jul 26 18:11:56.817945 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.817894    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/94b1d70f-c671-4fe3-bc2c-05b9426a496f-ovnkube-script-lib\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.818072 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.817979    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/49e6be1b-9cd1-4a86-a30b-1c414e573659-etc-systemd\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.818072 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.818016    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-log-socket\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.818295 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.818276    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/54acf4cd-cd75-4990-9ccf-274ef789daa5-kubelet-config\") pod \"global-pull-secret-syncer-fsm57\" (UID: \"54acf4cd-cd75-4990-9ccf-274ef789daa5\") " pod="kube-system/global-pull-secret-syncer-fsm57"
Jul 26 18:11:56.818331 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.818306    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/49e6be1b-9cd1-4a86-a30b-1c414e573659-run\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.818365 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.818345    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-system-cni-dir\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.818396 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.818377    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/49e6be1b-9cd1-4a86-a30b-1c414e573659-host\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.818453 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.818435    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-host-slash\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.818510 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.818494    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-run-openvswitch\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.818570 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.818556    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-multus-conf-dir\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.818709 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.818695    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/54acf4cd-cd75-4990-9ccf-274ef789daa5-dbus\") pod \"global-pull-secret-syncer-fsm57\" (UID: \"54acf4cd-cd75-4990-9ccf-274ef789daa5\") " pod="kube-system/global-pull-secret-syncer-fsm57"
Jul 26 18:11:56.818808 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.818788    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-cnibin\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.819011 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.818860    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-var-lib-openvswitch\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.819257 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.819236    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-node-log\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.819390 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.819281    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/49e6be1b-9cd1-4a86-a30b-1c414e573659-var-lib-kubelet\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.819390 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.819331    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/49e6be1b-9cd1-4a86-a30b-1c414e573659-etc-kubernetes\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.819539 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.819419    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-multus-cni-dir\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.819539 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.819499    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/04f08429-551f-4765-8985-6dc6d0256dab-iptables-alerter-script\") pod \"iptables-alerter-6rvcp\" (UID: \"04f08429-551f-4765-8985-6dc6d0256dab\") " pod="openshift-network-operator/iptables-alerter-6rvcp"
Jul 26 18:11:56.821581 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.819587    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-etc-openvswitch\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.821581 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.819677    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/49e6be1b-9cd1-4a86-a30b-1c414e573659-etc-sysctl-d\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.821581 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.819700    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e895b2b-026e-4228-9fc7-fbe90089f5a9-tmp-dir\") pod \"node-resolver-hz9mx\" (UID: \"7e895b2b-026e-4228-9fc7-fbe90089f5a9\") " pod="openshift-dns/node-resolver-hz9mx"
Jul 26 18:11:56.821581 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.819732    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-host-cni-netd\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.821581 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.819813    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-host-run-netns\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.821581 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.819812    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-host-var-lib-kubelet\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.821581 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.819888    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-host-cni-bin\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.821581 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.819940    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/261aba58-ddd5-4f9e-af39-7475dec519f9-host\") pod \"node-ca-k9khs\" (UID: \"261aba58-ddd5-4f9e-af39-7475dec519f9\") " pod="openshift-image-registry/node-ca-k9khs"
Jul 26 18:11:56.821581 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.819991    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-host-run-ovn-kubernetes\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.821581 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:56.820103    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 18:11:56.821581 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.820102    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5c253be6-2414-4ed1-b8b0-d0758f838ab9-multus-daemon-config\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.821581 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.820137    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/94b1d70f-c671-4fe3-bc2c-05b9426a496f-systemd-units\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.821581 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.820184    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5c253be6-2414-4ed1-b8b0-d0758f838ab9-hostroot\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.821581 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:56.820203    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97230a04-09c6-4445-b713-98d115f6b1aa-metrics-certs podName:97230a04-09c6-4445-b713-98d115f6b1aa nodeName:}" failed. No retries permitted until 2026-07-26 18:11:57.320144372 +0000 UTC m=+3.027551410 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97230a04-09c6-4445-b713-98d115f6b1aa-metrics-certs") pod "network-metrics-daemon-fxp88" (UID: "97230a04-09c6-4445-b713-98d115f6b1aa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 18:11:56.821581 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.820293    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/49e6be1b-9cd1-4a86-a30b-1c414e573659-etc-sysctl-conf\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.821581 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.821587    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49e6be1b-9cd1-4a86-a30b-1c414e573659-tmp\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.823962 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.821634    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/49e6be1b-9cd1-4a86-a30b-1c414e573659-etc-tuned\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.823962 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.823381    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/65820b5a-b897-45fc-ae26-e286b5444375-agent-certs\") pod \"konnectivity-agent-6vnjh\" (UID: \"65820b5a-b897-45fc-ae26-e286b5444375\") " pod="kube-system/konnectivity-agent-6vnjh"
Jul 26 18:11:56.825997 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.825973    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/94b1d70f-c671-4fe3-bc2c-05b9426a496f-ovn-node-metrics-cert\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.830435 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:56.830413    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 18:11:56.830537 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:56.830441    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 18:11:56.830537 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:56.830456    2576 projected.go:196] Error preparing data for projected volume kube-api-access-7qzgk for pod openshift-network-diagnostics/network-check-target-rrn49: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 18:11:56.830537 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:56.830511    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a2c5810-fcc5-4224-8d19-5fe5cc832d5a-kube-api-access-7qzgk podName:8a2c5810-fcc5-4224-8d19-5fe5cc832d5a nodeName:}" failed. No retries permitted until 2026-07-26 18:11:57.330494308 +0000 UTC m=+3.037901367 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7qzgk" (UniqueName: "kubernetes.io/projected/8a2c5810-fcc5-4224-8d19-5fe5cc832d5a-kube-api-access-7qzgk") pod "network-check-target-rrn49" (UID: "8a2c5810-fcc5-4224-8d19-5fe5cc832d5a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 18:11:56.831170 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.831145    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bm45\" (UniqueName: \"kubernetes.io/projected/49e6be1b-9cd1-4a86-a30b-1c414e573659-kube-api-access-4bm45\") pod \"tuned-hp9vz\" (UID: \"49e6be1b-9cd1-4a86-a30b-1c414e573659\") " pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:56.834031 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.833986    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qmwr\" (UniqueName: \"kubernetes.io/projected/261aba58-ddd5-4f9e-af39-7475dec519f9-kube-api-access-6qmwr\") pod \"node-ca-k9khs\" (UID: \"261aba58-ddd5-4f9e-af39-7475dec519f9\") " pod="openshift-image-registry/node-ca-k9khs"
Jul 26 18:11:56.834183 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.834161    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29pz7\" (UniqueName: \"kubernetes.io/projected/7e895b2b-026e-4228-9fc7-fbe90089f5a9-kube-api-access-29pz7\") pod \"node-resolver-hz9mx\" (UID: \"7e895b2b-026e-4228-9fc7-fbe90089f5a9\") " pod="openshift-dns/node-resolver-hz9mx"
Jul 26 18:11:56.834891 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.834865    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-925kz\" (UniqueName: \"kubernetes.io/projected/94b1d70f-c671-4fe3-bc2c-05b9426a496f-kube-api-access-925kz\") pod \"ovnkube-node-5wfmw\" (UID: \"94b1d70f-c671-4fe3-bc2c-05b9426a496f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:56.835000 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.834898    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpr97\" (UniqueName: \"kubernetes.io/projected/97230a04-09c6-4445-b713-98d115f6b1aa-kube-api-access-dpr97\") pod \"network-metrics-daemon-fxp88\" (UID: \"97230a04-09c6-4445-b713-98d115f6b1aa\") " pod="openshift-multus/network-metrics-daemon-fxp88"
Jul 26 18:11:56.836520 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.836496    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fb2kx\" (UniqueName: \"kubernetes.io/projected/5c253be6-2414-4ed1-b8b0-d0758f838ab9-kube-api-access-fb2kx\") pod \"multus-7z456\" (UID: \"5c253be6-2414-4ed1-b8b0-d0758f838ab9\") " pod="openshift-multus/multus-7z456"
Jul 26 18:11:56.836949 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.836929    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d94bk\" (UniqueName: \"kubernetes.io/projected/04f08429-551f-4765-8985-6dc6d0256dab-kube-api-access-d94bk\") pod \"iptables-alerter-6rvcp\" (UID: \"04f08429-551f-4765-8985-6dc6d0256dab\") " pod="openshift-network-operator/iptables-alerter-6rvcp"
Jul 26 18:11:56.848804 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.848714    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-7.ec2.internal" event={"ID":"4c3d4ae719d8eb92a6e5cea24a51f094","Type":"ContainerStarted","Data":"4b82f60c201536d7ce4acced6479cd872cede5b90068aba326fa5bfb82dccbe3"}
Jul 26 18:11:56.850010 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.849986    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-7.ec2.internal" event={"ID":"75dc32c0f7305fc42c8ffda6c7e0c513","Type":"ContainerStarted","Data":"bf3acaadeab5cd7d0cf66af4ddf0d4acc7f9b71925608eb4ea492d2bb96837da"}
Jul 26 18:11:56.917543 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.917459    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/63e8eeac-36f8-46a7-add2-5e222dac8d5f-os-release\") pod \"multus-additional-cni-plugins-rb2p7\" (UID: \"63e8eeac-36f8-46a7-add2-5e222dac8d5f\") " pod="openshift-multus/multus-additional-cni-plugins-rb2p7"
Jul 26 18:11:56.917543 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.917497    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqchl\" (UniqueName: \"kubernetes.io/projected/63e8eeac-36f8-46a7-add2-5e222dac8d5f-kube-api-access-kqchl\") pod \"multus-additional-cni-plugins-rb2p7\" (UID: \"63e8eeac-36f8-46a7-add2-5e222dac8d5f\") " pod="openshift-multus/multus-additional-cni-plugins-rb2p7"
Jul 26 18:11:56.917543 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.917514    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f7378af1-e3bd-48fa-a2d6-1b5db464dbb3-socket-dir\") pod \"aws-ebs-csi-driver-node-zfbtf\" (UID: \"f7378af1-e3bd-48fa-a2d6-1b5db464dbb3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zfbtf"
Jul 26 18:11:56.917543 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.917536    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f7378af1-e3bd-48fa-a2d6-1b5db464dbb3-device-dir\") pod \"aws-ebs-csi-driver-node-zfbtf\" (UID: \"f7378af1-e3bd-48fa-a2d6-1b5db464dbb3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zfbtf"
Jul 26 18:11:56.917974 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.917560    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f7378af1-e3bd-48fa-a2d6-1b5db464dbb3-sys-fs\") pod \"aws-ebs-csi-driver-node-zfbtf\" (UID: \"f7378af1-e3bd-48fa-a2d6-1b5db464dbb3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zfbtf"
Jul 26 18:11:56.917974 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.917586    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f7378af1-e3bd-48fa-a2d6-1b5db464dbb3-etc-selinux\") pod \"aws-ebs-csi-driver-node-zfbtf\" (UID: \"f7378af1-e3bd-48fa-a2d6-1b5db464dbb3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zfbtf"
Jul 26 18:11:56.917974 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.917610    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjp4l\" (UniqueName: \"kubernetes.io/projected/f7378af1-e3bd-48fa-a2d6-1b5db464dbb3-kube-api-access-qjp4l\") pod \"aws-ebs-csi-driver-node-zfbtf\" (UID: \"f7378af1-e3bd-48fa-a2d6-1b5db464dbb3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zfbtf"
Jul 26 18:11:56.917974 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.917638    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/63e8eeac-36f8-46a7-add2-5e222dac8d5f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rb2p7\" (UID: \"63e8eeac-36f8-46a7-add2-5e222dac8d5f\") " pod="openshift-multus/multus-additional-cni-plugins-rb2p7"
Jul 26 18:11:56.917974 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.917703    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f7378af1-e3bd-48fa-a2d6-1b5db464dbb3-socket-dir\") pod \"aws-ebs-csi-driver-node-zfbtf\" (UID: \"f7378af1-e3bd-48fa-a2d6-1b5db464dbb3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zfbtf"
Jul 26 18:11:56.917974 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.917708    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f7378af1-e3bd-48fa-a2d6-1b5db464dbb3-sys-fs\") pod \"aws-ebs-csi-driver-node-zfbtf\" (UID: \"f7378af1-e3bd-48fa-a2d6-1b5db464dbb3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zfbtf"
Jul 26 18:11:56.917974 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.917716    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f7378af1-e3bd-48fa-a2d6-1b5db464dbb3-device-dir\") pod \"aws-ebs-csi-driver-node-zfbtf\" (UID: \"f7378af1-e3bd-48fa-a2d6-1b5db464dbb3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zfbtf"
Jul 26 18:11:56.917974 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.917744    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/63e8eeac-36f8-46a7-add2-5e222dac8d5f-system-cni-dir\") pod \"multus-additional-cni-plugins-rb2p7\" (UID: \"63e8eeac-36f8-46a7-add2-5e222dac8d5f\") " pod="openshift-multus/multus-additional-cni-plugins-rb2p7"
Jul 26 18:11:56.917974 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.917801    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/63e8eeac-36f8-46a7-add2-5e222dac8d5f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rb2p7\" (UID: \"63e8eeac-36f8-46a7-add2-5e222dac8d5f\") " pod="openshift-multus/multus-additional-cni-plugins-rb2p7"
Jul 26 18:11:56.917974 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.917802    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/63e8eeac-36f8-46a7-add2-5e222dac8d5f-os-release\") pod \"multus-additional-cni-plugins-rb2p7\" (UID: \"63e8eeac-36f8-46a7-add2-5e222dac8d5f\") " pod="openshift-multus/multus-additional-cni-plugins-rb2p7"
Jul 26 18:11:56.917974 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.917861    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/63e8eeac-36f8-46a7-add2-5e222dac8d5f-system-cni-dir\") pod \"multus-additional-cni-plugins-rb2p7\" (UID: \"63e8eeac-36f8-46a7-add2-5e222dac8d5f\") " pod="openshift-multus/multus-additional-cni-plugins-rb2p7"
Jul 26 18:11:56.917974 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.917872    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f7378af1-e3bd-48fa-a2d6-1b5db464dbb3-etc-selinux\") pod \"aws-ebs-csi-driver-node-zfbtf\" (UID: \"f7378af1-e3bd-48fa-a2d6-1b5db464dbb3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zfbtf"
Jul 26 18:11:56.918437 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.917991    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/63e8eeac-36f8-46a7-add2-5e222dac8d5f-cnibin\") pod \"multus-additional-cni-plugins-rb2p7\" (UID: \"63e8eeac-36f8-46a7-add2-5e222dac8d5f\") " pod="openshift-multus/multus-additional-cni-plugins-rb2p7"
Jul 26 18:11:56.918437 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.918013    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/63e8eeac-36f8-46a7-add2-5e222dac8d5f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rb2p7\" (UID: \"63e8eeac-36f8-46a7-add2-5e222dac8d5f\") " pod="openshift-multus/multus-additional-cni-plugins-rb2p7"
Jul 26 18:11:56.918437 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.918030    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/63e8eeac-36f8-46a7-add2-5e222dac8d5f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rb2p7\" (UID: \"63e8eeac-36f8-46a7-add2-5e222dac8d5f\") " pod="openshift-multus/multus-additional-cni-plugins-rb2p7"
Jul 26 18:11:56.918437 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.918054    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f7378af1-e3bd-48fa-a2d6-1b5db464dbb3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zfbtf\" (UID: \"f7378af1-e3bd-48fa-a2d6-1b5db464dbb3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zfbtf"
Jul 26 18:11:56.918437 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.918060    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/63e8eeac-36f8-46a7-add2-5e222dac8d5f-cnibin\") pod \"multus-additional-cni-plugins-rb2p7\" (UID: \"63e8eeac-36f8-46a7-add2-5e222dac8d5f\") " pod="openshift-multus/multus-additional-cni-plugins-rb2p7"
Jul 26 18:11:56.918437 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.918077    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/63e8eeac-36f8-46a7-add2-5e222dac8d5f-cni-binary-copy\") pod \"multus-additional-cni-plugins-rb2p7\" (UID: \"63e8eeac-36f8-46a7-add2-5e222dac8d5f\") " pod="openshift-multus/multus-additional-cni-plugins-rb2p7"
Jul 26 18:11:56.918437 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.918115    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f7378af1-e3bd-48fa-a2d6-1b5db464dbb3-registration-dir\") pod \"aws-ebs-csi-driver-node-zfbtf\" (UID: \"f7378af1-e3bd-48fa-a2d6-1b5db464dbb3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zfbtf"
Jul 26 18:11:56.918437 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.918142    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f7378af1-e3bd-48fa-a2d6-1b5db464dbb3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zfbtf\" (UID: \"f7378af1-e3bd-48fa-a2d6-1b5db464dbb3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zfbtf"
Jul 26 18:11:56.918437 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.918196    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f7378af1-e3bd-48fa-a2d6-1b5db464dbb3-registration-dir\") pod \"aws-ebs-csi-driver-node-zfbtf\" (UID: \"f7378af1-e3bd-48fa-a2d6-1b5db464dbb3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zfbtf"
Jul 26 18:11:56.918437 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.918267    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/63e8eeac-36f8-46a7-add2-5e222dac8d5f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rb2p7\" (UID: \"63e8eeac-36f8-46a7-add2-5e222dac8d5f\") " pod="openshift-multus/multus-additional-cni-plugins-rb2p7"
Jul 26 18:11:56.918785 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.918500    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/63e8eeac-36f8-46a7-add2-5e222dac8d5f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rb2p7\" (UID: \"63e8eeac-36f8-46a7-add2-5e222dac8d5f\") " pod="openshift-multus/multus-additional-cni-plugins-rb2p7"
Jul 26 18:11:56.918785 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.918622    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/63e8eeac-36f8-46a7-add2-5e222dac8d5f-cni-binary-copy\") pod \"multus-additional-cni-plugins-rb2p7\" (UID: \"63e8eeac-36f8-46a7-add2-5e222dac8d5f\") " pod="openshift-multus/multus-additional-cni-plugins-rb2p7"
Jul 26 18:11:56.927506 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.927478    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjp4l\" (UniqueName: \"kubernetes.io/projected/f7378af1-e3bd-48fa-a2d6-1b5db464dbb3-kube-api-access-qjp4l\") pod \"aws-ebs-csi-driver-node-zfbtf\" (UID: \"f7378af1-e3bd-48fa-a2d6-1b5db464dbb3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zfbtf"
Jul 26 18:11:56.927701 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:56.927678    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqchl\" (UniqueName: \"kubernetes.io/projected/63e8eeac-36f8-46a7-add2-5e222dac8d5f-kube-api-access-kqchl\") pod \"multus-additional-cni-plugins-rb2p7\" (UID: \"63e8eeac-36f8-46a7-add2-5e222dac8d5f\") " pod="openshift-multus/multus-additional-cni-plugins-rb2p7"
Jul 26 18:11:57.001825 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:57.001787    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7z456"
Jul 26 18:11:57.011683 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:57.011658    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hz9mx"
Jul 26 18:11:57.025908 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:57.025879    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6vnjh"
Jul 26 18:11:57.031847 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:57.031819    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hp9vz"
Jul 26 18:11:57.043625 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:57.043594    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-k9khs"
Jul 26 18:11:57.051433 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:57.051406    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6rvcp"
Jul 26 18:11:57.059497 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:57.059473    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:11:57.066374 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:57.066340    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rb2p7"
Jul 26 18:11:57.073156 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:57.073127    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zfbtf"
Jul 26 18:11:57.321443 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:57.321346    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/54acf4cd-cd75-4990-9ccf-274ef789daa5-original-pull-secret\") pod \"global-pull-secret-syncer-fsm57\" (UID: \"54acf4cd-cd75-4990-9ccf-274ef789daa5\") " pod="kube-system/global-pull-secret-syncer-fsm57"
Jul 26 18:11:57.321443 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:57.321430    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97230a04-09c6-4445-b713-98d115f6b1aa-metrics-certs\") pod \"network-metrics-daemon-fxp88\" (UID: \"97230a04-09c6-4445-b713-98d115f6b1aa\") " pod="openshift-multus/network-metrics-daemon-fxp88"
Jul 26 18:11:57.321670 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:57.321532    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 18:11:57.321670 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:57.321544    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 18:11:57.321670 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:57.321628    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54acf4cd-cd75-4990-9ccf-274ef789daa5-original-pull-secret podName:54acf4cd-cd75-4990-9ccf-274ef789daa5 nodeName:}" failed. No retries permitted until 2026-07-26 18:11:58.321587146 +0000 UTC m=+4.028994187 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/54acf4cd-cd75-4990-9ccf-274ef789daa5-original-pull-secret") pod "global-pull-secret-syncer-fsm57" (UID: "54acf4cd-cd75-4990-9ccf-274ef789daa5") : object "kube-system"/"original-pull-secret" not registered
Jul 26 18:11:57.321670 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:57.321659    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97230a04-09c6-4445-b713-98d115f6b1aa-metrics-certs podName:97230a04-09c6-4445-b713-98d115f6b1aa nodeName:}" failed. No retries permitted until 2026-07-26 18:11:58.321648498 +0000 UTC m=+4.029055536 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97230a04-09c6-4445-b713-98d115f6b1aa-metrics-certs") pod "network-metrics-daemon-fxp88" (UID: "97230a04-09c6-4445-b713-98d115f6b1aa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 18:11:57.422726 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:57.422684    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qzgk\" (UniqueName: \"kubernetes.io/projected/8a2c5810-fcc5-4224-8d19-5fe5cc832d5a-kube-api-access-7qzgk\") pod \"network-check-target-rrn49\" (UID: \"8a2c5810-fcc5-4224-8d19-5fe5cc832d5a\") " pod="openshift-network-diagnostics/network-check-target-rrn49"
Jul 26 18:11:57.422931 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:57.422911    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 18:11:57.422978 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:57.422935    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 18:11:57.422978 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:57.422947    2576 projected.go:196] Error preparing data for projected volume kube-api-access-7qzgk for pod openshift-network-diagnostics/network-check-target-rrn49: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 18:11:57.423077 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:57.423008    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a2c5810-fcc5-4224-8d19-5fe5cc832d5a-kube-api-access-7qzgk podName:8a2c5810-fcc5-4224-8d19-5fe5cc832d5a nodeName:}" failed. No retries permitted until 2026-07-26 18:11:58.422991313 +0000 UTC m=+4.130398351 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-7qzgk" (UniqueName: "kubernetes.io/projected/8a2c5810-fcc5-4224-8d19-5fe5cc832d5a-kube-api-access-7qzgk") pod "network-check-target-rrn49" (UID: "8a2c5810-fcc5-4224-8d19-5fe5cc832d5a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 18:11:57.755339 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:57.755232    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-25 18:06:57 +0000 UTC" deadline="2028-02-23 08:25:52.057302049 +0000 UTC"
Jul 26 18:11:57.755339 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:57.755275    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13838h13m54.302030577s"
Jul 26 18:11:57.841065 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:57.841034    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63e8eeac_36f8_46a7_add2_5e222dac8d5f.slice/crio-ec6fc741e134e1b4d5ccab74b756b934beb22e2359b9600b14c87a3a87929b2d WatchSource:0}: Error finding container ec6fc741e134e1b4d5ccab74b756b934beb22e2359b9600b14c87a3a87929b2d: Status 404 returned error can't find the container with id ec6fc741e134e1b4d5ccab74b756b934beb22e2359b9600b14c87a3a87929b2d
Jul 26 18:11:57.842322 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:57.842295    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04f08429_551f_4765_8985_6dc6d0256dab.slice/crio-99f0552c1ea59a267bb1d18dcb0f9f9cc59f0ef5a3207685e2f18345d6409456 WatchSource:0}: Error finding container 99f0552c1ea59a267bb1d18dcb0f9f9cc59f0ef5a3207685e2f18345d6409456: Status 404 returned error can't find the container with id 99f0552c1ea59a267bb1d18dcb0f9f9cc59f0ef5a3207685e2f18345d6409456
Jul 26 18:11:57.843139 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:57.843114    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fsm57"
Jul 26 18:11:57.843264 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:57.843239    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fsm57" podUID="54acf4cd-cd75-4990-9ccf-274ef789daa5"
Jul 26 18:11:57.843696 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:57.843670    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e895b2b_026e_4228_9fc7_fbe90089f5a9.slice/crio-972235ae9fdbdaa44b0e06d0ccbe9949e76d9758b9572ca51d63dd3685fa2ca4 WatchSource:0}: Error finding container 972235ae9fdbdaa44b0e06d0ccbe9949e76d9758b9572ca51d63dd3685fa2ca4: Status 404 returned error can't find the container with id 972235ae9fdbdaa44b0e06d0ccbe9949e76d9758b9572ca51d63dd3685fa2ca4
Jul 26 18:11:57.844651 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:57.844595    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94b1d70f_c671_4fe3_bc2c_05b9426a496f.slice/crio-9d41f9660e8f118ed7c76f246ced2f24410436cbb55dc15e43be24938391d17d WatchSource:0}: Error finding container 9d41f9660e8f118ed7c76f246ced2f24410436cbb55dc15e43be24938391d17d: Status 404 returned error can't find the container with id 9d41f9660e8f118ed7c76f246ced2f24410436cbb55dc15e43be24938391d17d
Jul 26 18:11:57.847899 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:57.847865    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65820b5a_b897_45fc_ae26_e286b5444375.slice/crio-4e7ad28cc8802530fbf475ebca2f249e2bf98f27e8219ab3f2106010060bd06e WatchSource:0}: Error finding container 4e7ad28cc8802530fbf475ebca2f249e2bf98f27e8219ab3f2106010060bd06e: Status 404 returned error can't find the container with id 4e7ad28cc8802530fbf475ebca2f249e2bf98f27e8219ab3f2106010060bd06e
Jul 26 18:11:57.849138 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:57.849114    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49e6be1b_9cd1_4a86_a30b_1c414e573659.slice/crio-90e96d548ea1df174cf4336b739f0b03da8d9113f381a67a894c90aaad4b5e58 WatchSource:0}: Error finding container 90e96d548ea1df174cf4336b739f0b03da8d9113f381a67a894c90aaad4b5e58: Status 404 returned error can't find the container with id 90e96d548ea1df174cf4336b739f0b03da8d9113f381a67a894c90aaad4b5e58
Jul 26 18:11:57.855491 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:57.855468    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7378af1_e3bd_48fa_a2d6_1b5db464dbb3.slice/crio-0b9a985ab9aebfd5bada276408958b3aba10cdf14e8b335e28c9a1bbf448c74c WatchSource:0}: Error finding container 0b9a985ab9aebfd5bada276408958b3aba10cdf14e8b335e28c9a1bbf448c74c: Status 404 returned error can't find the container with id 0b9a985ab9aebfd5bada276408958b3aba10cdf14e8b335e28c9a1bbf448c74c
Jul 26 18:11:57.856770 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:57.856748    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod261aba58_ddd5_4f9e_af39_7475dec519f9.slice/crio-9058ef6071c993dd88c9300cc1ddf5ff63e516420f82745f0cca98d6be56f5aa WatchSource:0}: Error finding container 9058ef6071c993dd88c9300cc1ddf5ff63e516420f82745f0cca98d6be56f5aa: Status 404 returned error can't find the container with id 9058ef6071c993dd88c9300cc1ddf5ff63e516420f82745f0cca98d6be56f5aa
Jul 26 18:11:57.859733 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:11:57.858707    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c253be6_2414_4ed1_b8b0_d0758f838ab9.slice/crio-72a692e3ff2442a26a1d32bb43e9996005b23ffd6c2eeb9e6203dd64c7442db8 WatchSource:0}: Error finding container 72a692e3ff2442a26a1d32bb43e9996005b23ffd6c2eeb9e6203dd64c7442db8: Status 404 returned error can't find the container with id 72a692e3ff2442a26a1d32bb43e9996005b23ffd6c2eeb9e6203dd64c7442db8
Jul 26 18:11:57.860610 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:57.860063    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hp9vz" event={"ID":"49e6be1b-9cd1-4a86-a30b-1c414e573659","Type":"ContainerStarted","Data":"90e96d548ea1df174cf4336b739f0b03da8d9113f381a67a894c90aaad4b5e58"}
Jul 26 18:11:57.863418 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:57.863387    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6vnjh" event={"ID":"65820b5a-b897-45fc-ae26-e286b5444375","Type":"ContainerStarted","Data":"4e7ad28cc8802530fbf475ebca2f249e2bf98f27e8219ab3f2106010060bd06e"}
Jul 26 18:11:57.865981 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:57.865921    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6rvcp" event={"ID":"04f08429-551f-4765-8985-6dc6d0256dab","Type":"ContainerStarted","Data":"99f0552c1ea59a267bb1d18dcb0f9f9cc59f0ef5a3207685e2f18345d6409456"}
Jul 26 18:11:57.867505 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:57.867478    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw" event={"ID":"94b1d70f-c671-4fe3-bc2c-05b9426a496f","Type":"ContainerStarted","Data":"9d41f9660e8f118ed7c76f246ced2f24410436cbb55dc15e43be24938391d17d"}
Jul 26 18:11:57.869062 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:57.869031    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hz9mx" event={"ID":"7e895b2b-026e-4228-9fc7-fbe90089f5a9","Type":"ContainerStarted","Data":"972235ae9fdbdaa44b0e06d0ccbe9949e76d9758b9572ca51d63dd3685fa2ca4"}
Jul 26 18:11:57.870576 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:57.870549    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rb2p7" event={"ID":"63e8eeac-36f8-46a7-add2-5e222dac8d5f","Type":"ContainerStarted","Data":"ec6fc741e134e1b4d5ccab74b756b934beb22e2359b9600b14c87a3a87929b2d"}
Jul 26 18:11:58.330182 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:58.329326    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97230a04-09c6-4445-b713-98d115f6b1aa-metrics-certs\") pod \"network-metrics-daemon-fxp88\" (UID: \"97230a04-09c6-4445-b713-98d115f6b1aa\") " pod="openshift-multus/network-metrics-daemon-fxp88"
Jul 26 18:11:58.330182 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:58.329616    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 18:11:58.330182 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:58.329667    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/54acf4cd-cd75-4990-9ccf-274ef789daa5-original-pull-secret\") pod \"global-pull-secret-syncer-fsm57\" (UID: \"54acf4cd-cd75-4990-9ccf-274ef789daa5\") " pod="kube-system/global-pull-secret-syncer-fsm57"
Jul 26 18:11:58.330182 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:58.329696    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97230a04-09c6-4445-b713-98d115f6b1aa-metrics-certs podName:97230a04-09c6-4445-b713-98d115f6b1aa nodeName:}" failed. No retries permitted until 2026-07-26 18:12:00.329672977 +0000 UTC m=+6.037080020 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97230a04-09c6-4445-b713-98d115f6b1aa-metrics-certs") pod "network-metrics-daemon-fxp88" (UID: "97230a04-09c6-4445-b713-98d115f6b1aa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 18:11:58.330182 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:58.329766    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 18:11:58.330182 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:58.329834    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54acf4cd-cd75-4990-9ccf-274ef789daa5-original-pull-secret podName:54acf4cd-cd75-4990-9ccf-274ef789daa5 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:00.329818214 +0000 UTC m=+6.037225269 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/54acf4cd-cd75-4990-9ccf-274ef789daa5-original-pull-secret") pod "global-pull-secret-syncer-fsm57" (UID: "54acf4cd-cd75-4990-9ccf-274ef789daa5") : object "kube-system"/"original-pull-secret" not registered
Jul 26 18:11:58.430635 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:58.430591    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qzgk\" (UniqueName: \"kubernetes.io/projected/8a2c5810-fcc5-4224-8d19-5fe5cc832d5a-kube-api-access-7qzgk\") pod \"network-check-target-rrn49\" (UID: \"8a2c5810-fcc5-4224-8d19-5fe5cc832d5a\") " pod="openshift-network-diagnostics/network-check-target-rrn49"
Jul 26 18:11:58.430846 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:58.430812    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 18:11:58.430846 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:58.430847    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 18:11:58.431002 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:58.430861    2576 projected.go:196] Error preparing data for projected volume kube-api-access-7qzgk for pod openshift-network-diagnostics/network-check-target-rrn49: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 18:11:58.431002 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:58.430924    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a2c5810-fcc5-4224-8d19-5fe5cc832d5a-kube-api-access-7qzgk podName:8a2c5810-fcc5-4224-8d19-5fe5cc832d5a nodeName:}" failed. No retries permitted until 2026-07-26 18:12:00.430904125 +0000 UTC m=+6.138311170 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-7qzgk" (UniqueName: "kubernetes.io/projected/8a2c5810-fcc5-4224-8d19-5fe5cc832d5a-kube-api-access-7qzgk") pod "network-check-target-rrn49" (UID: "8a2c5810-fcc5-4224-8d19-5fe5cc832d5a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 18:11:58.759907 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:58.755953    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-25 18:06:57 +0000 UTC" deadline="2028-02-07 02:54:40.046559583 +0000 UTC"
Jul 26 18:11:58.759907 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:58.756023    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13448h42m41.290541432s"
Jul 26 18:11:58.844698 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:58.844019    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxp88"
Jul 26 18:11:58.844698 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:58.844169    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxp88" podUID="97230a04-09c6-4445-b713-98d115f6b1aa"
Jul 26 18:11:58.844698 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:58.844264    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rrn49"
Jul 26 18:11:58.844698 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:58.844343    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rrn49" podUID="8a2c5810-fcc5-4224-8d19-5fe5cc832d5a"
Jul 26 18:11:58.891807 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:58.891722    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-k9khs" event={"ID":"261aba58-ddd5-4f9e-af39-7475dec519f9","Type":"ContainerStarted","Data":"9058ef6071c993dd88c9300cc1ddf5ff63e516420f82745f0cca98d6be56f5aa"}
Jul 26 18:11:58.899720 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:58.899690    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zfbtf" event={"ID":"f7378af1-e3bd-48fa-a2d6-1b5db464dbb3","Type":"ContainerStarted","Data":"0b9a985ab9aebfd5bada276408958b3aba10cdf14e8b335e28c9a1bbf448c74c"}
Jul 26 18:11:58.937176 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:58.937131    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-7.ec2.internal" event={"ID":"4c3d4ae719d8eb92a6e5cea24a51f094","Type":"ContainerStarted","Data":"847ca99c268cf0e0dd89c4ecc0c6be8c3ac811f97f1113c64da4329ab538facb"}
Jul 26 18:11:58.954098 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:58.952856    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7z456" event={"ID":"5c253be6-2414-4ed1-b8b0-d0758f838ab9","Type":"ContainerStarted","Data":"72a692e3ff2442a26a1d32bb43e9996005b23ffd6c2eeb9e6203dd64c7442db8"}
Jul 26 18:11:58.960346 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:58.960305    2576 generic.go:356] "Generic (PLEG): container finished" podID="75dc32c0f7305fc42c8ffda6c7e0c513" containerID="d5027f1fe440dd220641d56adaae5541f4365b257e3d2ef5cf669686d7030a8f" exitCode=0
Jul 26 18:11:58.960346 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:58.960341    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-7.ec2.internal" event={"ID":"75dc32c0f7305fc42c8ffda6c7e0c513","Type":"ContainerDied","Data":"d5027f1fe440dd220641d56adaae5541f4365b257e3d2ef5cf669686d7030a8f"}
Jul 26 18:11:58.994366 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:58.994311    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-7.ec2.internal" podStartSLOduration=2.994289601 podStartE2EDuration="2.994289601s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:11:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:11:58.964832481 +0000 UTC m=+4.672239535" watchObservedRunningTime="2026-07-26 18:11:58.994289601 +0000 UTC m=+4.701696666"
Jul 26 18:11:59.843948 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:59.843378    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fsm57"
Jul 26 18:11:59.843948 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:11:59.843518    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fsm57" podUID="54acf4cd-cd75-4990-9ccf-274ef789daa5"
Jul 26 18:11:59.977592 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:11:59.977504    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-7.ec2.internal" event={"ID":"75dc32c0f7305fc42c8ffda6c7e0c513","Type":"ContainerStarted","Data":"8aca7950c8ab44d4e6f7ae899b7cdde8576331f42171300ddacf41f0980dc5a3"}
Jul 26 18:12:00.348719 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:00.347986    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/54acf4cd-cd75-4990-9ccf-274ef789daa5-original-pull-secret\") pod \"global-pull-secret-syncer-fsm57\" (UID: \"54acf4cd-cd75-4990-9ccf-274ef789daa5\") " pod="kube-system/global-pull-secret-syncer-fsm57"
Jul 26 18:12:00.348719 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:00.348080    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97230a04-09c6-4445-b713-98d115f6b1aa-metrics-certs\") pod \"network-metrics-daemon-fxp88\" (UID: \"97230a04-09c6-4445-b713-98d115f6b1aa\") " pod="openshift-multus/network-metrics-daemon-fxp88"
Jul 26 18:12:00.348719 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:00.348240    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 18:12:00.348719 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:00.348461    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97230a04-09c6-4445-b713-98d115f6b1aa-metrics-certs podName:97230a04-09c6-4445-b713-98d115f6b1aa nodeName:}" failed. No retries permitted until 2026-07-26 18:12:04.348431997 +0000 UTC m=+10.055839051 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97230a04-09c6-4445-b713-98d115f6b1aa-metrics-certs") pod "network-metrics-daemon-fxp88" (UID: "97230a04-09c6-4445-b713-98d115f6b1aa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 18:12:00.349143 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:00.348938    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 18:12:00.349143 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:00.348996    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54acf4cd-cd75-4990-9ccf-274ef789daa5-original-pull-secret podName:54acf4cd-cd75-4990-9ccf-274ef789daa5 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:04.34897879 +0000 UTC m=+10.056385834 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/54acf4cd-cd75-4990-9ccf-274ef789daa5-original-pull-secret") pod "global-pull-secret-syncer-fsm57" (UID: "54acf4cd-cd75-4990-9ccf-274ef789daa5") : object "kube-system"/"original-pull-secret" not registered
Jul 26 18:12:00.448693 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:00.448652    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qzgk\" (UniqueName: \"kubernetes.io/projected/8a2c5810-fcc5-4224-8d19-5fe5cc832d5a-kube-api-access-7qzgk\") pod \"network-check-target-rrn49\" (UID: \"8a2c5810-fcc5-4224-8d19-5fe5cc832d5a\") " pod="openshift-network-diagnostics/network-check-target-rrn49"
Jul 26 18:12:00.448902 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:00.448877    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 18:12:00.448902 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:00.448896    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 18:12:00.448902 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:00.448905    2576 projected.go:196] Error preparing data for projected volume kube-api-access-7qzgk for pod openshift-network-diagnostics/network-check-target-rrn49: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 18:12:00.449078 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:00.448952    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a2c5810-fcc5-4224-8d19-5fe5cc832d5a-kube-api-access-7qzgk podName:8a2c5810-fcc5-4224-8d19-5fe5cc832d5a nodeName:}" failed. No retries permitted until 2026-07-26 18:12:04.448937666 +0000 UTC m=+10.156344704 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-7qzgk" (UniqueName: "kubernetes.io/projected/8a2c5810-fcc5-4224-8d19-5fe5cc832d5a-kube-api-access-7qzgk") pod "network-check-target-rrn49" (UID: "8a2c5810-fcc5-4224-8d19-5fe5cc832d5a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 18:12:00.845003 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:00.844915    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxp88"
Jul 26 18:12:00.845452 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:00.845067    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxp88" podUID="97230a04-09c6-4445-b713-98d115f6b1aa"
Jul 26 18:12:00.845452 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:00.844925    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rrn49"
Jul 26 18:12:00.845452 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:00.845338    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rrn49" podUID="8a2c5810-fcc5-4224-8d19-5fe5cc832d5a"
Jul 26 18:12:01.843831 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:01.843788    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fsm57"
Jul 26 18:12:01.844040 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:01.843947    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fsm57" podUID="54acf4cd-cd75-4990-9ccf-274ef789daa5"
Jul 26 18:12:02.844193 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:02.844146    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rrn49"
Jul 26 18:12:02.844695 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:02.844234    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxp88"
Jul 26 18:12:02.844695 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:02.844401    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxp88" podUID="97230a04-09c6-4445-b713-98d115f6b1aa"
Jul 26 18:12:02.844695 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:02.844536    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rrn49" podUID="8a2c5810-fcc5-4224-8d19-5fe5cc832d5a"
Jul 26 18:12:03.844086 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:03.844000    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fsm57"
Jul 26 18:12:03.844275 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:03.844167    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fsm57" podUID="54acf4cd-cd75-4990-9ccf-274ef789daa5"
Jul 26 18:12:04.382347 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:04.382084    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/54acf4cd-cd75-4990-9ccf-274ef789daa5-original-pull-secret\") pod \"global-pull-secret-syncer-fsm57\" (UID: \"54acf4cd-cd75-4990-9ccf-274ef789daa5\") " pod="kube-system/global-pull-secret-syncer-fsm57"
Jul 26 18:12:04.382347 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:04.382180    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97230a04-09c6-4445-b713-98d115f6b1aa-metrics-certs\") pod \"network-metrics-daemon-fxp88\" (UID: \"97230a04-09c6-4445-b713-98d115f6b1aa\") " pod="openshift-multus/network-metrics-daemon-fxp88"
Jul 26 18:12:04.382347 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:04.382224    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 18:12:04.382347 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:04.382288    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 18:12:04.382347 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:04.382294    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54acf4cd-cd75-4990-9ccf-274ef789daa5-original-pull-secret podName:54acf4cd-cd75-4990-9ccf-274ef789daa5 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:12.382275419 +0000 UTC m=+18.089682456 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/54acf4cd-cd75-4990-9ccf-274ef789daa5-original-pull-secret") pod "global-pull-secret-syncer-fsm57" (UID: "54acf4cd-cd75-4990-9ccf-274ef789daa5") : object "kube-system"/"original-pull-secret" not registered
Jul 26 18:12:04.382347 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:04.382323    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97230a04-09c6-4445-b713-98d115f6b1aa-metrics-certs podName:97230a04-09c6-4445-b713-98d115f6b1aa nodeName:}" failed. No retries permitted until 2026-07-26 18:12:12.382311994 +0000 UTC m=+18.089719037 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97230a04-09c6-4445-b713-98d115f6b1aa-metrics-certs") pod "network-metrics-daemon-fxp88" (UID: "97230a04-09c6-4445-b713-98d115f6b1aa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 18:12:04.483535 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:04.483498    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qzgk\" (UniqueName: \"kubernetes.io/projected/8a2c5810-fcc5-4224-8d19-5fe5cc832d5a-kube-api-access-7qzgk\") pod \"network-check-target-rrn49\" (UID: \"8a2c5810-fcc5-4224-8d19-5fe5cc832d5a\") " pod="openshift-network-diagnostics/network-check-target-rrn49"
Jul 26 18:12:04.483705 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:04.483645    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 18:12:04.483705 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:04.483661    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 18:12:04.483705 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:04.483670    2576 projected.go:196] Error preparing data for projected volume kube-api-access-7qzgk for pod openshift-network-diagnostics/network-check-target-rrn49: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 18:12:04.483883 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:04.483726    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a2c5810-fcc5-4224-8d19-5fe5cc832d5a-kube-api-access-7qzgk podName:8a2c5810-fcc5-4224-8d19-5fe5cc832d5a nodeName:}" failed. No retries permitted until 2026-07-26 18:12:12.483709592 +0000 UTC m=+18.191116650 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-7qzgk" (UniqueName: "kubernetes.io/projected/8a2c5810-fcc5-4224-8d19-5fe5cc832d5a-kube-api-access-7qzgk") pod "network-check-target-rrn49" (UID: "8a2c5810-fcc5-4224-8d19-5fe5cc832d5a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 18:12:04.844321 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:04.844228    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxp88"
Jul 26 18:12:04.844799 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:04.844332    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxp88" podUID="97230a04-09c6-4445-b713-98d115f6b1aa"
Jul 26 18:12:04.844799 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:04.844645    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rrn49"
Jul 26 18:12:04.844799 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:04.844711    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rrn49" podUID="8a2c5810-fcc5-4224-8d19-5fe5cc832d5a"
Jul 26 18:12:05.843326 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:05.843219    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fsm57"
Jul 26 18:12:05.843501 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:05.843425    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fsm57" podUID="54acf4cd-cd75-4990-9ccf-274ef789daa5"
Jul 26 18:12:06.845900 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:06.845616    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxp88"
Jul 26 18:12:06.845900 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:06.845765    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxp88" podUID="97230a04-09c6-4445-b713-98d115f6b1aa"
Jul 26 18:12:06.845900 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:06.845833    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rrn49"
Jul 26 18:12:06.846538 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:06.846503    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rrn49" podUID="8a2c5810-fcc5-4224-8d19-5fe5cc832d5a"
Jul 26 18:12:07.843667 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:07.843628    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fsm57"
Jul 26 18:12:07.843842 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:07.843784    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fsm57" podUID="54acf4cd-cd75-4990-9ccf-274ef789daa5"
Jul 26 18:12:08.845902 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:08.845876    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rrn49"
Jul 26 18:12:08.846280 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:08.845878    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxp88"
Jul 26 18:12:08.846280 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:08.845968    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rrn49" podUID="8a2c5810-fcc5-4224-8d19-5fe5cc832d5a"
Jul 26 18:12:08.846280 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:08.846041    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxp88" podUID="97230a04-09c6-4445-b713-98d115f6b1aa"
Jul 26 18:12:09.853702 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:09.853669    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fsm57"
Jul 26 18:12:09.854153 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:09.853789    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fsm57" podUID="54acf4cd-cd75-4990-9ccf-274ef789daa5"
Jul 26 18:12:10.843411 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:10.843375    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rrn49"
Jul 26 18:12:10.843592 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:10.843508    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rrn49" podUID="8a2c5810-fcc5-4224-8d19-5fe5cc832d5a"
Jul 26 18:12:10.843978 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:10.843956    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxp88"
Jul 26 18:12:10.844089 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:10.844048    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxp88" podUID="97230a04-09c6-4445-b713-98d115f6b1aa"
Jul 26 18:12:11.843554 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:11.843518    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fsm57"
Jul 26 18:12:11.844082 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:11.843655    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fsm57" podUID="54acf4cd-cd75-4990-9ccf-274ef789daa5"
Jul 26 18:12:12.447728 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:12.447694    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/54acf4cd-cd75-4990-9ccf-274ef789daa5-original-pull-secret\") pod \"global-pull-secret-syncer-fsm57\" (UID: \"54acf4cd-cd75-4990-9ccf-274ef789daa5\") " pod="kube-system/global-pull-secret-syncer-fsm57"
Jul 26 18:12:12.447913 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:12.447780    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97230a04-09c6-4445-b713-98d115f6b1aa-metrics-certs\") pod \"network-metrics-daemon-fxp88\" (UID: \"97230a04-09c6-4445-b713-98d115f6b1aa\") " pod="openshift-multus/network-metrics-daemon-fxp88"
Jul 26 18:12:12.447913 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:12.447860    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 18:12:12.447913 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:12.447877    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 18:12:12.448073 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:12.447931    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54acf4cd-cd75-4990-9ccf-274ef789daa5-original-pull-secret podName:54acf4cd-cd75-4990-9ccf-274ef789daa5 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:28.44790942 +0000 UTC m=+34.155316460 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/54acf4cd-cd75-4990-9ccf-274ef789daa5-original-pull-secret") pod "global-pull-secret-syncer-fsm57" (UID: "54acf4cd-cd75-4990-9ccf-274ef789daa5") : object "kube-system"/"original-pull-secret" not registered
Jul 26 18:12:12.448073 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:12.447951    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97230a04-09c6-4445-b713-98d115f6b1aa-metrics-certs podName:97230a04-09c6-4445-b713-98d115f6b1aa nodeName:}" failed. No retries permitted until 2026-07-26 18:12:28.44794072 +0000 UTC m=+34.155347757 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97230a04-09c6-4445-b713-98d115f6b1aa-metrics-certs") pod "network-metrics-daemon-fxp88" (UID: "97230a04-09c6-4445-b713-98d115f6b1aa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 18:12:12.548223 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:12.548188    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qzgk\" (UniqueName: \"kubernetes.io/projected/8a2c5810-fcc5-4224-8d19-5fe5cc832d5a-kube-api-access-7qzgk\") pod \"network-check-target-rrn49\" (UID: \"8a2c5810-fcc5-4224-8d19-5fe5cc832d5a\") " pod="openshift-network-diagnostics/network-check-target-rrn49"
Jul 26 18:12:12.548382 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:12.548338    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 18:12:12.548382 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:12.548363    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 18:12:12.548382 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:12.548378    2576 projected.go:196] Error preparing data for projected volume kube-api-access-7qzgk for pod openshift-network-diagnostics/network-check-target-rrn49: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 18:12:12.548568 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:12.548431    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a2c5810-fcc5-4224-8d19-5fe5cc832d5a-kube-api-access-7qzgk podName:8a2c5810-fcc5-4224-8d19-5fe5cc832d5a nodeName:}" failed. No retries permitted until 2026-07-26 18:12:28.548417298 +0000 UTC m=+34.255824335 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-7qzgk" (UniqueName: "kubernetes.io/projected/8a2c5810-fcc5-4224-8d19-5fe5cc832d5a-kube-api-access-7qzgk") pod "network-check-target-rrn49" (UID: "8a2c5810-fcc5-4224-8d19-5fe5cc832d5a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 18:12:12.843881 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:12.843786    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rrn49"
Jul 26 18:12:12.844299 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:12.843928    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rrn49" podUID="8a2c5810-fcc5-4224-8d19-5fe5cc832d5a"
Jul 26 18:12:12.844299 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:12.843970    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxp88"
Jul 26 18:12:12.844299 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:12.844106    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxp88" podUID="97230a04-09c6-4445-b713-98d115f6b1aa"
Jul 26 18:12:13.843003 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:13.842966    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fsm57"
Jul 26 18:12:13.843202 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:13.843095    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fsm57" podUID="54acf4cd-cd75-4990-9ccf-274ef789daa5"
Jul 26 18:12:14.844314 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:14.844280    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rrn49"
Jul 26 18:12:14.844817 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:14.844383    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rrn49" podUID="8a2c5810-fcc5-4224-8d19-5fe5cc832d5a"
Jul 26 18:12:14.844817 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:14.844431    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxp88"
Jul 26 18:12:14.844817 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:14.844553    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxp88" podUID="97230a04-09c6-4445-b713-98d115f6b1aa"
Jul 26 18:12:15.843933 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:15.843892    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fsm57"
Jul 26 18:12:15.844116 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:15.844030    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fsm57" podUID="54acf4cd-cd75-4990-9ccf-274ef789daa5"
Jul 26 18:12:16.844213 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:16.843576    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rrn49"
Jul 26 18:12:16.844213 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:16.843903    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rrn49" podUID="8a2c5810-fcc5-4224-8d19-5fe5cc832d5a"
Jul 26 18:12:16.844213 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:16.843973    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxp88"
Jul 26 18:12:16.844213 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:16.844050    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxp88" podUID="97230a04-09c6-4445-b713-98d115f6b1aa"
Jul 26 18:12:17.013766 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:17.013508    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hp9vz" event={"ID":"49e6be1b-9cd1-4a86-a30b-1c414e573659","Type":"ContainerStarted","Data":"241f3f77789c8def452b5c1f1117052b8a0037c8c5cc13f79bb5c53cbdb41711"}
Jul 26 18:12:17.014993 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:17.014963    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6vnjh" event={"ID":"65820b5a-b897-45fc-ae26-e286b5444375","Type":"ContainerStarted","Data":"334cbae4e84803982d175b89b2b7f3ce5416f836a88e3de2dc062c5c35f9dd19"}
Jul 26 18:12:17.016676 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:17.016650    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw" event={"ID":"94b1d70f-c671-4fe3-bc2c-05b9426a496f","Type":"ContainerStarted","Data":"0c818418e9206df2e743e144459d5678bf193ae0325931781f6df3d012c3fd10"}
Jul 26 18:12:17.016798 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:17.016721    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw" event={"ID":"94b1d70f-c671-4fe3-bc2c-05b9426a496f","Type":"ContainerStarted","Data":"d1229a2424173d458c4e6c8798a1dd63378d5c6cb5803643440ecf5cf1aaaf65"}
Jul 26 18:12:17.018053 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:17.018023    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7z456" event={"ID":"5c253be6-2414-4ed1-b8b0-d0758f838ab9","Type":"ContainerStarted","Data":"fdfacec80135a26ca9d44c95c5d232207b06716669d9e87a8f2ba97bef9efdb6"}
Jul 26 18:12:17.019485 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:17.019461    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hz9mx" event={"ID":"7e895b2b-026e-4228-9fc7-fbe90089f5a9","Type":"ContainerStarted","Data":"67fe94c8943a7f5863461471d7caf193f46b86e5d267dd4b89be99f5444c365f"}
Jul 26 18:12:17.020974 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:17.020951    2576 generic.go:356] "Generic (PLEG): container finished" podID="63e8eeac-36f8-46a7-add2-5e222dac8d5f" containerID="9fafb77e16252ef901588ab617dc712396ddb1637bc0537588ce0ec959b7c379" exitCode=0
Jul 26 18:12:17.021087 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:17.021015    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rb2p7" event={"ID":"63e8eeac-36f8-46a7-add2-5e222dac8d5f","Type":"ContainerDied","Data":"9fafb77e16252ef901588ab617dc712396ddb1637bc0537588ce0ec959b7c379"}
Jul 26 18:12:17.022648 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:17.022596    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-k9khs" event={"ID":"261aba58-ddd5-4f9e-af39-7475dec519f9","Type":"ContainerStarted","Data":"6d105b7203b24e9acaa4f5fbddd6514dbfbf95a39cbd4a6ee5b9dddcdfdecda4"}
Jul 26 18:12:17.024023 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:17.024005    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zfbtf" event={"ID":"f7378af1-e3bd-48fa-a2d6-1b5db464dbb3","Type":"ContainerStarted","Data":"e5e133cca55b43699055cfbc854aca17d0ad380d58159c3024b1be9125d7707b"}
Jul 26 18:12:17.032372 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:17.032323    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-7.ec2.internal" podStartSLOduration=21.032305705 podStartE2EDuration="21.032305705s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:11:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:00.009351823 +0000 UTC m=+5.716758885" watchObservedRunningTime="2026-07-26 18:12:17.032305705 +0000 UTC m=+22.739712768"
Jul 26 18:12:17.032905 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:17.032863    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-hp9vz" podStartSLOduration=3.206028682 podStartE2EDuration="22.032852754s" totalImagesPullingTime="18.826824072s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:11:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:17.032237116 +0000 UTC m=+22.739644201" watchObservedRunningTime="2026-07-26 18:12:17.032852754 +0000 UTC m=+22.740259814"
Jul 26 18:12:17.071661 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:17.071620    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-7z456" podStartSLOduration=3.216014229 podStartE2EDuration="22.071607365s" totalImagesPullingTime="18.855593136s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:11:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:17.071288953 +0000 UTC m=+22.778696011" watchObservedRunningTime="2026-07-26 18:12:17.071607365 +0000 UTC m=+22.779014424"
Jul 26 18:12:17.086076 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:17.086037    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-hz9mx" podStartSLOduration=3.266209279 podStartE2EDuration="22.086023157s" totalImagesPullingTime="18.819813878s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:11:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:17.085617744 +0000 UTC m=+22.793024804" watchObservedRunningTime="2026-07-26 18:12:17.086023157 +0000 UTC m=+22.793430216"
Jul 26 18:12:17.100645 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:17.100599    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6vnjh" podStartSLOduration=3.311983181 podStartE2EDuration="22.100583274s" totalImagesPullingTime="18.788600093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:11:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:17.100265511 +0000 UTC m=+22.807672569" watchObservedRunningTime="2026-07-26 18:12:17.100583274 +0000 UTC m=+22.807990372"
Jul 26 18:12:17.118926 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:17.118884    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-k9khs" podStartSLOduration=3.31430547 podStartE2EDuration="22.118868617s" totalImagesPullingTime="18.804563147s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:11:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:17.118703168 +0000 UTC m=+22.826110237" watchObservedRunningTime="2026-07-26 18:12:17.118868617 +0000 UTC m=+22.826275676"
Jul 26 18:12:17.224324 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:17.224287    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:17.298561 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:17.298533    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:17.843176 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:17.843143    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fsm57"
Jul 26 18:12:17.843344 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:17.843257    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fsm57" podUID="54acf4cd-cd75-4990-9ccf-274ef789daa5"
Jul 26 18:12:18.030453 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:18.030169    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw" event={"ID":"94b1d70f-c671-4fe3-bc2c-05b9426a496f","Type":"ContainerStarted","Data":"7373627850f79a32b5d9d6ebaeae6a4843fda36d94c51c931f78506676d51018"}
Jul 26 18:12:18.030453 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:18.030279    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw" event={"ID":"94b1d70f-c671-4fe3-bc2c-05b9426a496f","Type":"ContainerStarted","Data":"1867d7e42bfa9c5e36db1de5ab2e0cb6dc7a1d7adb7a4148bbac65ee262b8dac"}
Jul 26 18:12:18.030453 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:18.030329    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw" event={"ID":"94b1d70f-c671-4fe3-bc2c-05b9426a496f","Type":"ContainerStarted","Data":"bdc10fbc2fae0edd818d7aa23e3be7d2ca00a28545d01a16c3010939b3139fc3"}
Jul 26 18:12:18.030453 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:18.030387    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw" event={"ID":"94b1d70f-c671-4fe3-bc2c-05b9426a496f","Type":"ContainerStarted","Data":"07c98a993d63c37714b738b245044a292737056723f4acf032858a3c1a4ba446"}
Jul 26 18:12:18.109287 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:18.109259    2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 26 18:12:18.795295 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:18.795203    2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-26T18:12:18.109281737Z","UUID":"c6bdcf6f-99e0-4338-b80e-761bcc6c53d4","Handler":null,"Name":"","Endpoint":""}
Jul 26 18:12:18.796787 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:18.796761    2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 26 18:12:18.796904 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:18.796807    2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 26 18:12:18.843327 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:18.843295    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxp88"
Jul 26 18:12:18.843511 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:18.843410    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxp88" podUID="97230a04-09c6-4445-b713-98d115f6b1aa"
Jul 26 18:12:18.843511 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:18.843464    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rrn49"
Jul 26 18:12:18.843624 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:18.843557    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rrn49" podUID="8a2c5810-fcc5-4224-8d19-5fe5cc832d5a"
Jul 26 18:12:19.034273 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:19.034237    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6rvcp" event={"ID":"04f08429-551f-4765-8985-6dc6d0256dab","Type":"ContainerStarted","Data":"5989adf171c957719ef7d63a8829c4aebaf7010eefdb122a72acb7f6508b581d"}
Jul 26 18:12:19.036206 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:19.036170    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zfbtf" event={"ID":"f7378af1-e3bd-48fa-a2d6-1b5db464dbb3","Type":"ContainerStarted","Data":"7228b9485f0a55bd6956e912173088046731f1b4357d0e6bc8c4a563cd371e11"}
Jul 26 18:12:19.049186 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:19.049090    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-6rvcp" podStartSLOduration=5.230945004 podStartE2EDuration="24.049070242s" totalImagesPullingTime="18.818125238s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:11:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:19.048511048 +0000 UTC m=+24.755918120" watchObservedRunningTime="2026-07-26 18:12:19.049070242 +0000 UTC m=+24.756477302"
Jul 26 18:12:19.167949 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:19.167921    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:19.271155 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:19.271125    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:19.844122 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:19.843942    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fsm57"
Jul 26 18:12:19.844278 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:19.844208    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fsm57" podUID="54acf4cd-cd75-4990-9ccf-274ef789daa5"
Jul 26 18:12:20.042418 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:20.042381    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw" event={"ID":"94b1d70f-c671-4fe3-bc2c-05b9426a496f","Type":"ContainerStarted","Data":"15455b5a5eba6a4a30865905e9154cc0e73e3aae9c13ff7dd3dff192c5406539"}
Jul 26 18:12:20.044562 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:20.044535    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zfbtf" event={"ID":"f7378af1-e3bd-48fa-a2d6-1b5db464dbb3","Type":"ContainerStarted","Data":"1b5bf1f947d92d19c92d80f1bdbc624ba940ee1bc78314da892fa582039347be"}
Jul 26 18:12:20.061161 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:20.061122    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zfbtf" podStartSLOduration=4.067179934 podStartE2EDuration="25.061109343s" totalImagesPullingTime="20.993929409s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:11:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:20.061063468 +0000 UTC m=+25.768470538" watchObservedRunningTime="2026-07-26 18:12:20.061109343 +0000 UTC m=+25.768516399"
Jul 26 18:12:20.558574 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:20.558538    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:20.662059 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:20.662023    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:20.844080 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:20.843994    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rrn49"
Jul 26 18:12:20.844242 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:20.844003    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxp88"
Jul 26 18:12:20.844242 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:20.844136    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rrn49" podUID="8a2c5810-fcc5-4224-8d19-5fe5cc832d5a"
Jul 26 18:12:20.844242 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:20.844205    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxp88" podUID="97230a04-09c6-4445-b713-98d115f6b1aa"
Jul 26 18:12:21.044540 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:21.044504    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:21.102258 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:21.102177    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:21.162057 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:21.162024    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:21.405201 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:21.405173    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6vnjh"
Jul 26 18:12:21.405734 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:21.405716    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6vnjh"
Jul 26 18:12:21.843369 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:21.843191    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fsm57"
Jul 26 18:12:21.843526 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:21.843458    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fsm57" podUID="54acf4cd-cd75-4990-9ccf-274ef789daa5"
Jul 26 18:12:22.052004 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:22.051972    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw" event={"ID":"94b1d70f-c671-4fe3-bc2c-05b9426a496f","Type":"ContainerStarted","Data":"727a4116a8efdd8f6b23d01fbec52b0040fb6c900b97a3652233cbad3060925f"}
Jul 26 18:12:22.052703 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:22.052238    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:12:22.052703 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:22.052309    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:12:22.053897 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:22.053865    2576 generic.go:356] "Generic (PLEG): container finished" podID="63e8eeac-36f8-46a7-add2-5e222dac8d5f" containerID="97844ede92de7efd69f6dffcb6aa95ceaba4439ae4c90538b6813584b5d1fe17" exitCode=0
Jul 26 18:12:22.054023 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:22.053947    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rb2p7" event={"ID":"63e8eeac-36f8-46a7-add2-5e222dac8d5f","Type":"ContainerDied","Data":"97844ede92de7efd69f6dffcb6aa95ceaba4439ae4c90538b6813584b5d1fe17"}
Jul 26 18:12:22.054023 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:22.054017    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6vnjh"
Jul 26 18:12:22.054626 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:22.054562    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6vnjh"
Jul 26 18:12:22.068328 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:22.068311    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:12:22.110816 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:22.110767    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw" podStartSLOduration=8.229465676 podStartE2EDuration="27.110737725s" totalImagesPullingTime="18.881272049s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:11:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:22.082151622 +0000 UTC m=+27.789558671" watchObservedRunningTime="2026-07-26 18:12:22.110737725 +0000 UTC m=+27.818144784"
Jul 26 18:12:22.184718 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:22.184689    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:22.844073 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:22.844037    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxp88"
Jul 26 18:12:22.844272 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:22.844080    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rrn49"
Jul 26 18:12:22.844272 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:22.844179    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxp88" podUID="97230a04-09c6-4445-b713-98d115f6b1aa"
Jul 26 18:12:22.844348 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:22.844286    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rrn49" podUID="8a2c5810-fcc5-4224-8d19-5fe5cc832d5a"
Jul 26 18:12:23.056624 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:23.056594    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:12:23.071974 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:23.071953    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:12:23.738247 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:23.738059    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rrn49"]
Jul 26 18:12:23.738457 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:23.738308    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rrn49"
Jul 26 18:12:23.738457 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:23.738396    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rrn49" podUID="8a2c5810-fcc5-4224-8d19-5fe5cc832d5a"
Jul 26 18:12:23.741537 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:23.741509    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fsm57"]
Jul 26 18:12:23.741708 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:23.741621    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fsm57"
Jul 26 18:12:23.741797 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:23.741705    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fsm57" podUID="54acf4cd-cd75-4990-9ccf-274ef789daa5"
Jul 26 18:12:23.742215 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:23.742184    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fxp88"]
Jul 26 18:12:23.742306 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:23.742282    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxp88"
Jul 26 18:12:23.742418 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:23.742392    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxp88" podUID="97230a04-09c6-4445-b713-98d115f6b1aa"
Jul 26 18:12:24.060603 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:24.060519    2576 generic.go:356] "Generic (PLEG): container finished" podID="63e8eeac-36f8-46a7-add2-5e222dac8d5f" containerID="80521ce9eae5706e3059893ffa3dd882b5c90048deaa28c7954bfe40473f9a96" exitCode=0
Jul 26 18:12:24.060984 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:24.060611    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rb2p7" event={"ID":"63e8eeac-36f8-46a7-add2-5e222dac8d5f","Type":"ContainerDied","Data":"80521ce9eae5706e3059893ffa3dd882b5c90048deaa28c7954bfe40473f9a96"}
Jul 26 18:12:25.843952 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:25.843909    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rrn49"
Jul 26 18:12:25.843952 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:25.843955    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fsm57"
Jul 26 18:12:25.844717 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:25.844018    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxp88"
Jul 26 18:12:25.844717 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:25.844058    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rrn49" podUID="8a2c5810-fcc5-4224-8d19-5fe5cc832d5a"
Jul 26 18:12:25.844717 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:25.844071    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fsm57" podUID="54acf4cd-cd75-4990-9ccf-274ef789daa5"
Jul 26 18:12:25.844717 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:25.844126    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxp88" podUID="97230a04-09c6-4445-b713-98d115f6b1aa"
Jul 26 18:12:26.067704 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:26.067670    2576 generic.go:356] "Generic (PLEG): container finished" podID="63e8eeac-36f8-46a7-add2-5e222dac8d5f" containerID="b4118f90632b7525e35d883bafa9c9817a99a837d8f8ffe99d05bc5d750b1d4d" exitCode=0
Jul 26 18:12:26.067704 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:26.067714    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rb2p7" event={"ID":"63e8eeac-36f8-46a7-add2-5e222dac8d5f","Type":"ContainerDied","Data":"b4118f90632b7525e35d883bafa9c9817a99a837d8f8ffe99d05bc5d750b1d4d"}
Jul 26 18:12:26.896247 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:26.896213    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:26.955968 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:26.955912    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:27.319101 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:27.319067    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:27.379937 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:27.379899    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:27.737192 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:27.737150    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:27.839388 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:27.839338    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:27.843704 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:27.843641    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rrn49"
Jul 26 18:12:27.843836 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:27.843641    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fsm57"
Jul 26 18:12:27.843836 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:27.843759    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rrn49" podUID="8a2c5810-fcc5-4224-8d19-5fe5cc832d5a"
Jul 26 18:12:27.843928 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:27.843852    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fsm57" podUID="54acf4cd-cd75-4990-9ccf-274ef789daa5"
Jul 26 18:12:27.843928 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:27.843642    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxp88"
Jul 26 18:12:27.844023 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:27.843955    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxp88" podUID="97230a04-09c6-4445-b713-98d115f6b1aa"
Jul 26 18:12:27.918475 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:27.918432    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:28.065846 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:28.065813    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hz9mx_7e895b2b-026e-4228-9fc7-fbe90089f5a9/dns-node-resolver/0.log"
Jul 26 18:12:28.168250 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:28.168150    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:28.269203 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:28.269171    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:28.445947 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:28.445911    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:28.466835 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:28.466803    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97230a04-09c6-4445-b713-98d115f6b1aa-metrics-certs\") pod \"network-metrics-daemon-fxp88\" (UID: \"97230a04-09c6-4445-b713-98d115f6b1aa\") " pod="openshift-multus/network-metrics-daemon-fxp88"
Jul 26 18:12:28.466971 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:28.466853    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/54acf4cd-cd75-4990-9ccf-274ef789daa5-original-pull-secret\") pod \"global-pull-secret-syncer-fsm57\" (UID: \"54acf4cd-cd75-4990-9ccf-274ef789daa5\") " pod="kube-system/global-pull-secret-syncer-fsm57"
Jul 26 18:12:28.466971 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:28.466959    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 18:12:28.466971 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:28.466964    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 18:12:28.467134 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:28.467019    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54acf4cd-cd75-4990-9ccf-274ef789daa5-original-pull-secret podName:54acf4cd-cd75-4990-9ccf-274ef789daa5 nodeName:}" failed. No retries permitted until 2026-07-26 18:13:00.467000982 +0000 UTC m=+66.174408044 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/54acf4cd-cd75-4990-9ccf-274ef789daa5-original-pull-secret") pod "global-pull-secret-syncer-fsm57" (UID: "54acf4cd-cd75-4990-9ccf-274ef789daa5") : object "kube-system"/"original-pull-secret" not registered
Jul 26 18:12:28.467134 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:28.467036    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97230a04-09c6-4445-b713-98d115f6b1aa-metrics-certs podName:97230a04-09c6-4445-b713-98d115f6b1aa nodeName:}" failed. No retries permitted until 2026-07-26 18:13:00.467027542 +0000 UTC m=+66.174434582 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97230a04-09c6-4445-b713-98d115f6b1aa-metrics-certs") pod "network-metrics-daemon-fxp88" (UID: "97230a04-09c6-4445-b713-98d115f6b1aa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 18:12:28.540410 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:28.540374    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:28.567421 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:28.567392    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qzgk\" (UniqueName: \"kubernetes.io/projected/8a2c5810-fcc5-4224-8d19-5fe5cc832d5a-kube-api-access-7qzgk\") pod \"network-check-target-rrn49\" (UID: \"8a2c5810-fcc5-4224-8d19-5fe5cc832d5a\") " pod="openshift-network-diagnostics/network-check-target-rrn49"
Jul 26 18:12:28.567571 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:28.567554    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 18:12:28.567618 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:28.567576    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 18:12:28.567618 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:28.567585    2576 projected.go:196] Error preparing data for projected volume kube-api-access-7qzgk for pod openshift-network-diagnostics/network-check-target-rrn49: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 18:12:28.567702 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:28.567633    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a2c5810-fcc5-4224-8d19-5fe5cc832d5a-kube-api-access-7qzgk podName:8a2c5810-fcc5-4224-8d19-5fe5cc832d5a nodeName:}" failed. No retries permitted until 2026-07-26 18:13:00.567620051 +0000 UTC m=+66.275027093 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-7qzgk" (UniqueName: "kubernetes.io/projected/8a2c5810-fcc5-4224-8d19-5fe5cc832d5a-kube-api-access-7qzgk") pod "network-check-target-rrn49" (UID: "8a2c5810-fcc5-4224-8d19-5fe5cc832d5a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 18:12:29.063129 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.062891    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-k9khs_261aba58-ddd5-4f9e-af39-7475dec519f9/node-ca/0.log"
Jul 26 18:12:29.621123 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.621091    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-7.ec2.internal" event="NodeReady"
Jul 26 18:12:29.621330 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.621240    2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 26 18:12:29.661250 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.660427    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5957bd7985-8qzt8"]
Jul 26 18:12:29.663558 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.663532    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5957bd7985-8qzt8"
Jul 26 18:12:29.666535 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.666243    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-kssjv\""
Jul 26 18:12:29.666535 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.666247    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 26 18:12:29.666535 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.666250    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 26 18:12:29.666535 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.666301    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 26 18:12:29.672885 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.672863    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 26 18:12:29.675816 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.675793    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-zwspk"]
Jul 26 18:12:29.677432 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.677416    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jjkbn"]
Jul 26 18:12:29.677554 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.677538    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zwspk"
Jul 26 18:12:29.679311 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.679293    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jjkbn"
Jul 26 18:12:29.679544 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.679525    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5957bd7985-8qzt8"]
Jul 26 18:12:29.680591 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.680481    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 26 18:12:29.680591 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.680546    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 26 18:12:29.680591 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.680569    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 26 18:12:29.680809 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.680661    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rv5hp\""
Jul 26 18:12:29.682358 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.682175    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-w6wpb\""
Jul 26 18:12:29.682358 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.682318    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 26 18:12:29.682551 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.682536    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 26 18:12:29.697661 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.697643    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zwspk"]
Jul 26 18:12:29.701981 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.701962    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-7glwq"]
Jul 26 18:12:29.704192 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.704175    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7glwq"
Jul 26 18:12:29.708502 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.708484    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 26 18:12:29.708584 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.708488    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 26 18:12:29.709112 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.709094    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 26 18:12:29.709984 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.709964    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 26 18:12:29.710440 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.710196    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zll5p\""
Jul 26 18:12:29.714490 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.714468    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jjkbn"]
Jul 26 18:12:29.719476 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.719418    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7glwq"]
Jul 26 18:12:29.775039 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.775010    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/02ea08e3-10e7-49f9-b932-32f5b7eeca14-registry-tls\") pod \"image-registry-5957bd7985-8qzt8\" (UID: \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\") " pod="openshift-image-registry/image-registry-5957bd7985-8qzt8"
Jul 26 18:12:29.775039 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.775044    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/02ea08e3-10e7-49f9-b932-32f5b7eeca14-ca-trust-extracted\") pod \"image-registry-5957bd7985-8qzt8\" (UID: \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\") " pod="openshift-image-registry/image-registry-5957bd7985-8qzt8"
Jul 26 18:12:29.775271 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.775065    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9a3a2c2f-54f6-46c5-8cd3-6f343cba9713-cert\") pod \"ingress-canary-zwspk\" (UID: \"9a3a2c2f-54f6-46c5-8cd3-6f343cba9713\") " pod="openshift-ingress-canary/ingress-canary-zwspk"
Jul 26 18:12:29.775271 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.775088    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wbd7\" (UniqueName: \"kubernetes.io/projected/9a3a2c2f-54f6-46c5-8cd3-6f343cba9713-kube-api-access-4wbd7\") pod \"ingress-canary-zwspk\" (UID: \"9a3a2c2f-54f6-46c5-8cd3-6f343cba9713\") " pod="openshift-ingress-canary/ingress-canary-zwspk"
Jul 26 18:12:29.775271 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.775117    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9efa7a18-6a50-4d18-834a-8a50301009c2-crio-socket\") pod \"insights-runtime-extractor-7glwq\" (UID: \"9efa7a18-6a50-4d18-834a-8a50301009c2\") " pod="openshift-insights/insights-runtime-extractor-7glwq"
Jul 26 18:12:29.775271 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.775178    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rb5w6\" (UniqueName: \"kubernetes.io/projected/02ea08e3-10e7-49f9-b932-32f5b7eeca14-kube-api-access-rb5w6\") pod \"image-registry-5957bd7985-8qzt8\" (UID: \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\") " pod="openshift-image-registry/image-registry-5957bd7985-8qzt8"
Jul 26 18:12:29.775271 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.775227    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/265041ab-6c9e-4b4b-834b-812bb4c3040e-metrics-tls\") pod \"dns-default-jjkbn\" (UID: \"265041ab-6c9e-4b4b-834b-812bb4c3040e\") " pod="openshift-dns/dns-default-jjkbn"
Jul 26 18:12:29.775499 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.775276    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/265041ab-6c9e-4b4b-834b-812bb4c3040e-tmp-dir\") pod \"dns-default-jjkbn\" (UID: \"265041ab-6c9e-4b4b-834b-812bb4c3040e\") " pod="openshift-dns/dns-default-jjkbn"
Jul 26 18:12:29.775499 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.775307    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/02ea08e3-10e7-49f9-b932-32f5b7eeca14-bound-sa-token\") pod \"image-registry-5957bd7985-8qzt8\" (UID: \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\") " pod="openshift-image-registry/image-registry-5957bd7985-8qzt8"
Jul 26 18:12:29.775499 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.775333    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxfxn\" (UniqueName: \"kubernetes.io/projected/9efa7a18-6a50-4d18-834a-8a50301009c2-kube-api-access-bxfxn\") pod \"insights-runtime-extractor-7glwq\" (UID: \"9efa7a18-6a50-4d18-834a-8a50301009c2\") " pod="openshift-insights/insights-runtime-extractor-7glwq"
Jul 26 18:12:29.775499 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.775391    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/02ea08e3-10e7-49f9-b932-32f5b7eeca14-image-registry-private-configuration\") pod \"image-registry-5957bd7985-8qzt8\" (UID: \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\") " pod="openshift-image-registry/image-registry-5957bd7985-8qzt8"
Jul 26 18:12:29.775499 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.775416    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/02ea08e3-10e7-49f9-b932-32f5b7eeca14-registry-certificates\") pod \"image-registry-5957bd7985-8qzt8\" (UID: \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\") " pod="openshift-image-registry/image-registry-5957bd7985-8qzt8"
Jul 26 18:12:29.775499 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.775441    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9efa7a18-6a50-4d18-834a-8a50301009c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7glwq\" (UID: \"9efa7a18-6a50-4d18-834a-8a50301009c2\") " pod="openshift-insights/insights-runtime-extractor-7glwq"
Jul 26 18:12:29.775499 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.775468    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/265041ab-6c9e-4b4b-834b-812bb4c3040e-config-volume\") pod \"dns-default-jjkbn\" (UID: \"265041ab-6c9e-4b4b-834b-812bb4c3040e\") " pod="openshift-dns/dns-default-jjkbn"
Jul 26 18:12:29.775499 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.775491    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7bsz\" (UniqueName: \"kubernetes.io/projected/265041ab-6c9e-4b4b-834b-812bb4c3040e-kube-api-access-z7bsz\") pod \"dns-default-jjkbn\" (UID: \"265041ab-6c9e-4b4b-834b-812bb4c3040e\") " pod="openshift-dns/dns-default-jjkbn"
Jul 26 18:12:29.775885 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.775608    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/02ea08e3-10e7-49f9-b932-32f5b7eeca14-trusted-ca\") pod \"image-registry-5957bd7985-8qzt8\" (UID: \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\") " pod="openshift-image-registry/image-registry-5957bd7985-8qzt8"
Jul 26 18:12:29.775885 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.775639    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9efa7a18-6a50-4d18-834a-8a50301009c2-data-volume\") pod \"insights-runtime-extractor-7glwq\" (UID: \"9efa7a18-6a50-4d18-834a-8a50301009c2\") " pod="openshift-insights/insights-runtime-extractor-7glwq"
Jul 26 18:12:29.775885 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.775663    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/02ea08e3-10e7-49f9-b932-32f5b7eeca14-installation-pull-secrets\") pod \"image-registry-5957bd7985-8qzt8\" (UID: \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\") " pod="openshift-image-registry/image-registry-5957bd7985-8qzt8"
Jul 26 18:12:29.775885 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.775689    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9efa7a18-6a50-4d18-834a-8a50301009c2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7glwq\" (UID: \"9efa7a18-6a50-4d18-834a-8a50301009c2\") " pod="openshift-insights/insights-runtime-extractor-7glwq"
Jul 26 18:12:29.843588 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.843552    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rrn49"
Jul 26 18:12:29.843783 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.843552    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fsm57"
Jul 26 18:12:29.843783 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.843567    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxp88"
Jul 26 18:12:29.846535 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.846497    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 26 18:12:29.846686 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.846607    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 26 18:12:29.846686 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.846659    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-l7f4l\""
Jul 26 18:12:29.846857 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.846687    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vf5dv\""
Jul 26 18:12:29.846857 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.846803    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 26 18:12:29.847009 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.846894    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 26 18:12:29.847392 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.847370    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:29.876739 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.876684    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/02ea08e3-10e7-49f9-b932-32f5b7eeca14-bound-sa-token\") pod \"image-registry-5957bd7985-8qzt8\" (UID: \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\") " pod="openshift-image-registry/image-registry-5957bd7985-8qzt8"
Jul 26 18:12:29.876739 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.876731    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxfxn\" (UniqueName: \"kubernetes.io/projected/9efa7a18-6a50-4d18-834a-8a50301009c2-kube-api-access-bxfxn\") pod \"insights-runtime-extractor-7glwq\" (UID: \"9efa7a18-6a50-4d18-834a-8a50301009c2\") " pod="openshift-insights/insights-runtime-extractor-7glwq"
Jul 26 18:12:29.876910 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.876779    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/02ea08e3-10e7-49f9-b932-32f5b7eeca14-image-registry-private-configuration\") pod \"image-registry-5957bd7985-8qzt8\" (UID: \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\") " pod="openshift-image-registry/image-registry-5957bd7985-8qzt8"
Jul 26 18:12:29.876910 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.876798    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/02ea08e3-10e7-49f9-b932-32f5b7eeca14-registry-certificates\") pod \"image-registry-5957bd7985-8qzt8\" (UID: \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\") " pod="openshift-image-registry/image-registry-5957bd7985-8qzt8"
Jul 26 18:12:29.876910 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.876822    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9efa7a18-6a50-4d18-834a-8a50301009c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7glwq\" (UID: \"9efa7a18-6a50-4d18-834a-8a50301009c2\") " pod="openshift-insights/insights-runtime-extractor-7glwq"
Jul 26 18:12:29.876910 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.876843    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/265041ab-6c9e-4b4b-834b-812bb4c3040e-config-volume\") pod \"dns-default-jjkbn\" (UID: \"265041ab-6c9e-4b4b-834b-812bb4c3040e\") " pod="openshift-dns/dns-default-jjkbn"
Jul 26 18:12:29.876910 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.876867    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7bsz\" (UniqueName: \"kubernetes.io/projected/265041ab-6c9e-4b4b-834b-812bb4c3040e-kube-api-access-z7bsz\") pod \"dns-default-jjkbn\" (UID: \"265041ab-6c9e-4b4b-834b-812bb4c3040e\") " pod="openshift-dns/dns-default-jjkbn"
Jul 26 18:12:29.877179 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.876934    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/02ea08e3-10e7-49f9-b932-32f5b7eeca14-trusted-ca\") pod \"image-registry-5957bd7985-8qzt8\" (UID: \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\") " pod="openshift-image-registry/image-registry-5957bd7985-8qzt8"
Jul 26 18:12:29.877179 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.876959    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9efa7a18-6a50-4d18-834a-8a50301009c2-data-volume\") pod \"insights-runtime-extractor-7glwq\" (UID: \"9efa7a18-6a50-4d18-834a-8a50301009c2\") " pod="openshift-insights/insights-runtime-extractor-7glwq"
Jul 26 18:12:29.877179 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:29.876978    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 26 18:12:29.877179 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.876983    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/02ea08e3-10e7-49f9-b932-32f5b7eeca14-installation-pull-secrets\") pod \"image-registry-5957bd7985-8qzt8\" (UID: \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\") " pod="openshift-image-registry/image-registry-5957bd7985-8qzt8"
Jul 26 18:12:29.877179 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.877003    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9efa7a18-6a50-4d18-834a-8a50301009c2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7glwq\" (UID: \"9efa7a18-6a50-4d18-834a-8a50301009c2\") " pod="openshift-insights/insights-runtime-extractor-7glwq"
Jul 26 18:12:29.877179 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:29.877057    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9efa7a18-6a50-4d18-834a-8a50301009c2-insights-runtime-extractor-tls podName:9efa7a18-6a50-4d18-834a-8a50301009c2 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:30.377035537 +0000 UTC m=+36.084442576 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9efa7a18-6a50-4d18-834a-8a50301009c2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7glwq" (UID: "9efa7a18-6a50-4d18-834a-8a50301009c2") : secret "insights-runtime-extractor-tls" not found
Jul 26 18:12:29.877179 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:29.877065    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 26 18:12:29.877179 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.877080    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/02ea08e3-10e7-49f9-b932-32f5b7eeca14-registry-tls\") pod \"image-registry-5957bd7985-8qzt8\" (UID: \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\") " pod="openshift-image-registry/image-registry-5957bd7985-8qzt8"
Jul 26 18:12:29.877179 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:29.877103    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9efa7a18-6a50-4d18-834a-8a50301009c2-kube-rbac-proxy-cm podName:9efa7a18-6a50-4d18-834a-8a50301009c2 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:30.377092946 +0000 UTC m=+36.084499983 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9efa7a18-6a50-4d18-834a-8a50301009c2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7glwq" (UID: "9efa7a18-6a50-4d18-834a-8a50301009c2") : configmap "kube-rbac-proxy" not found
Jul 26 18:12:29.877179 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.877123    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/02ea08e3-10e7-49f9-b932-32f5b7eeca14-ca-trust-extracted\") pod \"image-registry-5957bd7985-8qzt8\" (UID: \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\") " pod="openshift-image-registry/image-registry-5957bd7985-8qzt8"
Jul 26 18:12:29.877179 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:29.877137    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 26 18:12:29.877179 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:29.877148    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5957bd7985-8qzt8: secret "image-registry-tls" not found
Jul 26 18:12:29.877179 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:29.877178    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/02ea08e3-10e7-49f9-b932-32f5b7eeca14-registry-tls podName:02ea08e3-10e7-49f9-b932-32f5b7eeca14 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:30.377168249 +0000 UTC m=+36.084575286 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/02ea08e3-10e7-49f9-b932-32f5b7eeca14-registry-tls") pod "image-registry-5957bd7985-8qzt8" (UID: "02ea08e3-10e7-49f9-b932-32f5b7eeca14") : secret "image-registry-tls" not found
Jul 26 18:12:29.877179 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.877138    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9a3a2c2f-54f6-46c5-8cd3-6f343cba9713-cert\") pod \"ingress-canary-zwspk\" (UID: \"9a3a2c2f-54f6-46c5-8cd3-6f343cba9713\") " pod="openshift-ingress-canary/ingress-canary-zwspk"
Jul 26 18:12:29.877834 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.877205    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wbd7\" (UniqueName: \"kubernetes.io/projected/9a3a2c2f-54f6-46c5-8cd3-6f343cba9713-kube-api-access-4wbd7\") pod \"ingress-canary-zwspk\" (UID: \"9a3a2c2f-54f6-46c5-8cd3-6f343cba9713\") " pod="openshift-ingress-canary/ingress-canary-zwspk"
Jul 26 18:12:29.877834 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.877230    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9efa7a18-6a50-4d18-834a-8a50301009c2-crio-socket\") pod \"insights-runtime-extractor-7glwq\" (UID: \"9efa7a18-6a50-4d18-834a-8a50301009c2\") " pod="openshift-insights/insights-runtime-extractor-7glwq"
Jul 26 18:12:29.877834 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.877255    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb5w6\" (UniqueName: \"kubernetes.io/projected/02ea08e3-10e7-49f9-b932-32f5b7eeca14-kube-api-access-rb5w6\") pod \"image-registry-5957bd7985-8qzt8\" (UID: \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\") " pod="openshift-image-registry/image-registry-5957bd7985-8qzt8"
Jul 26 18:12:29.877834 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.877278    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/265041ab-6c9e-4b4b-834b-812bb4c3040e-metrics-tls\") pod \"dns-default-jjkbn\" (UID: \"265041ab-6c9e-4b4b-834b-812bb4c3040e\") " pod="openshift-dns/dns-default-jjkbn"
Jul 26 18:12:29.877834 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.877297    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/265041ab-6c9e-4b4b-834b-812bb4c3040e-tmp-dir\") pod \"dns-default-jjkbn\" (UID: \"265041ab-6c9e-4b4b-834b-812bb4c3040e\") " pod="openshift-dns/dns-default-jjkbn"
Jul 26 18:12:29.877834 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.877487    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/265041ab-6c9e-4b4b-834b-812bb4c3040e-config-volume\") pod \"dns-default-jjkbn\" (UID: \"265041ab-6c9e-4b4b-834b-812bb4c3040e\") " pod="openshift-dns/dns-default-jjkbn"
Jul 26 18:12:29.877834 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:29.877611    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 26 18:12:29.877834 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:29.877651    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/265041ab-6c9e-4b4b-834b-812bb4c3040e-metrics-tls podName:265041ab-6c9e-4b4b-834b-812bb4c3040e nodeName:}" failed. No retries permitted until 2026-07-26 18:12:30.377637341 +0000 UTC m=+36.085044380 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/265041ab-6c9e-4b4b-834b-812bb4c3040e-metrics-tls") pod "dns-default-jjkbn" (UID: "265041ab-6c9e-4b4b-834b-812bb4c3040e") : secret "dns-default-metrics-tls" not found
Jul 26 18:12:29.877834 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.877668    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/02ea08e3-10e7-49f9-b932-32f5b7eeca14-ca-trust-extracted\") pod \"image-registry-5957bd7985-8qzt8\" (UID: \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\") " pod="openshift-image-registry/image-registry-5957bd7985-8qzt8"
Jul 26 18:12:29.878242 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.877869    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9efa7a18-6a50-4d18-834a-8a50301009c2-crio-socket\") pod \"insights-runtime-extractor-7glwq\" (UID: \"9efa7a18-6a50-4d18-834a-8a50301009c2\") " pod="openshift-insights/insights-runtime-extractor-7glwq"
Jul 26 18:12:29.878242 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.877939    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/02ea08e3-10e7-49f9-b932-32f5b7eeca14-registry-certificates\") pod \"image-registry-5957bd7985-8qzt8\" (UID: \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\") " pod="openshift-image-registry/image-registry-5957bd7985-8qzt8"
Jul 26 18:12:29.878242 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.878116    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9efa7a18-6a50-4d18-834a-8a50301009c2-data-volume\") pod \"insights-runtime-extractor-7glwq\" (UID: \"9efa7a18-6a50-4d18-834a-8a50301009c2\") " pod="openshift-insights/insights-runtime-extractor-7glwq"
Jul 26 18:12:29.878242 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.878128    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/02ea08e3-10e7-49f9-b932-32f5b7eeca14-trusted-ca\") pod \"image-registry-5957bd7985-8qzt8\" (UID: \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\") " pod="openshift-image-registry/image-registry-5957bd7985-8qzt8"
Jul 26 18:12:29.878242 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.878169    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/265041ab-6c9e-4b4b-834b-812bb4c3040e-tmp-dir\") pod \"dns-default-jjkbn\" (UID: \"265041ab-6c9e-4b4b-834b-812bb4c3040e\") " pod="openshift-dns/dns-default-jjkbn"
Jul 26 18:12:29.881516 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.881493    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/02ea08e3-10e7-49f9-b932-32f5b7eeca14-installation-pull-secrets\") pod \"image-registry-5957bd7985-8qzt8\" (UID: \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\") " pod="openshift-image-registry/image-registry-5957bd7985-8qzt8"
Jul 26 18:12:29.881621 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.881529    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/02ea08e3-10e7-49f9-b932-32f5b7eeca14-image-registry-private-configuration\") pod \"image-registry-5957bd7985-8qzt8\" (UID: \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\") " pod="openshift-image-registry/image-registry-5957bd7985-8qzt8"
Jul 26 18:12:29.881621 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.881576    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9a3a2c2f-54f6-46c5-8cd3-6f343cba9713-cert\") pod \"ingress-canary-zwspk\" (UID: \"9a3a2c2f-54f6-46c5-8cd3-6f343cba9713\") " pod="openshift-ingress-canary/ingress-canary-zwspk"
Jul 26 18:12:29.886965 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.886944    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7bsz\" (UniqueName: \"kubernetes.io/projected/265041ab-6c9e-4b4b-834b-812bb4c3040e-kube-api-access-z7bsz\") pod \"dns-default-jjkbn\" (UID: \"265041ab-6c9e-4b4b-834b-812bb4c3040e\") " pod="openshift-dns/dns-default-jjkbn"
Jul 26 18:12:29.887069 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.886975    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxfxn\" (UniqueName: \"kubernetes.io/projected/9efa7a18-6a50-4d18-834a-8a50301009c2-kube-api-access-bxfxn\") pod \"insights-runtime-extractor-7glwq\" (UID: \"9efa7a18-6a50-4d18-834a-8a50301009c2\") " pod="openshift-insights/insights-runtime-extractor-7glwq"
Jul 26 18:12:29.887069 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.887012    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rb5w6\" (UniqueName: \"kubernetes.io/projected/02ea08e3-10e7-49f9-b932-32f5b7eeca14-kube-api-access-rb5w6\") pod \"image-registry-5957bd7985-8qzt8\" (UID: \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\") " pod="openshift-image-registry/image-registry-5957bd7985-8qzt8"
Jul 26 18:12:29.887348 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.887328    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wbd7\" (UniqueName: \"kubernetes.io/projected/9a3a2c2f-54f6-46c5-8cd3-6f343cba9713-kube-api-access-4wbd7\") pod \"ingress-canary-zwspk\" (UID: \"9a3a2c2f-54f6-46c5-8cd3-6f343cba9713\") " pod="openshift-ingress-canary/ingress-canary-zwspk"
Jul 26 18:12:29.888537 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.888518    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/02ea08e3-10e7-49f9-b932-32f5b7eeca14-bound-sa-token\") pod \"image-registry-5957bd7985-8qzt8\" (UID: \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\") " pod="openshift-image-registry/image-registry-5957bd7985-8qzt8"
Jul 26 18:12:29.951233 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.951194    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:29.989081 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:29.989057    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zwspk"
Jul 26 18:12:30.044777 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:30.044728    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-7qsn9"]
Jul 26 18:12:30.046813 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:30.046793    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-7qsn9"
Jul 26 18:12:30.049409 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:30.049267    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 26 18:12:30.049409 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:30.049292    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 26 18:12:30.049409 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:30.049366    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-g4zt2\""
Jul 26 18:12:30.049409 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:30.049384    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 26 18:12:30.049622 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:30.049573    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 26 18:12:30.056331 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:30.056306    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-7qsn9"]
Jul 26 18:12:30.082281 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:30.082243    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:30.146088 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:30.146002    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:30.179535 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:30.179505    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vtvg\" (UniqueName: \"kubernetes.io/projected/ff442dbb-9d1f-4a00-8093-5dc4c8cc6fea-kube-api-access-4vtvg\") pod \"service-ca-658d7b7c78-7qsn9\" (UID: \"ff442dbb-9d1f-4a00-8093-5dc4c8cc6fea\") " pod="openshift-service-ca/service-ca-658d7b7c78-7qsn9"
Jul 26 18:12:30.179700 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:30.179564    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ff442dbb-9d1f-4a00-8093-5dc4c8cc6fea-signing-cabundle\") pod \"service-ca-658d7b7c78-7qsn9\" (UID: \"ff442dbb-9d1f-4a00-8093-5dc4c8cc6fea\") " pod="openshift-service-ca/service-ca-658d7b7c78-7qsn9"
Jul 26 18:12:30.179700 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:30.179648    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ff442dbb-9d1f-4a00-8093-5dc4c8cc6fea-signing-key\") pod \"service-ca-658d7b7c78-7qsn9\" (UID: \"ff442dbb-9d1f-4a00-8093-5dc4c8cc6fea\") " pod="openshift-service-ca/service-ca-658d7b7c78-7qsn9"
Jul 26 18:12:30.280683 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:30.280657    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ff442dbb-9d1f-4a00-8093-5dc4c8cc6fea-signing-key\") pod \"service-ca-658d7b7c78-7qsn9\" (UID: \"ff442dbb-9d1f-4a00-8093-5dc4c8cc6fea\") " pod="openshift-service-ca/service-ca-658d7b7c78-7qsn9"
Jul 26 18:12:30.280809 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:30.280741    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vtvg\" (UniqueName: \"kubernetes.io/projected/ff442dbb-9d1f-4a00-8093-5dc4c8cc6fea-kube-api-access-4vtvg\") pod \"service-ca-658d7b7c78-7qsn9\" (UID: \"ff442dbb-9d1f-4a00-8093-5dc4c8cc6fea\") " pod="openshift-service-ca/service-ca-658d7b7c78-7qsn9"
Jul 26 18:12:30.280809 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:30.280804    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ff442dbb-9d1f-4a00-8093-5dc4c8cc6fea-signing-cabundle\") pod \"service-ca-658d7b7c78-7qsn9\" (UID: \"ff442dbb-9d1f-4a00-8093-5dc4c8cc6fea\") " pod="openshift-service-ca/service-ca-658d7b7c78-7qsn9"
Jul 26 18:12:30.281398 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:30.281374    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ff442dbb-9d1f-4a00-8093-5dc4c8cc6fea-signing-cabundle\") pod \"service-ca-658d7b7c78-7qsn9\" (UID: \"ff442dbb-9d1f-4a00-8093-5dc4c8cc6fea\") " pod="openshift-service-ca/service-ca-658d7b7c78-7qsn9"
Jul 26 18:12:30.283601 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:30.283580    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ff442dbb-9d1f-4a00-8093-5dc4c8cc6fea-signing-key\") pod \"service-ca-658d7b7c78-7qsn9\" (UID: \"ff442dbb-9d1f-4a00-8093-5dc4c8cc6fea\") " pod="openshift-service-ca/service-ca-658d7b7c78-7qsn9"
Jul 26 18:12:30.291927 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:30.291904    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vtvg\" (UniqueName: \"kubernetes.io/projected/ff442dbb-9d1f-4a00-8093-5dc4c8cc6fea-kube-api-access-4vtvg\") pod \"service-ca-658d7b7c78-7qsn9\" (UID: \"ff442dbb-9d1f-4a00-8093-5dc4c8cc6fea\") " pod="openshift-service-ca/service-ca-658d7b7c78-7qsn9"
Jul 26 18:12:30.314072 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:30.313843    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zwspk"]
Jul 26 18:12:30.317267 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:12:30.317244    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a3a2c2f_54f6_46c5_8cd3_6f343cba9713.slice/crio-c92424a8b8f73d5ec3fdf4b59ef68c1ed6deefb53d1ced325062615d93793dc6 WatchSource:0}: Error finding container c92424a8b8f73d5ec3fdf4b59ef68c1ed6deefb53d1ced325062615d93793dc6: Status 404 returned error can't find the container with id c92424a8b8f73d5ec3fdf4b59ef68c1ed6deefb53d1ced325062615d93793dc6
Jul 26 18:12:30.357904 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:30.357871    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-7qsn9"
Jul 26 18:12:30.382085 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:30.382061    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9efa7a18-6a50-4d18-834a-8a50301009c2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7glwq\" (UID: \"9efa7a18-6a50-4d18-834a-8a50301009c2\") " pod="openshift-insights/insights-runtime-extractor-7glwq"
Jul 26 18:12:30.382170 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:30.382123    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/02ea08e3-10e7-49f9-b932-32f5b7eeca14-registry-tls\") pod \"image-registry-5957bd7985-8qzt8\" (UID: \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\") " pod="openshift-image-registry/image-registry-5957bd7985-8qzt8"
Jul 26 18:12:30.382170 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:30.382164    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 26 18:12:30.382273 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:30.382185    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/265041ab-6c9e-4b4b-834b-812bb4c3040e-metrics-tls\") pod \"dns-default-jjkbn\" (UID: \"265041ab-6c9e-4b4b-834b-812bb4c3040e\") " pod="openshift-dns/dns-default-jjkbn"
Jul 26 18:12:30.382273 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:30.382237    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9efa7a18-6a50-4d18-834a-8a50301009c2-kube-rbac-proxy-cm podName:9efa7a18-6a50-4d18-834a-8a50301009c2 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:31.382214995 +0000 UTC m=+37.089622044 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9efa7a18-6a50-4d18-834a-8a50301009c2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7glwq" (UID: "9efa7a18-6a50-4d18-834a-8a50301009c2") : configmap "kube-rbac-proxy" not found
Jul 26 18:12:30.382731 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:30.382270    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9efa7a18-6a50-4d18-834a-8a50301009c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7glwq\" (UID: \"9efa7a18-6a50-4d18-834a-8a50301009c2\") " pod="openshift-insights/insights-runtime-extractor-7glwq"
Jul 26 18:12:30.382731 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:30.382337    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 26 18:12:30.382731 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:30.382359    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5957bd7985-8qzt8: secret "image-registry-tls" not found
Jul 26 18:12:30.382731 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:30.382406    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 26 18:12:30.382731 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:30.382429    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/02ea08e3-10e7-49f9-b932-32f5b7eeca14-registry-tls podName:02ea08e3-10e7-49f9-b932-32f5b7eeca14 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:31.382410936 +0000 UTC m=+37.089817987 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/02ea08e3-10e7-49f9-b932-32f5b7eeca14-registry-tls") pod "image-registry-5957bd7985-8qzt8" (UID: "02ea08e3-10e7-49f9-b932-32f5b7eeca14") : secret "image-registry-tls" not found
Jul 26 18:12:30.382731 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:30.382448    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9efa7a18-6a50-4d18-834a-8a50301009c2-insights-runtime-extractor-tls podName:9efa7a18-6a50-4d18-834a-8a50301009c2 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:31.382434727 +0000 UTC m=+37.089841769 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9efa7a18-6a50-4d18-834a-8a50301009c2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7glwq" (UID: "9efa7a18-6a50-4d18-834a-8a50301009c2") : secret "insights-runtime-extractor-tls" not found
Jul 26 18:12:30.383017 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:30.382825    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 26 18:12:30.383017 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:30.382872    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/265041ab-6c9e-4b4b-834b-812bb4c3040e-metrics-tls podName:265041ab-6c9e-4b4b-834b-812bb4c3040e nodeName:}" failed. No retries permitted until 2026-07-26 18:12:31.382862792 +0000 UTC m=+37.090269830 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/265041ab-6c9e-4b4b-834b-812bb4c3040e-metrics-tls") pod "dns-default-jjkbn" (UID: "265041ab-6c9e-4b4b-834b-812bb4c3040e") : secret "dns-default-metrics-tls" not found
Jul 26 18:12:30.497378 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:30.497347    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-7qsn9"]
Jul 26 18:12:30.500953 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:12:30.500929    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff442dbb_9d1f_4a00_8093_5dc4c8cc6fea.slice/crio-0a61789796e6cf19f8f3fb2898826abce71d7c6e88aad4f8613291cc57151345 WatchSource:0}: Error finding container 0a61789796e6cf19f8f3fb2898826abce71d7c6e88aad4f8613291cc57151345: Status 404 returned error can't find the container with id 0a61789796e6cf19f8f3fb2898826abce71d7c6e88aad4f8613291cc57151345
Jul 26 18:12:30.773527 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:30.773482    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:31.082730 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:31.082617    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-7qsn9" event={"ID":"ff442dbb-9d1f-4a00-8093-5dc4c8cc6fea","Type":"ContainerStarted","Data":"0a61789796e6cf19f8f3fb2898826abce71d7c6e88aad4f8613291cc57151345"}
Jul 26 18:12:31.086276 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:31.086246    2576 generic.go:356] "Generic (PLEG): container finished" podID="63e8eeac-36f8-46a7-add2-5e222dac8d5f" containerID="9a405be22b6972b774cac5c1cbc497038c76a427574bc440d3f816d62686a958" exitCode=0
Jul 26 18:12:31.086393 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:31.086339    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rb2p7" event={"ID":"63e8eeac-36f8-46a7-add2-5e222dac8d5f","Type":"ContainerDied","Data":"9a405be22b6972b774cac5c1cbc497038c76a427574bc440d3f816d62686a958"}
Jul 26 18:12:31.088123 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:31.088100    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zwspk" event={"ID":"9a3a2c2f-54f6-46c5-8cd3-6f343cba9713","Type":"ContainerStarted","Data":"c92424a8b8f73d5ec3fdf4b59ef68c1ed6deefb53d1ced325062615d93793dc6"}
Jul 26 18:12:31.390997 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:31.390920    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/02ea08e3-10e7-49f9-b932-32f5b7eeca14-registry-tls\") pod \"image-registry-5957bd7985-8qzt8\" (UID: \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\") " pod="openshift-image-registry/image-registry-5957bd7985-8qzt8"
Jul 26 18:12:31.390997 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:31.390972    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/265041ab-6c9e-4b4b-834b-812bb4c3040e-metrics-tls\") pod \"dns-default-jjkbn\" (UID: \"265041ab-6c9e-4b4b-834b-812bb4c3040e\") " pod="openshift-dns/dns-default-jjkbn"
Jul 26 18:12:31.391210 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:31.391017    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9efa7a18-6a50-4d18-834a-8a50301009c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7glwq\" (UID: \"9efa7a18-6a50-4d18-834a-8a50301009c2\") " pod="openshift-insights/insights-runtime-extractor-7glwq"
Jul 26 18:12:31.391210 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:31.391083    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9efa7a18-6a50-4d18-834a-8a50301009c2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7glwq\" (UID: \"9efa7a18-6a50-4d18-834a-8a50301009c2\") " pod="openshift-insights/insights-runtime-extractor-7glwq"
Jul 26 18:12:31.391210 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:31.391155    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 26 18:12:31.391372 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:31.391219    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9efa7a18-6a50-4d18-834a-8a50301009c2-kube-rbac-proxy-cm podName:9efa7a18-6a50-4d18-834a-8a50301009c2 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:33.391200661 +0000 UTC m=+39.098607716 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9efa7a18-6a50-4d18-834a-8a50301009c2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7glwq" (UID: "9efa7a18-6a50-4d18-834a-8a50301009c2") : configmap "kube-rbac-proxy" not found
Jul 26 18:12:31.391372 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:31.391219    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 26 18:12:31.391372 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:31.391257    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/265041ab-6c9e-4b4b-834b-812bb4c3040e-metrics-tls podName:265041ab-6c9e-4b4b-834b-812bb4c3040e nodeName:}" failed. No retries permitted until 2026-07-26 18:12:33.391247517 +0000 UTC m=+39.098654562 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/265041ab-6c9e-4b4b-834b-812bb4c3040e-metrics-tls") pod "dns-default-jjkbn" (UID: "265041ab-6c9e-4b4b-834b-812bb4c3040e") : secret "dns-default-metrics-tls" not found
Jul 26 18:12:31.391372 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:31.391156    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 26 18:12:31.391372 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:31.391273    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5957bd7985-8qzt8: secret "image-registry-tls" not found
Jul 26 18:12:31.391372 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:31.391299    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/02ea08e3-10e7-49f9-b932-32f5b7eeca14-registry-tls podName:02ea08e3-10e7-49f9-b932-32f5b7eeca14 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:33.391291099 +0000 UTC m=+39.098698150 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/02ea08e3-10e7-49f9-b932-32f5b7eeca14-registry-tls") pod "image-registry-5957bd7985-8qzt8" (UID: "02ea08e3-10e7-49f9-b932-32f5b7eeca14") : secret "image-registry-tls" not found
Jul 26 18:12:31.391372 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:31.391351    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 26 18:12:31.391706 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:31.391379    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9efa7a18-6a50-4d18-834a-8a50301009c2-insights-runtime-extractor-tls podName:9efa7a18-6a50-4d18-834a-8a50301009c2 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:33.391369012 +0000 UTC m=+39.098776052 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9efa7a18-6a50-4d18-834a-8a50301009c2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7glwq" (UID: "9efa7a18-6a50-4d18-834a-8a50301009c2") : secret "insights-runtime-extractor-tls" not found
Jul 26 18:12:32.093924 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:32.093885    2576 generic.go:356] "Generic (PLEG): container finished" podID="63e8eeac-36f8-46a7-add2-5e222dac8d5f" containerID="886f717b36da332a5218c118c7ef090986e8979cffe299f911a91181c6b766e5" exitCode=0
Jul 26 18:12:32.094453 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:32.093964    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rb2p7" event={"ID":"63e8eeac-36f8-46a7-add2-5e222dac8d5f","Type":"ContainerDied","Data":"886f717b36da332a5218c118c7ef090986e8979cffe299f911a91181c6b766e5"}
Jul 26 18:12:32.551526 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:32.551494    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:33.407910 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:33.407675    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9efa7a18-6a50-4d18-834a-8a50301009c2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7glwq\" (UID: \"9efa7a18-6a50-4d18-834a-8a50301009c2\") " pod="openshift-insights/insights-runtime-extractor-7glwq"
Jul 26 18:12:33.408394 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:33.407961    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/02ea08e3-10e7-49f9-b932-32f5b7eeca14-registry-tls\") pod \"image-registry-5957bd7985-8qzt8\" (UID: \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\") " pod="openshift-image-registry/image-registry-5957bd7985-8qzt8"
Jul 26 18:12:33.408394 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:33.408011    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/265041ab-6c9e-4b4b-834b-812bb4c3040e-metrics-tls\") pod \"dns-default-jjkbn\" (UID: \"265041ab-6c9e-4b4b-834b-812bb4c3040e\") " pod="openshift-dns/dns-default-jjkbn"
Jul 26 18:12:33.408394 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:33.408116    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 26 18:12:33.408394 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:33.408159    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9efa7a18-6a50-4d18-834a-8a50301009c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7glwq\" (UID: \"9efa7a18-6a50-4d18-834a-8a50301009c2\") " pod="openshift-insights/insights-runtime-extractor-7glwq"
Jul 26 18:12:33.408394 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:33.408173    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/265041ab-6c9e-4b4b-834b-812bb4c3040e-metrics-tls podName:265041ab-6c9e-4b4b-834b-812bb4c3040e nodeName:}" failed. No retries permitted until 2026-07-26 18:12:37.408156757 +0000 UTC m=+43.115563796 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/265041ab-6c9e-4b4b-834b-812bb4c3040e-metrics-tls") pod "dns-default-jjkbn" (UID: "265041ab-6c9e-4b4b-834b-812bb4c3040e") : secret "dns-default-metrics-tls" not found
Jul 26 18:12:33.408394 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:33.408119    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 26 18:12:33.408394 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:33.408216    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5957bd7985-8qzt8: secret "image-registry-tls" not found
Jul 26 18:12:33.408394 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:33.408251    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 26 18:12:33.408394 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:33.408272    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/02ea08e3-10e7-49f9-b932-32f5b7eeca14-registry-tls podName:02ea08e3-10e7-49f9-b932-32f5b7eeca14 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:37.408255404 +0000 UTC m=+43.115662444 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/02ea08e3-10e7-49f9-b932-32f5b7eeca14-registry-tls") pod "image-registry-5957bd7985-8qzt8" (UID: "02ea08e3-10e7-49f9-b932-32f5b7eeca14") : secret "image-registry-tls" not found
Jul 26 18:12:33.408394 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:33.408299    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9efa7a18-6a50-4d18-834a-8a50301009c2-insights-runtime-extractor-tls podName:9efa7a18-6a50-4d18-834a-8a50301009c2 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:37.408284039 +0000 UTC m=+43.115691093 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9efa7a18-6a50-4d18-834a-8a50301009c2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7glwq" (UID: "9efa7a18-6a50-4d18-834a-8a50301009c2") : secret "insights-runtime-extractor-tls" not found
Jul 26 18:12:33.408394 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:33.408396    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9efa7a18-6a50-4d18-834a-8a50301009c2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7glwq\" (UID: \"9efa7a18-6a50-4d18-834a-8a50301009c2\") " pod="openshift-insights/insights-runtime-extractor-7glwq"
Jul 26 18:12:34.100207 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:34.100173    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-7qsn9" event={"ID":"ff442dbb-9d1f-4a00-8093-5dc4c8cc6fea","Type":"ContainerStarted","Data":"3f16742fee9a4252e1c72acf7e2b81a468129beaf1d92bf3fa7bb1ec426632f5"}
Jul 26 18:12:34.103460 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:34.103431    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rb2p7" event={"ID":"63e8eeac-36f8-46a7-add2-5e222dac8d5f","Type":"ContainerStarted","Data":"a2cf037d27c2cdd06fcab35c9a0b2c62749f78143e3d490863da2c8f3927e6f1"}
Jul 26 18:12:34.104806 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:34.104782    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zwspk" event={"ID":"9a3a2c2f-54f6-46c5-8cd3-6f343cba9713","Type":"ContainerStarted","Data":"9a4e708c55c63f7e3f8f2ed0ce03dc1aa3f99e5c37ffadf20d4790ca952ac664"}
Jul 26 18:12:34.118321 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:34.118281    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-7qsn9" podStartSLOduration=1.371748315 podStartE2EDuration="4.118268168s" totalImagesPullingTime="2.746519853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:12:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:34.117831311 +0000 UTC m=+39.825238371" watchObservedRunningTime="2026-07-26 18:12:34.118268168 +0000 UTC m=+39.825675227"
Jul 26 18:12:34.134240 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:34.134184    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-zwspk" podStartSLOduration=2.198372082 podStartE2EDuration="5.134166947s" totalImagesPullingTime="2.935794865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:12:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:34.13304488 +0000 UTC m=+39.840451939" watchObservedRunningTime="2026-07-26 18:12:34.134166947 +0000 UTC m=+39.841574008"
Jul 26 18:12:34.156005 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:34.155967    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rb2p7" podStartSLOduration=9.628807426 podStartE2EDuration="39.155954951s" totalImagesPullingTime="29.527147525s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:11:55 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:34.154427183 +0000 UTC m=+39.861834242" watchObservedRunningTime="2026-07-26 18:12:34.155954951 +0000 UTC m=+39.863362010"
Jul 26 18:12:34.399960 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:34.399866    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:34.529588 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:34.529545    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:34.731888 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:34.731855    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:34.797792 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:34.797738    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:34.870526 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:34.870490    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:35.107099 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:35.107012    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:35.294968 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:35.294933    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:35.710326 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:35.710296    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:36.300865 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:36.300829    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:36.907169 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:36.907133    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:37.152933 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:37.152899    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:37.309094 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:37.309061    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:37.440798 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:37.440765    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/265041ab-6c9e-4b4b-834b-812bb4c3040e-metrics-tls\") pod \"dns-default-jjkbn\" (UID: \"265041ab-6c9e-4b4b-834b-812bb4c3040e\") " pod="openshift-dns/dns-default-jjkbn"
Jul 26 18:12:37.440979 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:37.440812    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9efa7a18-6a50-4d18-834a-8a50301009c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7glwq\" (UID: \"9efa7a18-6a50-4d18-834a-8a50301009c2\") " pod="openshift-insights/insights-runtime-extractor-7glwq"
Jul 26 18:12:37.440979 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:37.440870    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/02ea08e3-10e7-49f9-b932-32f5b7eeca14-registry-tls\") pod \"image-registry-5957bd7985-8qzt8\" (UID: \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\") " pod="openshift-image-registry/image-registry-5957bd7985-8qzt8"
Jul 26 18:12:37.440979 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:37.440921    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 26 18:12:37.440979 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:37.440954    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 26 18:12:37.440979 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:37.440963    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5957bd7985-8qzt8: secret "image-registry-tls" not found
Jul 26 18:12:37.441176 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:37.440979    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 26 18:12:37.441176 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:37.440994    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/265041ab-6c9e-4b4b-834b-812bb4c3040e-metrics-tls podName:265041ab-6c9e-4b4b-834b-812bb4c3040e nodeName:}" failed. No retries permitted until 2026-07-26 18:12:45.440976263 +0000 UTC m=+51.148383303 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/265041ab-6c9e-4b4b-834b-812bb4c3040e-metrics-tls") pod "dns-default-jjkbn" (UID: "265041ab-6c9e-4b4b-834b-812bb4c3040e") : secret "dns-default-metrics-tls" not found
Jul 26 18:12:37.441176 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:37.441009    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/02ea08e3-10e7-49f9-b932-32f5b7eeca14-registry-tls podName:02ea08e3-10e7-49f9-b932-32f5b7eeca14 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:45.441003719 +0000 UTC m=+51.148410756 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/02ea08e3-10e7-49f9-b932-32f5b7eeca14-registry-tls") pod "image-registry-5957bd7985-8qzt8" (UID: "02ea08e3-10e7-49f9-b932-32f5b7eeca14") : secret "image-registry-tls" not found
Jul 26 18:12:37.441176 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:12:37.441032    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9efa7a18-6a50-4d18-834a-8a50301009c2-insights-runtime-extractor-tls podName:9efa7a18-6a50-4d18-834a-8a50301009c2 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:45.44101448 +0000 UTC m=+51.148421517 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9efa7a18-6a50-4d18-834a-8a50301009c2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7glwq" (UID: "9efa7a18-6a50-4d18-834a-8a50301009c2") : secret "insights-runtime-extractor-tls" not found
Jul 26 18:12:37.621079 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:37.620982    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:37.723430 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:37.723397    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:38.098828 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:38.098798    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:38.336857 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:38.336823    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:38.502863 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:38.502831    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:38.694390 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:38.694359    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:39.096006 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:39.095968    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:39.154953 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:39.154920    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:39.302709 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:39.302677    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:39.718974 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:39.718944    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:39.812808 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:39.812771    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:39.913359 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:39.913316    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:40.312856 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:40.312819    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:40.727076 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:40.727044    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:40.927350 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:40.927302    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:41.102716 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:41.102626    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:41.302912 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:41.302879    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:41.539572 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:41.539535    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:42.317736 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:42.317697    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:42.383830 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:42.383800    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:42.497368 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:42.497335    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:42.899442 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:42.899408    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:43.101032 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:43.101001    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:43.303108 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:43.303073    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:43.698678 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:43.698647    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:44.296966 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:44.296925    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:44.697281 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:44.697247    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:44.905268 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:44.905241    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:45.102283 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:45.102173    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:45.507694 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:45.507655    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/02ea08e3-10e7-49f9-b932-32f5b7eeca14-registry-tls\") pod \"image-registry-5957bd7985-8qzt8\" (UID: \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\") " pod="openshift-image-registry/image-registry-5957bd7985-8qzt8"
Jul 26 18:12:45.508201 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:45.507703    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/265041ab-6c9e-4b4b-834b-812bb4c3040e-metrics-tls\") pod \"dns-default-jjkbn\" (UID: \"265041ab-6c9e-4b4b-834b-812bb4c3040e\") " pod="openshift-dns/dns-default-jjkbn"
Jul 26 18:12:45.508201 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:45.507773    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9efa7a18-6a50-4d18-834a-8a50301009c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7glwq\" (UID: \"9efa7a18-6a50-4d18-834a-8a50301009c2\") " pod="openshift-insights/insights-runtime-extractor-7glwq"
Jul 26 18:12:45.511243 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:45.511217    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9efa7a18-6a50-4d18-834a-8a50301009c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7glwq\" (UID: \"9efa7a18-6a50-4d18-834a-8a50301009c2\") " pod="openshift-insights/insights-runtime-extractor-7glwq"
Jul 26 18:12:45.511344 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:45.511221    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/265041ab-6c9e-4b4b-834b-812bb4c3040e-metrics-tls\") pod \"dns-default-jjkbn\" (UID: \"265041ab-6c9e-4b4b-834b-812bb4c3040e\") " pod="openshift-dns/dns-default-jjkbn"
Jul 26 18:12:45.511413 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:45.511306    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/02ea08e3-10e7-49f9-b932-32f5b7eeca14-registry-tls\") pod \"image-registry-5957bd7985-8qzt8\" (UID: \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\") " pod="openshift-image-registry/image-registry-5957bd7985-8qzt8"
Jul 26 18:12:45.574472 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:45.574433    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5957bd7985-8qzt8"
Jul 26 18:12:45.595926 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:45.595906    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jjkbn"
Jul 26 18:12:45.614508 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:45.614478    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7glwq"
Jul 26 18:12:45.727024 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:45.726871    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5957bd7985-8qzt8"]
Jul 26 18:12:45.744491 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:45.744461    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jjkbn"]
Jul 26 18:12:45.748309 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:12:45.748278    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod265041ab_6c9e_4b4b_834b_812bb4c3040e.slice/crio-ea4c6fb241b3d54a41829296793f42686b669c6a1076ec596fbc1b201af64651 WatchSource:0}: Error finding container ea4c6fb241b3d54a41829296793f42686b669c6a1076ec596fbc1b201af64651: Status 404 returned error can't find the container with id ea4c6fb241b3d54a41829296793f42686b669c6a1076ec596fbc1b201af64651
Jul 26 18:12:45.765148 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:45.765126    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7glwq"]
Jul 26 18:12:45.773876 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:12:45.773853    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9efa7a18_6a50_4d18_834a_8a50301009c2.slice/crio-a5f396061609feb0ffa2d680a75e1f8f0cfaa7c01cbf24ee35f159fc6782e307 WatchSource:0}: Error finding container a5f396061609feb0ffa2d680a75e1f8f0cfaa7c01cbf24ee35f159fc6782e307: Status 404 returned error can't find the container with id a5f396061609feb0ffa2d680a75e1f8f0cfaa7c01cbf24ee35f159fc6782e307
Jul 26 18:12:46.134351 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:46.134273    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jjkbn" event={"ID":"265041ab-6c9e-4b4b-834b-812bb4c3040e","Type":"ContainerStarted","Data":"ea4c6fb241b3d54a41829296793f42686b669c6a1076ec596fbc1b201af64651"}
Jul 26 18:12:46.135846 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:46.135822    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5957bd7985-8qzt8" event={"ID":"02ea08e3-10e7-49f9-b932-32f5b7eeca14","Type":"ContainerStarted","Data":"49fdc223e747125b0b4cee45d5758b81ebbf5d7e91966982c8fb8f51c480fb77"}
Jul 26 18:12:46.135957 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:46.135857    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5957bd7985-8qzt8" event={"ID":"02ea08e3-10e7-49f9-b932-32f5b7eeca14","Type":"ContainerStarted","Data":"228a9c964772625bc2e5b05de50c84e94e8a325b973c060548c7ecb61c83c071"}
Jul 26 18:12:46.135957 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:46.135920    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5957bd7985-8qzt8"
Jul 26 18:12:46.136983 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:46.136964    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7glwq" event={"ID":"9efa7a18-6a50-4d18-834a-8a50301009c2","Type":"ContainerStarted","Data":"a5f396061609feb0ffa2d680a75e1f8f0cfaa7c01cbf24ee35f159fc6782e307"}
Jul 26 18:12:46.157785 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:46.157717    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5957bd7985-8qzt8" podStartSLOduration=34.157699168 podStartE2EDuration="34.157699168s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:12:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:46.157035474 +0000 UTC m=+51.864442533" watchObservedRunningTime="2026-07-26 18:12:46.157699168 +0000 UTC m=+51.865106232"
Jul 26 18:12:49.128650 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:49.128604    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:49.146945 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:49.146916    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jjkbn" event={"ID":"265041ab-6c9e-4b4b-834b-812bb4c3040e","Type":"ContainerStarted","Data":"e4911d079fc531a2c3b7f64028dc6dbc6682d909abb1398be93e55269dcb66c7"}
Jul 26 18:12:49.147120 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:49.146975    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jjkbn" event={"ID":"265041ab-6c9e-4b4b-834b-812bb4c3040e","Type":"ContainerStarted","Data":"d43c23a98986612850e63730db95a36526493178d093933641279d5682b1dd61"}
Jul 26 18:12:49.147120 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:49.147065    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jjkbn"
Jul 26 18:12:49.201295 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:49.201254    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:49.396080 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:49.396002    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:49.763078 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:49.763039    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:49.825495 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:49.825453    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:50.002533 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:50.002495    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:50.067930 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:50.067847    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:50.181944 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:50.181902    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:50.792236 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:50.792199    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:50.983585 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:50.983559    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:51.096794 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:51.096683    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:51.153709 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:51.153678    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7glwq" event={"ID":"9efa7a18-6a50-4d18-834a-8a50301009c2","Type":"ContainerStarted","Data":"8fe1aaa864e7c6cfd7d8e960de88834169c1e762a0e8c9746bf9e56a6e3afb0d"}
Jul 26 18:12:51.994305 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:51.994243    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:52.493379 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.493315    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-jjkbn" podStartSLOduration=21.205691274 podStartE2EDuration="23.493290189s" totalImagesPullingTime="2.287598915s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:12:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:49.168440348 +0000 UTC m=+54.875847407" watchObservedRunningTime="2026-07-26 18:12:52.493290189 +0000 UTC m=+58.200697248"
Jul 26 18:12:52.494102 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.494074    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-88fdc7bd-pvssv"]
Jul 26 18:12:52.500664 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.500638    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-568889d6c5-r6gfn"]
Jul 26 18:12:52.500812 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.500792    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-88fdc7bd-pvssv"
Jul 26 18:12:52.503405 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.503383    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b99866955-6qc6m"]
Jul 26 18:12:52.503513 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.503502    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-568889d6c5-r6gfn"
Jul 26 18:12:52.503916 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.503892    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 26 18:12:52.504065 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.503914    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 26 18:12:52.504065 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.503983    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-sjp5t\""
Jul 26 18:12:52.504193 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.504137    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 26 18:12:52.504255 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.504229    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 26 18:12:52.505892 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.505860    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b99866955-6qc6m"
Jul 26 18:12:52.506402 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.506384    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 26 18:12:52.510263 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.510241    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 26 18:12:52.511281 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.511113    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-88fdc7bd-pvssv"]
Jul 26 18:12:52.512096 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.512049    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-568889d6c5-r6gfn"]
Jul 26 18:12:52.512434 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.512418    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 26 18:12:52.512920 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.512905    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 26 18:12:52.513315 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.513298    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 26 18:12:52.514005 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.513965    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b99866955-6qc6m"]
Jul 26 18:12:52.522183 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.522162    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5957bd7985-8qzt8"]
Jul 26 18:12:52.557998 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.557962    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcmr7\" (UniqueName: \"kubernetes.io/projected/14f11537-8b51-41d9-a057-5df0d60dd970-kube-api-access-xcmr7\") pod \"cluster-proxy-proxy-agent-6b99866955-6qc6m\" (UID: \"14f11537-8b51-41d9-a057-5df0d60dd970\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b99866955-6qc6m"
Jul 26 18:12:52.558200 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.558182    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/14f11537-8b51-41d9-a057-5df0d60dd970-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6b99866955-6qc6m\" (UID: \"14f11537-8b51-41d9-a057-5df0d60dd970\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b99866955-6qc6m"
Jul 26 18:12:52.558316 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.558265    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wscdk\" (UniqueName: \"kubernetes.io/projected/8dd206b4-b7eb-4a1c-8203-b17702b8a32e-kube-api-access-wscdk\") pod \"klusterlet-addon-workmgr-568889d6c5-r6gfn\" (UID: \"8dd206b4-b7eb-4a1c-8203-b17702b8a32e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-568889d6c5-r6gfn"
Jul 26 18:12:52.558316 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.558306    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blqv4\" (UniqueName: \"kubernetes.io/projected/b57a27f1-ce1b-4b47-851f-d981132c3441-kube-api-access-blqv4\") pod \"managed-serviceaccount-addon-agent-88fdc7bd-pvssv\" (UID: \"b57a27f1-ce1b-4b47-851f-d981132c3441\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-88fdc7bd-pvssv"
Jul 26 18:12:52.558430 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.558381    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/14f11537-8b51-41d9-a057-5df0d60dd970-ca\") pod \"cluster-proxy-proxy-agent-6b99866955-6qc6m\" (UID: \"14f11537-8b51-41d9-a057-5df0d60dd970\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b99866955-6qc6m"
Jul 26 18:12:52.558486 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.558457    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/14f11537-8b51-41d9-a057-5df0d60dd970-hub\") pod \"cluster-proxy-proxy-agent-6b99866955-6qc6m\" (UID: \"14f11537-8b51-41d9-a057-5df0d60dd970\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b99866955-6qc6m"
Jul 26 18:12:52.558541 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.558494    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/14f11537-8b51-41d9-a057-5df0d60dd970-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6b99866955-6qc6m\" (UID: \"14f11537-8b51-41d9-a057-5df0d60dd970\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b99866955-6qc6m"
Jul 26 18:12:52.558590 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.558547    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b57a27f1-ce1b-4b47-851f-d981132c3441-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-88fdc7bd-pvssv\" (UID: \"b57a27f1-ce1b-4b47-851f-d981132c3441\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-88fdc7bd-pvssv"
Jul 26 18:12:52.558641 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.558604    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/14f11537-8b51-41d9-a057-5df0d60dd970-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6b99866955-6qc6m\" (UID: \"14f11537-8b51-41d9-a057-5df0d60dd970\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b99866955-6qc6m"
Jul 26 18:12:52.558701 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.558636    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8dd206b4-b7eb-4a1c-8203-b17702b8a32e-tmp\") pod \"klusterlet-addon-workmgr-568889d6c5-r6gfn\" (UID: \"8dd206b4-b7eb-4a1c-8203-b17702b8a32e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-568889d6c5-r6gfn"
Jul 26 18:12:52.558788 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.558768    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8dd206b4-b7eb-4a1c-8203-b17702b8a32e-klusterlet-config\") pod \"klusterlet-addon-workmgr-568889d6c5-r6gfn\" (UID: \"8dd206b4-b7eb-4a1c-8203-b17702b8a32e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-568889d6c5-r6gfn"
Jul 26 18:12:52.565503 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.565475    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:52.597048 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.597013    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6964ccb4d8-t226q"]
Jul 26 18:12:52.599996 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.599975    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6964ccb4d8-t226q"
Jul 26 18:12:52.622658 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.622615    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6964ccb4d8-t226q"]
Jul 26 18:12:52.659520 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.659484    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wscdk\" (UniqueName: \"kubernetes.io/projected/8dd206b4-b7eb-4a1c-8203-b17702b8a32e-kube-api-access-wscdk\") pod \"klusterlet-addon-workmgr-568889d6c5-r6gfn\" (UID: \"8dd206b4-b7eb-4a1c-8203-b17702b8a32e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-568889d6c5-r6gfn"
Jul 26 18:12:52.659680 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.659533    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blqv4\" (UniqueName: \"kubernetes.io/projected/b57a27f1-ce1b-4b47-851f-d981132c3441-kube-api-access-blqv4\") pod \"managed-serviceaccount-addon-agent-88fdc7bd-pvssv\" (UID: \"b57a27f1-ce1b-4b47-851f-d981132c3441\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-88fdc7bd-pvssv"
Jul 26 18:12:52.659680 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.659595    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d2839e3a-08fe-4fdd-9e83-792cab573778-bound-sa-token\") pod \"image-registry-6964ccb4d8-t226q\" (UID: \"d2839e3a-08fe-4fdd-9e83-792cab573778\") " pod="openshift-image-registry/image-registry-6964ccb4d8-t226q"
Jul 26 18:12:52.659680 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.659622    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/14f11537-8b51-41d9-a057-5df0d60dd970-ca\") pod \"cluster-proxy-proxy-agent-6b99866955-6qc6m\" (UID: \"14f11537-8b51-41d9-a057-5df0d60dd970\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b99866955-6qc6m"
Jul 26 18:12:52.659680 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.659651    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/14f11537-8b51-41d9-a057-5df0d60dd970-hub\") pod \"cluster-proxy-proxy-agent-6b99866955-6qc6m\" (UID: \"14f11537-8b51-41d9-a057-5df0d60dd970\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b99866955-6qc6m"
Jul 26 18:12:52.659680 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.659678    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/14f11537-8b51-41d9-a057-5df0d60dd970-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6b99866955-6qc6m\" (UID: \"14f11537-8b51-41d9-a057-5df0d60dd970\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b99866955-6qc6m"
Jul 26 18:12:52.659927 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.659707    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d2839e3a-08fe-4fdd-9e83-792cab573778-image-registry-private-configuration\") pod \"image-registry-6964ccb4d8-t226q\" (UID: \"d2839e3a-08fe-4fdd-9e83-792cab573778\") " pod="openshift-image-registry/image-registry-6964ccb4d8-t226q"
Jul 26 18:12:52.659965 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.659921    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d2839e3a-08fe-4fdd-9e83-792cab573778-registry-tls\") pod \"image-registry-6964ccb4d8-t226q\" (UID: \"d2839e3a-08fe-4fdd-9e83-792cab573778\") " pod="openshift-image-registry/image-registry-6964ccb4d8-t226q"
Jul 26 18:12:52.660005 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.659968    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b57a27f1-ce1b-4b47-851f-d981132c3441-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-88fdc7bd-pvssv\" (UID: \"b57a27f1-ce1b-4b47-851f-d981132c3441\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-88fdc7bd-pvssv"
Jul 26 18:12:52.660057 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.660027    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/14f11537-8b51-41d9-a057-5df0d60dd970-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6b99866955-6qc6m\" (UID: \"14f11537-8b51-41d9-a057-5df0d60dd970\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b99866955-6qc6m"
Jul 26 18:12:52.660108 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.660053    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8dd206b4-b7eb-4a1c-8203-b17702b8a32e-tmp\") pod \"klusterlet-addon-workmgr-568889d6c5-r6gfn\" (UID: \"8dd206b4-b7eb-4a1c-8203-b17702b8a32e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-568889d6c5-r6gfn"
Jul 26 18:12:52.660468 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.660272    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d2839e3a-08fe-4fdd-9e83-792cab573778-ca-trust-extracted\") pod \"image-registry-6964ccb4d8-t226q\" (UID: \"d2839e3a-08fe-4fdd-9e83-792cab573778\") " pod="openshift-image-registry/image-registry-6964ccb4d8-t226q"
Jul 26 18:12:52.660468 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.660324    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8dd206b4-b7eb-4a1c-8203-b17702b8a32e-klusterlet-config\") pod \"klusterlet-addon-workmgr-568889d6c5-r6gfn\" (UID: \"8dd206b4-b7eb-4a1c-8203-b17702b8a32e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-568889d6c5-r6gfn"
Jul 26 18:12:52.660468 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.660351    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqnjw\" (UniqueName: \"kubernetes.io/projected/d2839e3a-08fe-4fdd-9e83-792cab573778-kube-api-access-fqnjw\") pod \"image-registry-6964ccb4d8-t226q\" (UID: \"d2839e3a-08fe-4fdd-9e83-792cab573778\") " pod="openshift-image-registry/image-registry-6964ccb4d8-t226q"
Jul 26 18:12:52.660468 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.660405    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d2839e3a-08fe-4fdd-9e83-792cab573778-registry-certificates\") pod \"image-registry-6964ccb4d8-t226q\" (UID: \"d2839e3a-08fe-4fdd-9e83-792cab573778\") " pod="openshift-image-registry/image-registry-6964ccb4d8-t226q"
Jul 26 18:12:52.660468 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.660444    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcmr7\" (UniqueName: \"kubernetes.io/projected/14f11537-8b51-41d9-a057-5df0d60dd970-kube-api-access-xcmr7\") pod \"cluster-proxy-proxy-agent-6b99866955-6qc6m\" (UID: \"14f11537-8b51-41d9-a057-5df0d60dd970\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b99866955-6qc6m"
Jul 26 18:12:52.660813 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.660475    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/14f11537-8b51-41d9-a057-5df0d60dd970-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6b99866955-6qc6m\" (UID: \"14f11537-8b51-41d9-a057-5df0d60dd970\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b99866955-6qc6m"
Jul 26 18:12:52.660813 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.660517    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d2839e3a-08fe-4fdd-9e83-792cab573778-installation-pull-secrets\") pod \"image-registry-6964ccb4d8-t226q\" (UID: \"d2839e3a-08fe-4fdd-9e83-792cab573778\") " pod="openshift-image-registry/image-registry-6964ccb4d8-t226q"
Jul 26 18:12:52.660813 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.660560    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d2839e3a-08fe-4fdd-9e83-792cab573778-trusted-ca\") pod \"image-registry-6964ccb4d8-t226q\" (UID: \"d2839e3a-08fe-4fdd-9e83-792cab573778\") " pod="openshift-image-registry/image-registry-6964ccb4d8-t226q"
Jul 26 18:12:52.660813 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.660560    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8dd206b4-b7eb-4a1c-8203-b17702b8a32e-tmp\") pod \"klusterlet-addon-workmgr-568889d6c5-r6gfn\" (UID: \"8dd206b4-b7eb-4a1c-8203-b17702b8a32e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-568889d6c5-r6gfn"
Jul 26 18:12:52.661391 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.661292    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/14f11537-8b51-41d9-a057-5df0d60dd970-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6b99866955-6qc6m\" (UID: \"14f11537-8b51-41d9-a057-5df0d60dd970\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b99866955-6qc6m"
Jul 26 18:12:52.663124 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.662956    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/14f11537-8b51-41d9-a057-5df0d60dd970-hub\") pod \"cluster-proxy-proxy-agent-6b99866955-6qc6m\" (UID: \"14f11537-8b51-41d9-a057-5df0d60dd970\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b99866955-6qc6m"
Jul 26 18:12:52.663124 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.662982    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/14f11537-8b51-41d9-a057-5df0d60dd970-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6b99866955-6qc6m\" (UID: \"14f11537-8b51-41d9-a057-5df0d60dd970\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b99866955-6qc6m"
Jul 26 18:12:52.663461 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.663437    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/14f11537-8b51-41d9-a057-5df0d60dd970-ca\") pod \"cluster-proxy-proxy-agent-6b99866955-6qc6m\" (UID: \"14f11537-8b51-41d9-a057-5df0d60dd970\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b99866955-6qc6m"
Jul 26 18:12:52.663928 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.663898    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/14f11537-8b51-41d9-a057-5df0d60dd970-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6b99866955-6qc6m\" (UID: \"14f11537-8b51-41d9-a057-5df0d60dd970\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b99866955-6qc6m"
Jul 26 18:12:52.664275 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.664248    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b57a27f1-ce1b-4b47-851f-d981132c3441-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-88fdc7bd-pvssv\" (UID: \"b57a27f1-ce1b-4b47-851f-d981132c3441\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-88fdc7bd-pvssv"
Jul 26 18:12:52.664387 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.664371    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8dd206b4-b7eb-4a1c-8203-b17702b8a32e-klusterlet-config\") pod \"klusterlet-addon-workmgr-568889d6c5-r6gfn\" (UID: \"8dd206b4-b7eb-4a1c-8203-b17702b8a32e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-568889d6c5-r6gfn"
Jul 26 18:12:52.672911 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.672885    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wscdk\" (UniqueName: \"kubernetes.io/projected/8dd206b4-b7eb-4a1c-8203-b17702b8a32e-kube-api-access-wscdk\") pod \"klusterlet-addon-workmgr-568889d6c5-r6gfn\" (UID: \"8dd206b4-b7eb-4a1c-8203-b17702b8a32e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-568889d6c5-r6gfn"
Jul 26 18:12:52.673329 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.673311    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcmr7\" (UniqueName: \"kubernetes.io/projected/14f11537-8b51-41d9-a057-5df0d60dd970-kube-api-access-xcmr7\") pod \"cluster-proxy-proxy-agent-6b99866955-6qc6m\" (UID: \"14f11537-8b51-41d9-a057-5df0d60dd970\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b99866955-6qc6m"
Jul 26 18:12:52.674923 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.674906    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blqv4\" (UniqueName: \"kubernetes.io/projected/b57a27f1-ce1b-4b47-851f-d981132c3441-kube-api-access-blqv4\") pod \"managed-serviceaccount-addon-agent-88fdc7bd-pvssv\" (UID: \"b57a27f1-ce1b-4b47-851f-d981132c3441\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-88fdc7bd-pvssv"
Jul 26 18:12:52.682147 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.682120    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:52.761726 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.761643    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d2839e3a-08fe-4fdd-9e83-792cab573778-bound-sa-token\") pod \"image-registry-6964ccb4d8-t226q\" (UID: \"d2839e3a-08fe-4fdd-9e83-792cab573778\") " pod="openshift-image-registry/image-registry-6964ccb4d8-t226q"
Jul 26 18:12:52.761726 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.761693    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d2839e3a-08fe-4fdd-9e83-792cab573778-image-registry-private-configuration\") pod \"image-registry-6964ccb4d8-t226q\" (UID: \"d2839e3a-08fe-4fdd-9e83-792cab573778\") " pod="openshift-image-registry/image-registry-6964ccb4d8-t226q"
Jul 26 18:12:52.761953 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.761726    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d2839e3a-08fe-4fdd-9e83-792cab573778-registry-tls\") pod \"image-registry-6964ccb4d8-t226q\" (UID: \"d2839e3a-08fe-4fdd-9e83-792cab573778\") " pod="openshift-image-registry/image-registry-6964ccb4d8-t226q"
Jul 26 18:12:52.761953 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.761802    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d2839e3a-08fe-4fdd-9e83-792cab573778-ca-trust-extracted\") pod \"image-registry-6964ccb4d8-t226q\" (UID: \"d2839e3a-08fe-4fdd-9e83-792cab573778\") " pod="openshift-image-registry/image-registry-6964ccb4d8-t226q"
Jul 26 18:12:52.761953 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.761832    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqnjw\" (UniqueName: \"kubernetes.io/projected/d2839e3a-08fe-4fdd-9e83-792cab573778-kube-api-access-fqnjw\") pod \"image-registry-6964ccb4d8-t226q\" (UID: \"d2839e3a-08fe-4fdd-9e83-792cab573778\") " pod="openshift-image-registry/image-registry-6964ccb4d8-t226q"
Jul 26 18:12:52.761953 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.761873    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d2839e3a-08fe-4fdd-9e83-792cab573778-registry-certificates\") pod \"image-registry-6964ccb4d8-t226q\" (UID: \"d2839e3a-08fe-4fdd-9e83-792cab573778\") " pod="openshift-image-registry/image-registry-6964ccb4d8-t226q"
Jul 26 18:12:52.761953 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.761911    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d2839e3a-08fe-4fdd-9e83-792cab573778-installation-pull-secrets\") pod \"image-registry-6964ccb4d8-t226q\" (UID: \"d2839e3a-08fe-4fdd-9e83-792cab573778\") " pod="openshift-image-registry/image-registry-6964ccb4d8-t226q"
Jul 26 18:12:52.761953 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.761948    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d2839e3a-08fe-4fdd-9e83-792cab573778-trusted-ca\") pod \"image-registry-6964ccb4d8-t226q\" (UID: \"d2839e3a-08fe-4fdd-9e83-792cab573778\") " pod="openshift-image-registry/image-registry-6964ccb4d8-t226q"
Jul 26 18:12:52.762288 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.762262    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d2839e3a-08fe-4fdd-9e83-792cab573778-ca-trust-extracted\") pod \"image-registry-6964ccb4d8-t226q\" (UID: \"d2839e3a-08fe-4fdd-9e83-792cab573778\") " pod="openshift-image-registry/image-registry-6964ccb4d8-t226q"
Jul 26 18:12:52.762776 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.762731    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d2839e3a-08fe-4fdd-9e83-792cab573778-registry-certificates\") pod \"image-registry-6964ccb4d8-t226q\" (UID: \"d2839e3a-08fe-4fdd-9e83-792cab573778\") " pod="openshift-image-registry/image-registry-6964ccb4d8-t226q"
Jul 26 18:12:52.762893 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.762876    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d2839e3a-08fe-4fdd-9e83-792cab573778-trusted-ca\") pod \"image-registry-6964ccb4d8-t226q\" (UID: \"d2839e3a-08fe-4fdd-9e83-792cab573778\") " pod="openshift-image-registry/image-registry-6964ccb4d8-t226q"
Jul 26 18:12:52.764646 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.764566    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d2839e3a-08fe-4fdd-9e83-792cab573778-image-registry-private-configuration\") pod \"image-registry-6964ccb4d8-t226q\" (UID: \"d2839e3a-08fe-4fdd-9e83-792cab573778\") " pod="openshift-image-registry/image-registry-6964ccb4d8-t226q"
Jul 26 18:12:52.764646 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.764598    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d2839e3a-08fe-4fdd-9e83-792cab573778-registry-tls\") pod \"image-registry-6964ccb4d8-t226q\" (UID: \"d2839e3a-08fe-4fdd-9e83-792cab573778\") " pod="openshift-image-registry/image-registry-6964ccb4d8-t226q"
Jul 26 18:12:52.764646 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.764610    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d2839e3a-08fe-4fdd-9e83-792cab573778-installation-pull-secrets\") pod \"image-registry-6964ccb4d8-t226q\" (UID: \"d2839e3a-08fe-4fdd-9e83-792cab573778\") " pod="openshift-image-registry/image-registry-6964ccb4d8-t226q"
Jul 26 18:12:52.771221 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.770846    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d2839e3a-08fe-4fdd-9e83-792cab573778-bound-sa-token\") pod \"image-registry-6964ccb4d8-t226q\" (UID: \"d2839e3a-08fe-4fdd-9e83-792cab573778\") " pod="openshift-image-registry/image-registry-6964ccb4d8-t226q"
Jul 26 18:12:52.771221 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.771192    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqnjw\" (UniqueName: \"kubernetes.io/projected/d2839e3a-08fe-4fdd-9e83-792cab573778-kube-api-access-fqnjw\") pod \"image-registry-6964ccb4d8-t226q\" (UID: \"d2839e3a-08fe-4fdd-9e83-792cab573778\") " pod="openshift-image-registry/image-registry-6964ccb4d8-t226q"
Jul 26 18:12:52.821980 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.821948    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-88fdc7bd-pvssv"
Jul 26 18:12:52.831143 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.831113    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-568889d6c5-r6gfn"
Jul 26 18:12:52.838041 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.838020    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b99866955-6qc6m"
Jul 26 18:12:52.888455 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.888419    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:52.921120 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.921086    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6964ccb4d8-t226q"
Jul 26 18:12:52.982815 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:52.982775    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:54.551624 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:54.551595    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-568889d6c5-r6gfn"]
Jul 26 18:12:54.727791 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:12:54.724238    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dd206b4_b7eb_4a1c_8203_b17702b8a32e.slice/crio-a6b7a2483938ac5b8a583cdf04f040b7f6a37fe94e26bf20d1422005d04c28ae WatchSource:0}: Error finding container a6b7a2483938ac5b8a583cdf04f040b7f6a37fe94e26bf20d1422005d04c28ae: Status 404 returned error can't find the container with id a6b7a2483938ac5b8a583cdf04f040b7f6a37fe94e26bf20d1422005d04c28ae
Jul 26 18:12:54.769569 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:54.769547    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-88fdc7bd-pvssv"]
Jul 26 18:12:54.771868 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:12:54.771843    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb57a27f1_ce1b_4b47_851f_d981132c3441.slice/crio-0c58e4dc2edbc245cd8c4e80026aa5111bba06591824b13615c24c81ac6fffd9 WatchSource:0}: Error finding container 0c58e4dc2edbc245cd8c4e80026aa5111bba06591824b13615c24c81ac6fffd9: Status 404 returned error can't find the container with id 0c58e4dc2edbc245cd8c4e80026aa5111bba06591824b13615c24c81ac6fffd9
Jul 26 18:12:54.776373 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:54.776284    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b99866955-6qc6m"]
Jul 26 18:12:54.777766 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:54.777640    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6964ccb4d8-t226q"]
Jul 26 18:12:54.779522 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:12:54.779495    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod14f11537_8b51_41d9_a057_5df0d60dd970.slice/crio-edf1244a20b6c83ad84f3f7aec4d1b005557ea35de05708ce7bc413c473decf5 WatchSource:0}: Error finding container edf1244a20b6c83ad84f3f7aec4d1b005557ea35de05708ce7bc413c473decf5: Status 404 returned error can't find the container with id edf1244a20b6c83ad84f3f7aec4d1b005557ea35de05708ce7bc413c473decf5
Jul 26 18:12:54.780281 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:12:54.780190    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2839e3a_08fe_4fdd_9e83_792cab573778.slice/crio-5f73ee0410947220fb5fad698da6ae5d19700989343fa39743b16ed0e1d1563c WatchSource:0}: Error finding container 5f73ee0410947220fb5fad698da6ae5d19700989343fa39743b16ed0e1d1563c: Status 404 returned error can't find the container with id 5f73ee0410947220fb5fad698da6ae5d19700989343fa39743b16ed0e1d1563c
Jul 26 18:12:55.079661 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:55.079590    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5wfmw"
Jul 26 18:12:55.167576 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:55.167540    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-88fdc7bd-pvssv" event={"ID":"b57a27f1-ce1b-4b47-851f-d981132c3441","Type":"ContainerStarted","Data":"0c58e4dc2edbc245cd8c4e80026aa5111bba06591824b13615c24c81ac6fffd9"}
Jul 26 18:12:55.169487 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:55.169462    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7glwq" event={"ID":"9efa7a18-6a50-4d18-834a-8a50301009c2","Type":"ContainerStarted","Data":"d1a40c8bc72035c263d7d18ef1c4e9c74414e374c6ad68b93ea6e6612707e2ce"}
Jul 26 18:12:55.170616 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:55.170590    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b99866955-6qc6m" event={"ID":"14f11537-8b51-41d9-a057-5df0d60dd970","Type":"ContainerStarted","Data":"edf1244a20b6c83ad84f3f7aec4d1b005557ea35de05708ce7bc413c473decf5"}
Jul 26 18:12:55.172054 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:55.172027    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6964ccb4d8-t226q" event={"ID":"d2839e3a-08fe-4fdd-9e83-792cab573778","Type":"ContainerStarted","Data":"dc68e295db932b4ce58867b6717b74447cdffad3ec91764d1f8c1231fc12784c"}
Jul 26 18:12:55.172143 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:55.172064    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6964ccb4d8-t226q" event={"ID":"d2839e3a-08fe-4fdd-9e83-792cab573778","Type":"ContainerStarted","Data":"5f73ee0410947220fb5fad698da6ae5d19700989343fa39743b16ed0e1d1563c"}
Jul 26 18:12:55.172143 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:55.172122    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6964ccb4d8-t226q"
Jul 26 18:12:55.173241 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:55.173220    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-568889d6c5-r6gfn" event={"ID":"8dd206b4-b7eb-4a1c-8203-b17702b8a32e","Type":"ContainerStarted","Data":"a6b7a2483938ac5b8a583cdf04f040b7f6a37fe94e26bf20d1422005d04c28ae"}
Jul 26 18:12:55.193922 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:55.193868    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6964ccb4d8-t226q" podStartSLOduration=3.193850942 podStartE2EDuration="3.193850942s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:12:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:55.193467454 +0000 UTC m=+60.900874507" watchObservedRunningTime="2026-07-26 18:12:55.193850942 +0000 UTC m=+60.901257992"
Jul 26 18:12:55.201913 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:55.201880    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:55.261599 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:55.261565    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:57.327966 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:57.327878    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:57.429495 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:57.429461    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:57.810052 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:57.810017    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:57.890230 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:57.890172    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:58.197967 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:58.197932    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:58.270184 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:58.270149    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:58.368448 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:58.368384    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:58.433025 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:58.432953    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:58.588154 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:58.588073    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:58.683851 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:58.683804    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:58.737778 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:58.737733    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:59.153119 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:59.153090    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jjkbn"
Jul 26 18:12:59.366553 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:59.366514    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:59.426430 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:59.426395    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:59.826938 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:59.826847    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:12:59.888699 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:12:59.888665    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:00.529520 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:00.529472    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97230a04-09c6-4445-b713-98d115f6b1aa-metrics-certs\") pod \"network-metrics-daemon-fxp88\" (UID: \"97230a04-09c6-4445-b713-98d115f6b1aa\") " pod="openshift-multus/network-metrics-daemon-fxp88"
Jul 26 18:13:00.530084 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:00.529537    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/54acf4cd-cd75-4990-9ccf-274ef789daa5-original-pull-secret\") pod \"global-pull-secret-syncer-fsm57\" (UID: \"54acf4cd-cd75-4990-9ccf-274ef789daa5\") " pod="kube-system/global-pull-secret-syncer-fsm57"
Jul 26 18:13:00.532314 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:00.532291    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 26 18:13:00.533326 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:00.533304    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 26 18:13:00.542679 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:00.542654    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/54acf4cd-cd75-4990-9ccf-274ef789daa5-original-pull-secret\") pod \"global-pull-secret-syncer-fsm57\" (UID: \"54acf4cd-cd75-4990-9ccf-274ef789daa5\") " pod="kube-system/global-pull-secret-syncer-fsm57"
Jul 26 18:13:00.542843 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:00.542822    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97230a04-09c6-4445-b713-98d115f6b1aa-metrics-certs\") pod \"network-metrics-daemon-fxp88\" (UID: \"97230a04-09c6-4445-b713-98d115f6b1aa\") " pod="openshift-multus/network-metrics-daemon-fxp88"
Jul 26 18:13:00.630278 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:00.630227    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qzgk\" (UniqueName: \"kubernetes.io/projected/8a2c5810-fcc5-4224-8d19-5fe5cc832d5a-kube-api-access-7qzgk\") pod \"network-check-target-rrn49\" (UID: \"8a2c5810-fcc5-4224-8d19-5fe5cc832d5a\") " pod="openshift-network-diagnostics/network-check-target-rrn49"
Jul 26 18:13:00.633324 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:00.633281    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 26 18:13:00.643299 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:00.643271    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 26 18:13:00.654152 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:00.654127    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qzgk\" (UniqueName: \"kubernetes.io/projected/8a2c5810-fcc5-4224-8d19-5fe5cc832d5a-kube-api-access-7qzgk\") pod \"network-check-target-rrn49\" (UID: \"8a2c5810-fcc5-4224-8d19-5fe5cc832d5a\") " pod="openshift-network-diagnostics/network-check-target-rrn49"
Jul 26 18:13:00.756907 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:00.756879    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-l7f4l\""
Jul 26 18:13:00.760397 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:00.760347    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fsm57"
Jul 26 18:13:00.764276 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:00.764243    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rrn49"
Jul 26 18:13:00.768423 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:00.768405    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vf5dv\""
Jul 26 18:13:00.776808 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:00.776789    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxp88"
Jul 26 18:13:01.389968 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:01.389933    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:01.462912 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:01.462881    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:02.214868 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:02.214822    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:02.530066 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:02.529997    2576 patch_prober.go:28] interesting pod/image-registry-5957bd7985-8qzt8 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 26 18:13:02.530066 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:02.530051    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5957bd7985-8qzt8" podUID="02ea08e3-10e7-49f9-b932-32f5b7eeca14" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 26 18:13:02.673235 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:02.673200    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:02.731061 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:02.731020    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:03.391095 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:03.390989    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fxp88"]
Jul 26 18:13:03.396564 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:13:03.396511    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97230a04_09c6_4445_b713_98d115f6b1aa.slice/crio-d7ab794354833159e864aedafbac92390b2509feb16df6bfe47ec9e8d3302306 WatchSource:0}: Error finding container d7ab794354833159e864aedafbac92390b2509feb16df6bfe47ec9e8d3302306: Status 404 returned error can't find the container with id d7ab794354833159e864aedafbac92390b2509feb16df6bfe47ec9e8d3302306
Jul 26 18:13:03.413915 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:03.413847    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rrn49"]
Jul 26 18:13:03.418382 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:13:03.418349    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a2c5810_fcc5_4224_8d19_5fe5cc832d5a.slice/crio-dc546a74653ef41b7913dba0753f3ef4aa5d8b2b85e7ee05f21bf8b9601742c5 WatchSource:0}: Error finding container dc546a74653ef41b7913dba0753f3ef4aa5d8b2b85e7ee05f21bf8b9601742c5: Status 404 returned error can't find the container with id dc546a74653ef41b7913dba0753f3ef4aa5d8b2b85e7ee05f21bf8b9601742c5
Jul 26 18:13:03.430399 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:03.430372    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fsm57"]
Jul 26 18:13:03.433950 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:13:03.433885    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54acf4cd_cd75_4990_9ccf_274ef789daa5.slice/crio-26672eddfc382553b47c6e2530980f111fa1042ebae902d102c413abcc774e72 WatchSource:0}: Error finding container 26672eddfc382553b47c6e2530980f111fa1042ebae902d102c413abcc774e72: Status 404 returned error can't find the container with id 26672eddfc382553b47c6e2530980f111fa1042ebae902d102c413abcc774e72
Jul 26 18:13:04.219647 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:04.219606    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fsm57" event={"ID":"54acf4cd-cd75-4990-9ccf-274ef789daa5","Type":"ContainerStarted","Data":"26672eddfc382553b47c6e2530980f111fa1042ebae902d102c413abcc774e72"}
Jul 26 18:13:04.221796 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:04.221765    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-88fdc7bd-pvssv" event={"ID":"b57a27f1-ce1b-4b47-851f-d981132c3441","Type":"ContainerStarted","Data":"6645b79d0d2f68df6ceb695c9627ecc3747cdd207615c1e4924248b6b1237cfc"}
Jul 26 18:13:04.223297 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:04.223273    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fxp88" event={"ID":"97230a04-09c6-4445-b713-98d115f6b1aa","Type":"ContainerStarted","Data":"d7ab794354833159e864aedafbac92390b2509feb16df6bfe47ec9e8d3302306"}
Jul 26 18:13:04.226169 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:04.226130    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rrn49" event={"ID":"8a2c5810-fcc5-4224-8d19-5fe5cc832d5a","Type":"ContainerStarted","Data":"dc546a74653ef41b7913dba0753f3ef4aa5d8b2b85e7ee05f21bf8b9601742c5"}
Jul 26 18:13:04.229769 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:04.229664    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7glwq" event={"ID":"9efa7a18-6a50-4d18-834a-8a50301009c2","Type":"ContainerStarted","Data":"06b38875e9530aa9f6f9765ee24fa85b3b06419d12a8a9256b22400f00216844"}
Jul 26 18:13:04.233317 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:04.233292    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b99866955-6qc6m" event={"ID":"14f11537-8b51-41d9-a057-5df0d60dd970","Type":"ContainerStarted","Data":"82c06d49f7deba80fe9882062d6cc049de9ccb619eab14dc82dad576cea8a199"}
Jul 26 18:13:04.233422 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:04.233332    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b99866955-6qc6m" event={"ID":"14f11537-8b51-41d9-a057-5df0d60dd970","Type":"ContainerStarted","Data":"23fac29e651bd10e98f92b518fe8d1e5dc7bab146f99720fb96337ba6bbb1eb6"}
Jul 26 18:13:04.233422 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:04.233349    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b99866955-6qc6m" event={"ID":"14f11537-8b51-41d9-a057-5df0d60dd970","Type":"ContainerStarted","Data":"2d1184ba8414693c7d36abb3f4e033ab906c8ca859040dc695ec73e8d6d3e65c"}
Jul 26 18:13:04.236435 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:04.236093    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-568889d6c5-r6gfn" event={"ID":"8dd206b4-b7eb-4a1c-8203-b17702b8a32e","Type":"ContainerStarted","Data":"f0fe7caa9d3a88e0a5185b9988610b7c072c16c241240192addb6dc9b3b97068"}
Jul 26 18:13:04.236435 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:04.236407    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-568889d6c5-r6gfn"
Jul 26 18:13:04.240031 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:04.238880    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-568889d6c5-r6gfn"
Jul 26 18:13:04.240031 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:04.238873    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-88fdc7bd-pvssv" podStartSLOduration=3.790861966 podStartE2EDuration="12.238857425s" totalImagesPullingTime="8.447995459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:12:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:13:04.238675234 +0000 UTC m=+69.946082296" watchObservedRunningTime="2026-07-26 18:13:04.238857425 +0000 UTC m=+69.946264485"
Jul 26 18:13:04.257204 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:04.256833    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-568889d6c5-r6gfn" podStartSLOduration=3.7787742079999997 podStartE2EDuration="12.256810697s" totalImagesPullingTime="8.478036489s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:12:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:13:04.255663957 +0000 UTC m=+69.963071016" watchObservedRunningTime="2026-07-26 18:13:04.256810697 +0000 UTC m=+69.964217736"
Jul 26 18:13:04.274687 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:04.274045    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-7glwq" podStartSLOduration=17.999494143 podStartE2EDuration="35.274024608s" totalImagesPullingTime="17.274530465s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:12:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:13:04.273234276 +0000 UTC m=+69.980641335" watchObservedRunningTime="2026-07-26 18:13:04.274024608 +0000 UTC m=+69.981431666"
Jul 26 18:13:04.292911 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:04.292785    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b99866955-6qc6m" podStartSLOduration=3.831521757 podStartE2EDuration="12.292762966s" totalImagesPullingTime="8.461241209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:12:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:13:04.292177888 +0000 UTC m=+69.999584946" watchObservedRunningTime="2026-07-26 18:13:04.292762966 +0000 UTC m=+70.000170024"
Jul 26 18:13:04.391880 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:04.391841    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:04.455926 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:04.455890    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:05.242907 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:05.242865    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fxp88" event={"ID":"97230a04-09c6-4445-b713-98d115f6b1aa","Type":"ContainerStarted","Data":"235f2e9c72221a534d260ab9c49f72c0ef6414cf0fa7696a3e686d3d58d516df"}
Jul 26 18:13:05.243087 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:05.242964    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fxp88" event={"ID":"97230a04-09c6-4445-b713-98d115f6b1aa","Type":"ContainerStarted","Data":"b66bff4ccf528a2c584abe3a911f8a7d6647178770ef05ea3864bd3c9ac78ffc"}
Jul 26 18:13:05.260029 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:05.259970    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-fxp88" podStartSLOduration=68.790904805 podStartE2EDuration="1m10.259951172s" totalImagesPullingTime="1.469046367s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:11:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:13:05.258619609 +0000 UTC m=+70.966026669" watchObservedRunningTime="2026-07-26 18:13:05.259951172 +0000 UTC m=+70.967358233"
Jul 26 18:13:05.457825 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:05.457791    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:05.529961 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:05.529924    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:06.203871 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:06.203833    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:08.256048 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:08.256003    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rrn49" event={"ID":"8a2c5810-fcc5-4224-8d19-5fe5cc832d5a","Type":"ContainerStarted","Data":"14da1b44ecc5ff1f98f09098ca0defcd3c58894ca3974b4b2e44e43602531454"}
Jul 26 18:13:08.256485 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:08.256065    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rrn49"
Jul 26 18:13:08.257513 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:08.257488    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fsm57" event={"ID":"54acf4cd-cd75-4990-9ccf-274ef789daa5","Type":"ContainerStarted","Data":"e1545f1cf7e7cfee5579c4e2bf95f828384b54b41b723c83c4801f7e404195a3"}
Jul 26 18:13:08.272946 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:08.272906    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-rrn49" podStartSLOduration=68.564269527 podStartE2EDuration="1m13.272891328s" totalImagesPullingTime="4.708621801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:11:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:13:08.271673634 +0000 UTC m=+73.979080693" watchObservedRunningTime="2026-07-26 18:13:08.272891328 +0000 UTC m=+73.980298386"
Jul 26 18:13:08.284336 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:08.284305    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:08.287256 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:08.287216    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-fsm57" podStartSLOduration=68.589573012 podStartE2EDuration="1m13.287199495s" totalImagesPullingTime="4.697626483s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:11:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:13:08.286517585 +0000 UTC m=+73.993924646" watchObservedRunningTime="2026-07-26 18:13:08.287199495 +0000 UTC m=+73.994606552"
Jul 26 18:13:08.385076 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:08.384999    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:10.820388 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:10.820349    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:11.029547 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:11.029503    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:11.113045 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:11.112926    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:11.408228 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:11.408142    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:11.508911 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:11.508877    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:12.333838 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:12.333805    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:12.432362 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:12.432322    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:12.526888 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:12.526860    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5957bd7985-8qzt8"
Jul 26 18:13:12.533399 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:12.533377    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:13.533270 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:13.533226    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:13.630816 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:13.630777    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:13.744122 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:13.744094    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:13.789248 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:13.789155    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-cdmwl"]
Jul 26 18:13:13.791918 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:13.791897    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cdmwl"
Jul 26 18:13:13.795393 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:13.795370    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 26 18:13:13.795613 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:13.795398    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-g7w8p\""
Jul 26 18:13:13.795687 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:13.795420    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 26 18:13:13.795766 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:13.795504    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 26 18:13:13.796598 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:13.796470    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 26 18:13:13.796598 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:13.796481    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 26 18:13:13.796598 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:13.796491    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 26 18:13:13.847550 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:13.847482    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:13.933351 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:13.933317    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49qbp\" (UniqueName: \"kubernetes.io/projected/cfc21e49-9040-4bbe-8a78-a4717cfe3e92-kube-api-access-49qbp\") pod \"node-exporter-cdmwl\" (UID: \"cfc21e49-9040-4bbe-8a78-a4717cfe3e92\") " pod="openshift-monitoring/node-exporter-cdmwl"
Jul 26 18:13:13.933492 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:13.933368    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cfc21e49-9040-4bbe-8a78-a4717cfe3e92-node-exporter-tls\") pod \"node-exporter-cdmwl\" (UID: \"cfc21e49-9040-4bbe-8a78-a4717cfe3e92\") " pod="openshift-monitoring/node-exporter-cdmwl"
Jul 26 18:13:13.933492 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:13.933393    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cfc21e49-9040-4bbe-8a78-a4717cfe3e92-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cdmwl\" (UID: \"cfc21e49-9040-4bbe-8a78-a4717cfe3e92\") " pod="openshift-monitoring/node-exporter-cdmwl"
Jul 26 18:13:13.933492 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:13.933437    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cfc21e49-9040-4bbe-8a78-a4717cfe3e92-node-exporter-wtmp\") pod \"node-exporter-cdmwl\" (UID: \"cfc21e49-9040-4bbe-8a78-a4717cfe3e92\") " pod="openshift-monitoring/node-exporter-cdmwl"
Jul 26 18:13:13.933492 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:13.933460    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cfc21e49-9040-4bbe-8a78-a4717cfe3e92-node-exporter-accelerators-collector-config\") pod \"node-exporter-cdmwl\" (UID: \"cfc21e49-9040-4bbe-8a78-a4717cfe3e92\") " pod="openshift-monitoring/node-exporter-cdmwl"
Jul 26 18:13:13.933492 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:13.933479    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cfc21e49-9040-4bbe-8a78-a4717cfe3e92-root\") pod \"node-exporter-cdmwl\" (UID: \"cfc21e49-9040-4bbe-8a78-a4717cfe3e92\") " pod="openshift-monitoring/node-exporter-cdmwl"
Jul 26 18:13:13.933701 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:13.933498    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cfc21e49-9040-4bbe-8a78-a4717cfe3e92-node-exporter-textfile\") pod \"node-exporter-cdmwl\" (UID: \"cfc21e49-9040-4bbe-8a78-a4717cfe3e92\") " pod="openshift-monitoring/node-exporter-cdmwl"
Jul 26 18:13:13.933701 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:13.933551    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cfc21e49-9040-4bbe-8a78-a4717cfe3e92-sys\") pod \"node-exporter-cdmwl\" (UID: \"cfc21e49-9040-4bbe-8a78-a4717cfe3e92\") " pod="openshift-monitoring/node-exporter-cdmwl"
Jul 26 18:13:13.933701 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:13.933593    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cfc21e49-9040-4bbe-8a78-a4717cfe3e92-metrics-client-ca\") pod \"node-exporter-cdmwl\" (UID: \"cfc21e49-9040-4bbe-8a78-a4717cfe3e92\") " pod="openshift-monitoring/node-exporter-cdmwl"
Jul 26 18:13:13.987227 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:13.987192    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:14.034214 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:14.034185    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cfc21e49-9040-4bbe-8a78-a4717cfe3e92-metrics-client-ca\") pod \"node-exporter-cdmwl\" (UID: \"cfc21e49-9040-4bbe-8a78-a4717cfe3e92\") " pod="openshift-monitoring/node-exporter-cdmwl"
Jul 26 18:13:14.034385 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:14.034227    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49qbp\" (UniqueName: \"kubernetes.io/projected/cfc21e49-9040-4bbe-8a78-a4717cfe3e92-kube-api-access-49qbp\") pod \"node-exporter-cdmwl\" (UID: \"cfc21e49-9040-4bbe-8a78-a4717cfe3e92\") " pod="openshift-monitoring/node-exporter-cdmwl"
Jul 26 18:13:14.034385 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:14.034249    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cfc21e49-9040-4bbe-8a78-a4717cfe3e92-node-exporter-tls\") pod \"node-exporter-cdmwl\" (UID: \"cfc21e49-9040-4bbe-8a78-a4717cfe3e92\") " pod="openshift-monitoring/node-exporter-cdmwl"
Jul 26 18:13:14.034385 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:14.034265    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cfc21e49-9040-4bbe-8a78-a4717cfe3e92-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cdmwl\" (UID: \"cfc21e49-9040-4bbe-8a78-a4717cfe3e92\") " pod="openshift-monitoring/node-exporter-cdmwl"
Jul 26 18:13:14.034385 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:14.034292    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cfc21e49-9040-4bbe-8a78-a4717cfe3e92-node-exporter-wtmp\") pod \"node-exporter-cdmwl\" (UID: \"cfc21e49-9040-4bbe-8a78-a4717cfe3e92\") " pod="openshift-monitoring/node-exporter-cdmwl"
Jul 26 18:13:14.034385 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:14.034331    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cfc21e49-9040-4bbe-8a78-a4717cfe3e92-node-exporter-accelerators-collector-config\") pod \"node-exporter-cdmwl\" (UID: \"cfc21e49-9040-4bbe-8a78-a4717cfe3e92\") " pod="openshift-monitoring/node-exporter-cdmwl"
Jul 26 18:13:14.034385 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:14.034351    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cfc21e49-9040-4bbe-8a78-a4717cfe3e92-root\") pod \"node-exporter-cdmwl\" (UID: \"cfc21e49-9040-4bbe-8a78-a4717cfe3e92\") " pod="openshift-monitoring/node-exporter-cdmwl"
Jul 26 18:13:14.034385 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:14.034384    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cfc21e49-9040-4bbe-8a78-a4717cfe3e92-node-exporter-textfile\") pod \"node-exporter-cdmwl\" (UID: \"cfc21e49-9040-4bbe-8a78-a4717cfe3e92\") " pod="openshift-monitoring/node-exporter-cdmwl"
Jul 26 18:13:14.034732 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:14.034407    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cfc21e49-9040-4bbe-8a78-a4717cfe3e92-sys\") pod \"node-exporter-cdmwl\" (UID: \"cfc21e49-9040-4bbe-8a78-a4717cfe3e92\") " pod="openshift-monitoring/node-exporter-cdmwl"
Jul 26 18:13:14.034732 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:13:14.034432    2576 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 26 18:13:14.034732 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:14.034471    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cfc21e49-9040-4bbe-8a78-a4717cfe3e92-sys\") pod \"node-exporter-cdmwl\" (UID: \"cfc21e49-9040-4bbe-8a78-a4717cfe3e92\") " pod="openshift-monitoring/node-exporter-cdmwl"
Jul 26 18:13:14.034732 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:13:14.034504    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfc21e49-9040-4bbe-8a78-a4717cfe3e92-node-exporter-tls podName:cfc21e49-9040-4bbe-8a78-a4717cfe3e92 nodeName:}" failed. No retries permitted until 2026-07-26 18:13:14.534482285 +0000 UTC m=+80.241889330 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/cfc21e49-9040-4bbe-8a78-a4717cfe3e92-node-exporter-tls") pod "node-exporter-cdmwl" (UID: "cfc21e49-9040-4bbe-8a78-a4717cfe3e92") : secret "node-exporter-tls" not found
Jul 26 18:13:14.034971 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:14.034780    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cfc21e49-9040-4bbe-8a78-a4717cfe3e92-root\") pod \"node-exporter-cdmwl\" (UID: \"cfc21e49-9040-4bbe-8a78-a4717cfe3e92\") " pod="openshift-monitoring/node-exporter-cdmwl"
Jul 26 18:13:14.034971 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:14.034906    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cfc21e49-9040-4bbe-8a78-a4717cfe3e92-metrics-client-ca\") pod \"node-exporter-cdmwl\" (UID: \"cfc21e49-9040-4bbe-8a78-a4717cfe3e92\") " pod="openshift-monitoring/node-exporter-cdmwl"
Jul 26 18:13:14.034971 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:14.034919    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cfc21e49-9040-4bbe-8a78-a4717cfe3e92-node-exporter-wtmp\") pod \"node-exporter-cdmwl\" (UID: \"cfc21e49-9040-4bbe-8a78-a4717cfe3e92\") " pod="openshift-monitoring/node-exporter-cdmwl"
Jul 26 18:13:14.035096 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:14.035073    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cfc21e49-9040-4bbe-8a78-a4717cfe3e92-node-exporter-accelerators-collector-config\") pod \"node-exporter-cdmwl\" (UID: \"cfc21e49-9040-4bbe-8a78-a4717cfe3e92\") " pod="openshift-monitoring/node-exporter-cdmwl"
Jul 26 18:13:14.035135 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:14.035045    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cfc21e49-9040-4bbe-8a78-a4717cfe3e92-node-exporter-textfile\") pod \"node-exporter-cdmwl\" (UID: \"cfc21e49-9040-4bbe-8a78-a4717cfe3e92\") " pod="openshift-monitoring/node-exporter-cdmwl"
Jul 26 18:13:14.036786 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:14.036741    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cfc21e49-9040-4bbe-8a78-a4717cfe3e92-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cdmwl\" (UID: \"cfc21e49-9040-4bbe-8a78-a4717cfe3e92\") " pod="openshift-monitoring/node-exporter-cdmwl"
Jul 26 18:13:14.042531 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:14.042477    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49qbp\" (UniqueName: \"kubernetes.io/projected/cfc21e49-9040-4bbe-8a78-a4717cfe3e92-kube-api-access-49qbp\") pod \"node-exporter-cdmwl\" (UID: \"cfc21e49-9040-4bbe-8a78-a4717cfe3e92\") " pod="openshift-monitoring/node-exporter-cdmwl"
Jul 26 18:13:14.126110 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:14.126073    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:14.226965 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:14.226928    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:14.326761 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:14.326646    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:14.439133 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:14.439099    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:14.537765 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:14.537718    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cfc21e49-9040-4bbe-8a78-a4717cfe3e92-node-exporter-tls\") pod \"node-exporter-cdmwl\" (UID: \"cfc21e49-9040-4bbe-8a78-a4717cfe3e92\") " pod="openshift-monitoring/node-exporter-cdmwl"
Jul 26 18:13:14.540321 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:14.540299    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cfc21e49-9040-4bbe-8a78-a4717cfe3e92-node-exporter-tls\") pod \"node-exporter-cdmwl\" (UID: \"cfc21e49-9040-4bbe-8a78-a4717cfe3e92\") " pod="openshift-monitoring/node-exporter-cdmwl"
Jul 26 18:13:14.702182 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:14.702155    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cdmwl"
Jul 26 18:13:14.712986 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:13:14.712962    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfc21e49_9040_4bbe_8a78_a4717cfe3e92.slice/crio-e0bdff0e7b757370552c5fd9d59b2afe44300ab8988358cf30de70637720778d WatchSource:0}: Error finding container e0bdff0e7b757370552c5fd9d59b2afe44300ab8988358cf30de70637720778d: Status 404 returned error can't find the container with id e0bdff0e7b757370552c5fd9d59b2afe44300ab8988358cf30de70637720778d
Jul 26 18:13:14.863918 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:14.863890    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:14.974705 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:14.974621    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:15.059066 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:15.059033    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:15.282646 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:15.282564    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cdmwl" event={"ID":"cfc21e49-9040-4bbe-8a78-a4717cfe3e92","Type":"ContainerStarted","Data":"e0bdff0e7b757370552c5fd9d59b2afe44300ab8988358cf30de70637720778d"}
Jul 26 18:13:16.185591 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:16.185564    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6964ccb4d8-t226q"
Jul 26 18:13:16.288487 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:16.288452    2576 generic.go:356] "Generic (PLEG): container finished" podID="cfc21e49-9040-4bbe-8a78-a4717cfe3e92" containerID="40e9bcacc7f7a28b3a972ab24dcb8af32abf60aee795f377a6db246f45f9986d" exitCode=0
Jul 26 18:13:16.288714 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:16.288503    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cdmwl" event={"ID":"cfc21e49-9040-4bbe-8a78-a4717cfe3e92","Type":"ContainerDied","Data":"40e9bcacc7f7a28b3a972ab24dcb8af32abf60aee795f377a6db246f45f9986d"}
Jul 26 18:13:16.446782 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:16.446672    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:16.528985 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:16.528949    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:16.600526 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:16.600497    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:16.895930 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:16.895839    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:16.996202 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:16.996172    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:17.097713 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:17.097596    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:17.204685 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:17.204654    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:17.294218 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:17.294183    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cdmwl" event={"ID":"cfc21e49-9040-4bbe-8a78-a4717cfe3e92","Type":"ContainerStarted","Data":"4d5c7ecf2820de8433db171e6f11aba3a1488981864c7833dc5f0508182d95d7"}
Jul 26 18:13:17.294384 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:17.294244    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cdmwl" event={"ID":"cfc21e49-9040-4bbe-8a78-a4717cfe3e92","Type":"ContainerStarted","Data":"d25505816aaa3452efcd35109448d5886be456481f2d245e4ad972eb6e041ca5"}
Jul 26 18:13:17.295868 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:17.295842    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:17.396693 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:17.396657    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:17.543158 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:17.543065    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5957bd7985-8qzt8" podUID="02ea08e3-10e7-49f9-b932-32f5b7eeca14" containerName="registry" containerID="cri-o://49fdc223e747125b0b4cee45d5758b81ebbf5d7e91966982c8fb8f51c480fb77" gracePeriod=30
Jul 26 18:13:17.774634 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:17.774612    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5957bd7985-8qzt8"
Jul 26 18:13:17.796826 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:17.796725    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-cdmwl" podStartSLOduration=3.8875360839999997 podStartE2EDuration="4.796707106s" totalImagesPullingTime="846.062816ms" totalInitContainerRuntime="63.108206ms" isStatefulPod=true podCreationTimestamp="2026-07-26 18:13:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:13:17.315072036 +0000 UTC m=+83.022479094" watchObservedRunningTime="2026-07-26 18:13:17.796707106 +0000 UTC m=+83.504114165"
Jul 26 18:13:17.862145 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:17.862107    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/02ea08e3-10e7-49f9-b932-32f5b7eeca14-installation-pull-secrets\") pod \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\" (UID: \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\") "
Jul 26 18:13:17.862145 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:17.862154    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rb5w6\" (UniqueName: \"kubernetes.io/projected/02ea08e3-10e7-49f9-b932-32f5b7eeca14-kube-api-access-rb5w6\") pod \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\" (UID: \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\") "
Jul 26 18:13:17.862393 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:17.862174    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/02ea08e3-10e7-49f9-b932-32f5b7eeca14-trusted-ca\") pod \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\" (UID: \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\") "
Jul 26 18:13:17.862393 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:17.862188    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/02ea08e3-10e7-49f9-b932-32f5b7eeca14-bound-sa-token\") pod \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\" (UID: \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\") "
Jul 26 18:13:17.862393 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:17.862210    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/02ea08e3-10e7-49f9-b932-32f5b7eeca14-registry-tls\") pod \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\" (UID: \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\") "
Jul 26 18:13:17.862393 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:17.862270    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/02ea08e3-10e7-49f9-b932-32f5b7eeca14-ca-trust-extracted\") pod \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\" (UID: \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\") "
Jul 26 18:13:17.862393 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:17.862304    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/02ea08e3-10e7-49f9-b932-32f5b7eeca14-registry-certificates\") pod \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\" (UID: \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\") "
Jul 26 18:13:17.862393 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:17.862365    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/02ea08e3-10e7-49f9-b932-32f5b7eeca14-image-registry-private-configuration\") pod \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\" (UID: \"02ea08e3-10e7-49f9-b932-32f5b7eeca14\") "
Jul 26 18:13:17.862701 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:17.862665    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02ea08e3-10e7-49f9-b932-32f5b7eeca14-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "02ea08e3-10e7-49f9-b932-32f5b7eeca14" (UID: "02ea08e3-10e7-49f9-b932-32f5b7eeca14"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:13:17.862970 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:17.862928    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02ea08e3-10e7-49f9-b932-32f5b7eeca14-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "02ea08e3-10e7-49f9-b932-32f5b7eeca14" (UID: "02ea08e3-10e7-49f9-b932-32f5b7eeca14"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:13:17.864655 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:17.864625    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02ea08e3-10e7-49f9-b932-32f5b7eeca14-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "02ea08e3-10e7-49f9-b932-32f5b7eeca14" (UID: "02ea08e3-10e7-49f9-b932-32f5b7eeca14"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:13:17.865015 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:17.864988    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02ea08e3-10e7-49f9-b932-32f5b7eeca14-kube-api-access-rb5w6" (OuterVolumeSpecName: "kube-api-access-rb5w6") pod "02ea08e3-10e7-49f9-b932-32f5b7eeca14" (UID: "02ea08e3-10e7-49f9-b932-32f5b7eeca14"). InnerVolumeSpecName "kube-api-access-rb5w6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:13:17.865128 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:17.864991    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02ea08e3-10e7-49f9-b932-32f5b7eeca14-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "02ea08e3-10e7-49f9-b932-32f5b7eeca14" (UID: "02ea08e3-10e7-49f9-b932-32f5b7eeca14"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:13:17.865128 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:17.865026    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02ea08e3-10e7-49f9-b932-32f5b7eeca14-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "02ea08e3-10e7-49f9-b932-32f5b7eeca14" (UID: "02ea08e3-10e7-49f9-b932-32f5b7eeca14"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:13:17.865128 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:17.865067    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02ea08e3-10e7-49f9-b932-32f5b7eeca14-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "02ea08e3-10e7-49f9-b932-32f5b7eeca14" (UID: "02ea08e3-10e7-49f9-b932-32f5b7eeca14"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:13:17.867950 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:17.867922    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:17.871243 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:17.871220    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02ea08e3-10e7-49f9-b932-32f5b7eeca14-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "02ea08e3-10e7-49f9-b932-32f5b7eeca14" (UID: "02ea08e3-10e7-49f9-b932-32f5b7eeca14"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:13:17.963041 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:17.963000    2576 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/02ea08e3-10e7-49f9-b932-32f5b7eeca14-ca-trust-extracted\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:13:17.963041 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:17.963036    2576 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/02ea08e3-10e7-49f9-b932-32f5b7eeca14-registry-certificates\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:13:17.963041 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:17.963049    2576 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/02ea08e3-10e7-49f9-b932-32f5b7eeca14-image-registry-private-configuration\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:13:17.963241 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:17.963059    2576 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/02ea08e3-10e7-49f9-b932-32f5b7eeca14-installation-pull-secrets\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:13:17.963241 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:17.963067    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rb5w6\" (UniqueName: \"kubernetes.io/projected/02ea08e3-10e7-49f9-b932-32f5b7eeca14-kube-api-access-rb5w6\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:13:17.963241 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:17.963078    2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/02ea08e3-10e7-49f9-b932-32f5b7eeca14-trusted-ca\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:13:17.963241 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:17.963086    2576 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/02ea08e3-10e7-49f9-b932-32f5b7eeca14-bound-sa-token\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:13:17.963241 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:17.963094    2576 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/02ea08e3-10e7-49f9-b932-32f5b7eeca14-registry-tls\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:13:17.969598 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:17.969569    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:18.023054 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:18.023026    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:18.299246 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:18.299211    2576 generic.go:356] "Generic (PLEG): container finished" podID="02ea08e3-10e7-49f9-b932-32f5b7eeca14" containerID="49fdc223e747125b0b4cee45d5758b81ebbf5d7e91966982c8fb8f51c480fb77" exitCode=0
Jul 26 18:13:18.299667 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:18.299255    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5957bd7985-8qzt8"
Jul 26 18:13:18.299667 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:18.299298    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5957bd7985-8qzt8" event={"ID":"02ea08e3-10e7-49f9-b932-32f5b7eeca14","Type":"ContainerDied","Data":"49fdc223e747125b0b4cee45d5758b81ebbf5d7e91966982c8fb8f51c480fb77"}
Jul 26 18:13:18.299667 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:18.299347    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5957bd7985-8qzt8" event={"ID":"02ea08e3-10e7-49f9-b932-32f5b7eeca14","Type":"ContainerDied","Data":"228a9c964772625bc2e5b05de50c84e94e8a325b973c060548c7ecb61c83c071"}
Jul 26 18:13:18.299667 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:18.299371    2576 scope.go:117] "RemoveContainer" containerID="49fdc223e747125b0b4cee45d5758b81ebbf5d7e91966982c8fb8f51c480fb77"
Jul 26 18:13:18.307663 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:18.307648    2576 scope.go:117] "RemoveContainer" containerID="49fdc223e747125b0b4cee45d5758b81ebbf5d7e91966982c8fb8f51c480fb77"
Jul 26 18:13:18.307964 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:13:18.307944    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49fdc223e747125b0b4cee45d5758b81ebbf5d7e91966982c8fb8f51c480fb77\": container with ID starting with 49fdc223e747125b0b4cee45d5758b81ebbf5d7e91966982c8fb8f51c480fb77 not found: ID does not exist" containerID="49fdc223e747125b0b4cee45d5758b81ebbf5d7e91966982c8fb8f51c480fb77"
Jul 26 18:13:18.308026 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:18.307973    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49fdc223e747125b0b4cee45d5758b81ebbf5d7e91966982c8fb8f51c480fb77"} err="failed to get container status \"49fdc223e747125b0b4cee45d5758b81ebbf5d7e91966982c8fb8f51c480fb77\": rpc error: code = NotFound desc = could not find container \"49fdc223e747125b0b4cee45d5758b81ebbf5d7e91966982c8fb8f51c480fb77\": container with ID starting with 49fdc223e747125b0b4cee45d5758b81ebbf5d7e91966982c8fb8f51c480fb77 not found: ID does not exist"
Jul 26 18:13:18.320566 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:18.320544    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5957bd7985-8qzt8"]
Jul 26 18:13:18.324900 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:18.324882    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5957bd7985-8qzt8"]
Jul 26 18:13:18.585865 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:18.585791    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:18.646685 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:18.646652    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:18.749128 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:18.749078    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:18.820070 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:18.820040    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:18.847345 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:18.847280    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02ea08e3-10e7-49f9-b932-32f5b7eeca14" path="/var/lib/kubelet/pods/02ea08e3-10e7-49f9-b932-32f5b7eeca14/volumes"
Jul 26 18:13:19.891200 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:19.891160    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:19.991078 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:19.991003    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:20.195829 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:20.195791    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:20.301462 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:20.301424    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:21.228287 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:21.228251    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:21.299581 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:21.299531    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:27.322755 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:27.322716    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:27.423133 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:27.423091    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:27.586549 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:27.586461    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:27.665118 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:27.665075    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:30.351420 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:30.351386    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:30.431657 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:30.431619    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:30.833481 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:30.833444    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:30.931686 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:30.931647    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:31.041882 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:31.041849    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:39.263218 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:39.263182    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-rrn49"
Jul 26 18:13:42.335020 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:42.334977    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:42.401405 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:42.401372    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:49.564250 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:49.564218    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:13:56.114203 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:13:56.114164    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:14:33.300509 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:14:33.300461    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:14:33.800005 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:14:33.799966    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:14:43.895308 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:14:43.895271    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:14:43.955827 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:14:43.955789    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:14:48.539388 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:14:48.539284    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:14:48.595928 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:14:48.595892    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:15:00.550154 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:15:00.550112    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:15:00.604125 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:15:00.604087    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:15:00.900998 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:15:00.900905    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:15:01.027481 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:15:01.027404    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:15:01.115506 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:15:01.115472    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:15:12.330116 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:15:12.330044    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:15:12.431420 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:15:12.431387    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:15:12.494040 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:15:12.493999    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:15:13.112203 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:15:13.112169    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:15:13.172652 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:15:13.172616    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:16:04.655100 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:16:04.655065    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:16:04.752904 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:16:04.752871    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:16:22.377671 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:16:22.377587    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:16:22.441877 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:16:22.441846    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:16:26.514008 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:16:26.513975    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:16:27.898560 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:16:27.898517    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:16:30.132012 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:16:30.131943    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:16:30.229434 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:16:30.229400    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:16:32.299832 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:16:32.299792    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:16:32.498703 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:16:32.498667    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:16:32.851763 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:16:32.851711    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:16:32.986961 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:16:32.986928    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:16:34.556150 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:16:34.556111    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:16:35.933967 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:16:35.933937    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:16:36.918633 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:16:36.918526    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:16:37.019100 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:16:37.019063    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:16:39.565706 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:16:39.565670    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:16:39.655795 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:16:39.655769    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:16:42.184909 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:16:42.184870    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:16:44.353648 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:16:44.353614    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:16:44.447981 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:16:44.447950    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:16:51.188221 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:16:51.188180    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:16:51.291266 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:16:51.291232    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:16:53.035482 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:16:53.035448    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:16:54.713871 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:16:54.713843    2576 kubelet.go:1648] "Image garbage collection succeeded"
Jul 26 18:17:03.608875 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:03.608830    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:04.935656 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:04.935607    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:05.039013 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:05.038975    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:05.571032 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:05.570989    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:06.081580 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:06.081545    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:06.188962 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:06.188922    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:06.207766 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:06.207716    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-758747dbfc-2l78q"]
Jul 26 18:17:06.207971 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:06.207958    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02ea08e3-10e7-49f9-b932-32f5b7eeca14" containerName="registry"
Jul 26 18:17:06.208017 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:06.207973    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="02ea08e3-10e7-49f9-b932-32f5b7eeca14" containerName="registry"
Jul 26 18:17:06.208050 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:06.208024    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="02ea08e3-10e7-49f9-b932-32f5b7eeca14" containerName="registry"
Jul 26 18:17:06.210561 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:06.210547    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-2l78q"
Jul 26 18:17:06.217867 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:06.217845    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 26 18:17:06.217976 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:06.217853    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 26 18:17:06.217976 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:06.217940    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 26 18:17:06.220033 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:06.220015    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 26 18:17:06.220308 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:06.220242    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-zw2hr\""
Jul 26 18:17:06.226693 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:06.226672    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-758747dbfc-2l78q"]
Jul 26 18:17:06.379667 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:06.379575    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvj96\" (UniqueName: \"kubernetes.io/projected/204bb694-4d92-4d85-b6d0-863fe7b98ca0-kube-api-access-gvj96\") pod \"opendatahub-operator-controller-manager-758747dbfc-2l78q\" (UID: \"204bb694-4d92-4d85-b6d0-863fe7b98ca0\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-2l78q"
Jul 26 18:17:06.379667 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:06.379631    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/204bb694-4d92-4d85-b6d0-863fe7b98ca0-tmp\") pod \"opendatahub-operator-controller-manager-758747dbfc-2l78q\" (UID: \"204bb694-4d92-4d85-b6d0-863fe7b98ca0\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-2l78q"
Jul 26 18:17:06.379894 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:06.379689    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/204bb694-4d92-4d85-b6d0-863fe7b98ca0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-758747dbfc-2l78q\" (UID: \"204bb694-4d92-4d85-b6d0-863fe7b98ca0\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-2l78q"
Jul 26 18:17:06.379894 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:06.379725    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/204bb694-4d92-4d85-b6d0-863fe7b98ca0-webhook-cert\") pod \"opendatahub-operator-controller-manager-758747dbfc-2l78q\" (UID: \"204bb694-4d92-4d85-b6d0-863fe7b98ca0\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-2l78q"
Jul 26 18:17:06.379894 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:06.379796    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/204bb694-4d92-4d85-b6d0-863fe7b98ca0-manifests\") pod \"opendatahub-operator-controller-manager-758747dbfc-2l78q\" (UID: \"204bb694-4d92-4d85-b6d0-863fe7b98ca0\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-2l78q"
Jul 26 18:17:06.481078 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:06.481040    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/204bb694-4d92-4d85-b6d0-863fe7b98ca0-tmp\") pod \"opendatahub-operator-controller-manager-758747dbfc-2l78q\" (UID: \"204bb694-4d92-4d85-b6d0-863fe7b98ca0\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-2l78q"
Jul 26 18:17:06.481078 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:06.481081    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/204bb694-4d92-4d85-b6d0-863fe7b98ca0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-758747dbfc-2l78q\" (UID: \"204bb694-4d92-4d85-b6d0-863fe7b98ca0\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-2l78q"
Jul 26 18:17:06.481275 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:06.481102    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/204bb694-4d92-4d85-b6d0-863fe7b98ca0-webhook-cert\") pod \"opendatahub-operator-controller-manager-758747dbfc-2l78q\" (UID: \"204bb694-4d92-4d85-b6d0-863fe7b98ca0\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-2l78q"
Jul 26 18:17:06.481275 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:06.481147    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/204bb694-4d92-4d85-b6d0-863fe7b98ca0-manifests\") pod \"opendatahub-operator-controller-manager-758747dbfc-2l78q\" (UID: \"204bb694-4d92-4d85-b6d0-863fe7b98ca0\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-2l78q"
Jul 26 18:17:06.481275 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:06.481171    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvj96\" (UniqueName: \"kubernetes.io/projected/204bb694-4d92-4d85-b6d0-863fe7b98ca0-kube-api-access-gvj96\") pod \"opendatahub-operator-controller-manager-758747dbfc-2l78q\" (UID: \"204bb694-4d92-4d85-b6d0-863fe7b98ca0\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-2l78q"
Jul 26 18:17:06.481578 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:06.481558    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/204bb694-4d92-4d85-b6d0-863fe7b98ca0-manifests\") pod \"opendatahub-operator-controller-manager-758747dbfc-2l78q\" (UID: \"204bb694-4d92-4d85-b6d0-863fe7b98ca0\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-2l78q"
Jul 26 18:17:06.483457 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:06.483418    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/204bb694-4d92-4d85-b6d0-863fe7b98ca0-tmp\") pod \"opendatahub-operator-controller-manager-758747dbfc-2l78q\" (UID: \"204bb694-4d92-4d85-b6d0-863fe7b98ca0\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-2l78q"
Jul 26 18:17:06.483575 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:06.483472    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/204bb694-4d92-4d85-b6d0-863fe7b98ca0-webhook-cert\") pod \"opendatahub-operator-controller-manager-758747dbfc-2l78q\" (UID: \"204bb694-4d92-4d85-b6d0-863fe7b98ca0\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-2l78q"
Jul 26 18:17:06.483618 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:06.483584    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/204bb694-4d92-4d85-b6d0-863fe7b98ca0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-758747dbfc-2l78q\" (UID: \"204bb694-4d92-4d85-b6d0-863fe7b98ca0\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-2l78q"
Jul 26 18:17:06.486609 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:06.486586    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:06.495716 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:06.495691    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvj96\" (UniqueName: \"kubernetes.io/projected/204bb694-4d92-4d85-b6d0-863fe7b98ca0-kube-api-access-gvj96\") pod \"opendatahub-operator-controller-manager-758747dbfc-2l78q\" (UID: \"204bb694-4d92-4d85-b6d0-863fe7b98ca0\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-2l78q"
Jul 26 18:17:06.520563 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:06.520536    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-2l78q"
Jul 26 18:17:06.578849 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:06.578817    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:06.654281 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:06.654184    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-758747dbfc-2l78q"]
Jul 26 18:17:06.656720 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:06.656689    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:06.657299 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:17:06.657277    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod204bb694_4d92_4d85_b6d0_863fe7b98ca0.slice/crio-194dddd0ea03b2765699fa5f26c8bb6ef07db4a2ad18d7f18523886a2a0a174b WatchSource:0}: Error finding container 194dddd0ea03b2765699fa5f26c8bb6ef07db4a2ad18d7f18523886a2a0a174b: Status 404 returned error can't find the container with id 194dddd0ea03b2765699fa5f26c8bb6ef07db4a2ad18d7f18523886a2a0a174b
Jul 26 18:17:06.659042 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:06.659025    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 18:17:06.733097 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:06.733051    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:06.834506 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:06.834468    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:06.942367 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:06.942314    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:07.035612 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:07.035578    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-2l78q" event={"ID":"204bb694-4d92-4d85-b6d0-863fe7b98ca0","Type":"ContainerStarted","Data":"194dddd0ea03b2765699fa5f26c8bb6ef07db4a2ad18d7f18523886a2a0a174b"}
Jul 26 18:17:09.385444 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:09.385378    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:09.487692 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:09.487639    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:10.048700 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:10.048665    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-2l78q" event={"ID":"204bb694-4d92-4d85-b6d0-863fe7b98ca0","Type":"ContainerStarted","Data":"2cf43e06f6eb0228f8415456f3bc656db66bb62dd0825d604c5a49f0d2161155"}
Jul 26 18:17:11.054017 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:11.053919    2576 generic.go:356] "Generic (PLEG): container finished" podID="204bb694-4d92-4d85-b6d0-863fe7b98ca0" containerID="2cf43e06f6eb0228f8415456f3bc656db66bb62dd0825d604c5a49f0d2161155" exitCode=0
Jul 26 18:17:11.054017 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:11.054004    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-2l78q" event={"ID":"204bb694-4d92-4d85-b6d0-863fe7b98ca0","Type":"ContainerDied","Data":"2cf43e06f6eb0228f8415456f3bc656db66bb62dd0825d604c5a49f0d2161155"}
Jul 26 18:17:12.060578 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:12.060535    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-2l78q" event={"ID":"204bb694-4d92-4d85-b6d0-863fe7b98ca0","Type":"ContainerStarted","Data":"e60b63e2981ea60721faa9d76740b20be8ce5e128631f40df7956a4b51caf5b3"}
Jul 26 18:17:12.060990 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:12.060697    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-2l78q"
Jul 26 18:17:12.088307 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:12.088253    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-2l78q" podStartSLOduration=1.7186942200000002 podStartE2EDuration="6.088235627s" totalImagesPullingTime="3.348733487s" totalInitContainerRuntime="1.02080792s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:17:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:17:12.086639853 +0000 UTC m=+317.794046911" watchObservedRunningTime="2026-07-26 18:17:12.088235627 +0000 UTC m=+317.795642683"
Jul 26 18:17:19.433400 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:19.433365    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:19.499541 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:19.499503    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:23.066579 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:23.066549    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-2l78q"
Jul 26 18:17:25.239130 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:25.239091    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:25.345233 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:25.345186    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:26.088437 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:26.088401    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-ssbqj"]
Jul 26 18:17:26.096821 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:26.096796    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-ssbqj"
Jul 26 18:17:26.099540 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:26.099521    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 26 18:17:26.099664 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:26.099524    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-qqpxx\""
Jul 26 18:17:26.107493 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:26.107468    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-ssbqj"]
Jul 26 18:17:26.126411 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:26.126383    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4q577\" (UniqueName: \"kubernetes.io/projected/591441e9-ec06-416d-b803-d6688584f504-kube-api-access-4q577\") pod \"odh-model-controller-7998c67dc4-ssbqj\" (UID: \"591441e9-ec06-416d-b803-d6688584f504\") " pod="opendatahub/odh-model-controller-7998c67dc4-ssbqj"
Jul 26 18:17:26.126537 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:26.126425    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/591441e9-ec06-416d-b803-d6688584f504-cert\") pod \"odh-model-controller-7998c67dc4-ssbqj\" (UID: \"591441e9-ec06-416d-b803-d6688584f504\") " pod="opendatahub/odh-model-controller-7998c67dc4-ssbqj"
Jul 26 18:17:26.138228 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:26.138193    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:26.227366 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:26.227329    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q577\" (UniqueName: \"kubernetes.io/projected/591441e9-ec06-416d-b803-d6688584f504-kube-api-access-4q577\") pod \"odh-model-controller-7998c67dc4-ssbqj\" (UID: \"591441e9-ec06-416d-b803-d6688584f504\") " pod="opendatahub/odh-model-controller-7998c67dc4-ssbqj"
Jul 26 18:17:26.227528 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:26.227375    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/591441e9-ec06-416d-b803-d6688584f504-cert\") pod \"odh-model-controller-7998c67dc4-ssbqj\" (UID: \"591441e9-ec06-416d-b803-d6688584f504\") " pod="opendatahub/odh-model-controller-7998c67dc4-ssbqj"
Jul 26 18:17:26.227528 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:17:26.227485    2576 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 26 18:17:26.227595 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:17:26.227537    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/591441e9-ec06-416d-b803-d6688584f504-cert podName:591441e9-ec06-416d-b803-d6688584f504 nodeName:}" failed. No retries permitted until 2026-07-26 18:17:26.727519942 +0000 UTC m=+332.434926978 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/591441e9-ec06-416d-b803-d6688584f504-cert") pod "odh-model-controller-7998c67dc4-ssbqj" (UID: "591441e9-ec06-416d-b803-d6688584f504") : secret "odh-model-controller-webhook-cert" not found
Jul 26 18:17:26.237055 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:26.237029    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4q577\" (UniqueName: \"kubernetes.io/projected/591441e9-ec06-416d-b803-d6688584f504-kube-api-access-4q577\") pod \"odh-model-controller-7998c67dc4-ssbqj\" (UID: \"591441e9-ec06-416d-b803-d6688584f504\") " pod="opendatahub/odh-model-controller-7998c67dc4-ssbqj"
Jul 26 18:17:26.237774 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:26.237728    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:26.345994 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:26.345884    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:26.450594 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:26.450548    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:26.731909 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:26.731870    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/591441e9-ec06-416d-b803-d6688584f504-cert\") pod \"odh-model-controller-7998c67dc4-ssbqj\" (UID: \"591441e9-ec06-416d-b803-d6688584f504\") " pod="opendatahub/odh-model-controller-7998c67dc4-ssbqj"
Jul 26 18:17:26.734200 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:26.734166    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/591441e9-ec06-416d-b803-d6688584f504-cert\") pod \"odh-model-controller-7998c67dc4-ssbqj\" (UID: \"591441e9-ec06-416d-b803-d6688584f504\") " pod="opendatahub/odh-model-controller-7998c67dc4-ssbqj"
Jul 26 18:17:26.737264 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:26.737240    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-hcvtf"]
Jul 26 18:17:26.740769 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:26.740735    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-hcvtf"
Jul 26 18:17:26.743461 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:26.743437    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 26 18:17:26.745601 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:26.745577    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:26.751588 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:26.751505    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-hcvtf"]
Jul 26 18:17:26.832586 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:26.832539    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h87pk\" (UniqueName: \"kubernetes.io/projected/4e6dd317-bb81-452c-a779-25aec4db179a-kube-api-access-h87pk\") pod \"model-serving-api-54d89bd5fc-hcvtf\" (UID: \"4e6dd317-bb81-452c-a779-25aec4db179a\") " pod="opendatahub/model-serving-api-54d89bd5fc-hcvtf"
Jul 26 18:17:26.832824 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:26.832595    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e6dd317-bb81-452c-a779-25aec4db179a-tls-certs\") pod \"model-serving-api-54d89bd5fc-hcvtf\" (UID: \"4e6dd317-bb81-452c-a779-25aec4db179a\") " pod="opendatahub/model-serving-api-54d89bd5fc-hcvtf"
Jul 26 18:17:26.933540 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:26.933509    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h87pk\" (UniqueName: \"kubernetes.io/projected/4e6dd317-bb81-452c-a779-25aec4db179a-kube-api-access-h87pk\") pod \"model-serving-api-54d89bd5fc-hcvtf\" (UID: \"4e6dd317-bb81-452c-a779-25aec4db179a\") " pod="opendatahub/model-serving-api-54d89bd5fc-hcvtf"
Jul 26 18:17:26.933540 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:26.933549    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e6dd317-bb81-452c-a779-25aec4db179a-tls-certs\") pod \"model-serving-api-54d89bd5fc-hcvtf\" (UID: \"4e6dd317-bb81-452c-a779-25aec4db179a\") " pod="opendatahub/model-serving-api-54d89bd5fc-hcvtf"
Jul 26 18:17:26.935826 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:26.935794    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e6dd317-bb81-452c-a779-25aec4db179a-tls-certs\") pod \"model-serving-api-54d89bd5fc-hcvtf\" (UID: \"4e6dd317-bb81-452c-a779-25aec4db179a\") " pod="opendatahub/model-serving-api-54d89bd5fc-hcvtf"
Jul 26 18:17:26.941853 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:26.941831    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h87pk\" (UniqueName: \"kubernetes.io/projected/4e6dd317-bb81-452c-a779-25aec4db179a-kube-api-access-h87pk\") pod \"model-serving-api-54d89bd5fc-hcvtf\" (UID: \"4e6dd317-bb81-452c-a779-25aec4db179a\") " pod="opendatahub/model-serving-api-54d89bd5fc-hcvtf"
Jul 26 18:17:26.946978 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:26.946951    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:27.007251 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:27.007170    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-ssbqj"
Jul 26 18:17:27.053311 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:27.053274    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-hcvtf"
Jul 26 18:17:27.147229 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:27.147132    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-ssbqj"]
Jul 26 18:17:27.152897 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:17:27.152866    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod591441e9_ec06_416d_b803_d6688584f504.slice/crio-b68aa69f5fcf037b040ecb98404e6b6894f65156a29f534af7c2bd46f13ad62e WatchSource:0}: Error finding container b68aa69f5fcf037b040ecb98404e6b6894f65156a29f534af7c2bd46f13ad62e: Status 404 returned error can't find the container with id b68aa69f5fcf037b040ecb98404e6b6894f65156a29f534af7c2bd46f13ad62e
Jul 26 18:17:27.199456 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:27.199424    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-hcvtf"]
Jul 26 18:17:27.203702 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:17:27.203679    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e6dd317_bb81_452c_a779_25aec4db179a.slice/crio-e68508665d908bab324c6d026dfd222f7160de5f5031ea74cce7d3dab81c738e WatchSource:0}: Error finding container e68508665d908bab324c6d026dfd222f7160de5f5031ea74cce7d3dab81c738e: Status 404 returned error can't find the container with id e68508665d908bab324c6d026dfd222f7160de5f5031ea74cce7d3dab81c738e
Jul 26 18:17:28.116544 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:28.116504    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-hcvtf" event={"ID":"4e6dd317-bb81-452c-a779-25aec4db179a","Type":"ContainerStarted","Data":"e68508665d908bab324c6d026dfd222f7160de5f5031ea74cce7d3dab81c738e"}
Jul 26 18:17:28.118089 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:28.118064    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-ssbqj" event={"ID":"591441e9-ec06-416d-b803-d6688584f504","Type":"ContainerStarted","Data":"b68aa69f5fcf037b040ecb98404e6b6894f65156a29f534af7c2bd46f13ad62e"}
Jul 26 18:17:31.131335 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:31.131296    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-hcvtf" event={"ID":"4e6dd317-bb81-452c-a779-25aec4db179a","Type":"ContainerStarted","Data":"e46cca69310d71af660adbd215f4f12ead2a16c549e62928dd7e3dcad9d44226"}
Jul 26 18:17:31.131794 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:31.131364    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-hcvtf"
Jul 26 18:17:31.132655 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:31.132633    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-ssbqj" event={"ID":"591441e9-ec06-416d-b803-d6688584f504","Type":"ContainerStarted","Data":"d0b49b526e8b2663fc5aa7aa0db9486657851175b018276afe6225c433daec62"}
Jul 26 18:17:31.132781 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:31.132763    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-ssbqj"
Jul 26 18:17:31.156817 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:31.156741    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-hcvtf" podStartSLOduration=2.05857842 podStartE2EDuration="5.156720903s" totalImagesPullingTime="3.098142483s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:17:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:17:31.156418123 +0000 UTC m=+336.863825184" watchObservedRunningTime="2026-07-26 18:17:31.156720903 +0000 UTC m=+336.864127963"
Jul 26 18:17:31.186545 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:31.186496    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-ssbqj" podStartSLOduration=1.990352428 podStartE2EDuration="5.186477183s" totalImagesPullingTime="3.196124755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:17:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:17:31.185428538 +0000 UTC m=+336.892835609" watchObservedRunningTime="2026-07-26 18:17:31.186477183 +0000 UTC m=+336.893884242"
Jul 26 18:17:36.210170 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:36.210126    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:36.241850 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:36.241812    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5c7c67d4f6-wg28w"]
Jul 26 18:17:36.244868 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:36.244847    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5c7c67d4f6-wg28w"
Jul 26 18:17:36.249126 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:36.249099    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 26 18:17:36.252397 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:36.252351    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 26 18:17:36.252397 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:36.252360    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-659n7\""
Jul 26 18:17:36.252592 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:36.252365    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 26 18:17:36.252592 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:36.252359    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 26 18:17:36.282563 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:36.282535    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5c7c67d4f6-wg28w"]
Jul 26 18:17:36.307612 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:36.307584    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwjzb\" (UniqueName: \"kubernetes.io/projected/b2611181-ba1a-488b-be78-c6f4459823e5-kube-api-access-hwjzb\") pod \"kube-auth-proxy-5c7c67d4f6-wg28w\" (UID: \"b2611181-ba1a-488b-be78-c6f4459823e5\") " pod="openshift-ingress/kube-auth-proxy-5c7c67d4f6-wg28w"
Jul 26 18:17:36.307779 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:36.307643    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2611181-ba1a-488b-be78-c6f4459823e5-tls-certs\") pod \"kube-auth-proxy-5c7c67d4f6-wg28w\" (UID: \"b2611181-ba1a-488b-be78-c6f4459823e5\") " pod="openshift-ingress/kube-auth-proxy-5c7c67d4f6-wg28w"
Jul 26 18:17:36.307779 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:36.307720    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b2611181-ba1a-488b-be78-c6f4459823e5-tmp\") pod \"kube-auth-proxy-5c7c67d4f6-wg28w\" (UID: \"b2611181-ba1a-488b-be78-c6f4459823e5\") " pod="openshift-ingress/kube-auth-proxy-5c7c67d4f6-wg28w"
Jul 26 18:17:36.348699 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:36.348666    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:36.408562 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:36.408522    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b2611181-ba1a-488b-be78-c6f4459823e5-tmp\") pod \"kube-auth-proxy-5c7c67d4f6-wg28w\" (UID: \"b2611181-ba1a-488b-be78-c6f4459823e5\") " pod="openshift-ingress/kube-auth-proxy-5c7c67d4f6-wg28w"
Jul 26 18:17:36.408698 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:36.408571    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwjzb\" (UniqueName: \"kubernetes.io/projected/b2611181-ba1a-488b-be78-c6f4459823e5-kube-api-access-hwjzb\") pod \"kube-auth-proxy-5c7c67d4f6-wg28w\" (UID: \"b2611181-ba1a-488b-be78-c6f4459823e5\") " pod="openshift-ingress/kube-auth-proxy-5c7c67d4f6-wg28w"
Jul 26 18:17:36.408698 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:36.408604    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2611181-ba1a-488b-be78-c6f4459823e5-tls-certs\") pod \"kube-auth-proxy-5c7c67d4f6-wg28w\" (UID: \"b2611181-ba1a-488b-be78-c6f4459823e5\") " pod="openshift-ingress/kube-auth-proxy-5c7c67d4f6-wg28w"
Jul 26 18:17:36.410804 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:36.410779    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b2611181-ba1a-488b-be78-c6f4459823e5-tmp\") pod \"kube-auth-proxy-5c7c67d4f6-wg28w\" (UID: \"b2611181-ba1a-488b-be78-c6f4459823e5\") " pod="openshift-ingress/kube-auth-proxy-5c7c67d4f6-wg28w"
Jul 26 18:17:36.411080 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:36.411050    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2611181-ba1a-488b-be78-c6f4459823e5-tls-certs\") pod \"kube-auth-proxy-5c7c67d4f6-wg28w\" (UID: \"b2611181-ba1a-488b-be78-c6f4459823e5\") " pod="openshift-ingress/kube-auth-proxy-5c7c67d4f6-wg28w"
Jul 26 18:17:36.418734 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:36.418713    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwjzb\" (UniqueName: \"kubernetes.io/projected/b2611181-ba1a-488b-be78-c6f4459823e5-kube-api-access-hwjzb\") pod \"kube-auth-proxy-5c7c67d4f6-wg28w\" (UID: \"b2611181-ba1a-488b-be78-c6f4459823e5\") " pod="openshift-ingress/kube-auth-proxy-5c7c67d4f6-wg28w"
Jul 26 18:17:36.555245 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:36.555207    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5c7c67d4f6-wg28w"
Jul 26 18:17:36.705272 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:36.705239    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5c7c67d4f6-wg28w"]
Jul 26 18:17:36.709776 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:17:36.709729    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2611181_ba1a_488b_be78_c6f4459823e5.slice/crio-281d870b1c2a5b9c96d870725a87dabbf79d2a80bf89582258540356cdafe081 WatchSource:0}: Error finding container 281d870b1c2a5b9c96d870725a87dabbf79d2a80bf89582258540356cdafe081: Status 404 returned error can't find the container with id 281d870b1c2a5b9c96d870725a87dabbf79d2a80bf89582258540356cdafe081
Jul 26 18:17:37.155806 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:37.155762    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5c7c67d4f6-wg28w" event={"ID":"b2611181-ba1a-488b-be78-c6f4459823e5","Type":"ContainerStarted","Data":"281d870b1c2a5b9c96d870725a87dabbf79d2a80bf89582258540356cdafe081"}
Jul 26 18:17:37.283135 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:37.283102    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:37.395525 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:37.395490    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-hwn8m"]
Jul 26 18:17:37.398825 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:37.398798    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hwn8m"
Jul 26 18:17:37.403290 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:37.403226    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 26 18:17:37.404079 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:37.404053    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-mr882\""
Jul 26 18:17:37.406000 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:37.405890    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xrbql"]
Jul 26 18:17:37.409060 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:37.409041    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xrbql"
Jul 26 18:17:37.416367 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:37.416190    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 26 18:17:37.416367 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:37.416191    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-tzkbh\""
Jul 26 18:17:37.423343 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:37.423312    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-hwn8m"]
Jul 26 18:17:37.432874 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:37.432532    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xrbql"]
Jul 26 18:17:37.476269 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:37.476232    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:37.516322 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:37.516288    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3128f681-a36c-4e7e-9c9b-a594b32e1567-cert\") pod \"kserve-controller-manager-d6f4b65b8-hwn8m\" (UID: \"3128f681-a36c-4e7e-9c9b-a594b32e1567\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hwn8m"
Jul 26 18:17:37.516322 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:37.516324    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2473a76b-1a39-4732-b37c-f92386120068-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xrbql\" (UID: \"2473a76b-1a39-4732-b37c-f92386120068\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xrbql"
Jul 26 18:17:37.516566 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:37.516349    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79jr8\" (UniqueName: \"kubernetes.io/projected/2473a76b-1a39-4732-b37c-f92386120068-kube-api-access-79jr8\") pod \"llmisvc-controller-manager-777c6bd7d5-xrbql\" (UID: \"2473a76b-1a39-4732-b37c-f92386120068\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xrbql"
Jul 26 18:17:37.516566 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:37.516469    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqlvh\" (UniqueName: \"kubernetes.io/projected/3128f681-a36c-4e7e-9c9b-a594b32e1567-kube-api-access-hqlvh\") pod \"kserve-controller-manager-d6f4b65b8-hwn8m\" (UID: \"3128f681-a36c-4e7e-9c9b-a594b32e1567\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hwn8m"
Jul 26 18:17:37.617446 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:37.617402    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3128f681-a36c-4e7e-9c9b-a594b32e1567-cert\") pod \"kserve-controller-manager-d6f4b65b8-hwn8m\" (UID: \"3128f681-a36c-4e7e-9c9b-a594b32e1567\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hwn8m"
Jul 26 18:17:37.617636 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:37.617457    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2473a76b-1a39-4732-b37c-f92386120068-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xrbql\" (UID: \"2473a76b-1a39-4732-b37c-f92386120068\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xrbql"
Jul 26 18:17:37.617636 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:37.617485    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79jr8\" (UniqueName: \"kubernetes.io/projected/2473a76b-1a39-4732-b37c-f92386120068-kube-api-access-79jr8\") pod \"llmisvc-controller-manager-777c6bd7d5-xrbql\" (UID: \"2473a76b-1a39-4732-b37c-f92386120068\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xrbql"
Jul 26 18:17:37.617636 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:37.617537    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqlvh\" (UniqueName: \"kubernetes.io/projected/3128f681-a36c-4e7e-9c9b-a594b32e1567-kube-api-access-hqlvh\") pod \"kserve-controller-manager-d6f4b65b8-hwn8m\" (UID: \"3128f681-a36c-4e7e-9c9b-a594b32e1567\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hwn8m"
Jul 26 18:17:37.617636 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:17:37.617566    2576 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 26 18:17:37.617879 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:17:37.617648    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3128f681-a36c-4e7e-9c9b-a594b32e1567-cert podName:3128f681-a36c-4e7e-9c9b-a594b32e1567 nodeName:}" failed. No retries permitted until 2026-07-26 18:17:38.117624103 +0000 UTC m=+343.825031153 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/3128f681-a36c-4e7e-9c9b-a594b32e1567-cert") pod "kserve-controller-manager-d6f4b65b8-hwn8m" (UID: "3128f681-a36c-4e7e-9c9b-a594b32e1567") : secret "kserve-webhook-server-cert" not found
Jul 26 18:17:37.620657 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:37.620625    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2473a76b-1a39-4732-b37c-f92386120068-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xrbql\" (UID: \"2473a76b-1a39-4732-b37c-f92386120068\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xrbql"
Jul 26 18:17:37.630179 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:37.630143    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqlvh\" (UniqueName: \"kubernetes.io/projected/3128f681-a36c-4e7e-9c9b-a594b32e1567-kube-api-access-hqlvh\") pod \"kserve-controller-manager-d6f4b65b8-hwn8m\" (UID: \"3128f681-a36c-4e7e-9c9b-a594b32e1567\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hwn8m"
Jul 26 18:17:37.635183 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:37.635134    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79jr8\" (UniqueName: \"kubernetes.io/projected/2473a76b-1a39-4732-b37c-f92386120068-kube-api-access-79jr8\") pod \"llmisvc-controller-manager-777c6bd7d5-xrbql\" (UID: \"2473a76b-1a39-4732-b37c-f92386120068\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xrbql"
Jul 26 18:17:37.730436 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:37.730397    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xrbql"
Jul 26 18:17:37.767133 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:37.767095    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:37.876211 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:37.876082    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xrbql"]
Jul 26 18:17:37.879287 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:17:37.878838    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2473a76b_1a39_4732_b37c_f92386120068.slice/crio-cddcb36d22732150d74ad4aaa6418369d7cbc304b04b5f6f82c9990085fa0f69 WatchSource:0}: Error finding container cddcb36d22732150d74ad4aaa6418369d7cbc304b04b5f6f82c9990085fa0f69: Status 404 returned error can't find the container with id cddcb36d22732150d74ad4aaa6418369d7cbc304b04b5f6f82c9990085fa0f69
Jul 26 18:17:38.122378 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:38.122277    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3128f681-a36c-4e7e-9c9b-a594b32e1567-cert\") pod \"kserve-controller-manager-d6f4b65b8-hwn8m\" (UID: \"3128f681-a36c-4e7e-9c9b-a594b32e1567\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hwn8m"
Jul 26 18:17:38.125229 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:38.125173    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3128f681-a36c-4e7e-9c9b-a594b32e1567-cert\") pod \"kserve-controller-manager-d6f4b65b8-hwn8m\" (UID: \"3128f681-a36c-4e7e-9c9b-a594b32e1567\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hwn8m"
Jul 26 18:17:38.161041 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:38.161006    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xrbql" event={"ID":"2473a76b-1a39-4732-b37c-f92386120068","Type":"ContainerStarted","Data":"cddcb36d22732150d74ad4aaa6418369d7cbc304b04b5f6f82c9990085fa0f69"}
Jul 26 18:17:38.312251 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:38.312215    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hwn8m"
Jul 26 18:17:38.478707 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:38.478671    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-hwn8m"]
Jul 26 18:17:38.482071 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:17:38.482028    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3128f681_a36c_4e7e_9c9b_a594b32e1567.slice/crio-ea06db055d627efd2da54d19b525f0faca681af24669690d2b0480d0d4e24454 WatchSource:0}: Error finding container ea06db055d627efd2da54d19b525f0faca681af24669690d2b0480d0d4e24454: Status 404 returned error can't find the container with id ea06db055d627efd2da54d19b525f0faca681af24669690d2b0480d0d4e24454
Jul 26 18:17:38.636107 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:38.635973    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:38.755569 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:38.755529    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:38.889966 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:38.889882    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-d92ps"]
Jul 26 18:17:38.893101 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:38.893079    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d92ps"
Jul 26 18:17:38.897565 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:38.897541    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 26 18:17:38.897716 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:38.897541    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 26 18:17:38.898992 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:38.898969    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 26 18:17:38.898992 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:38.898987    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-hqlsd\""
Jul 26 18:17:38.910149 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:38.910124    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 26 18:17:38.928611 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:38.928576    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4aab415-9a5c-457c-b5af-18a0ef16ab42-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-d92ps\" (UID: \"a4aab415-9a5c-457c-b5af-18a0ef16ab42\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d92ps"
Jul 26 18:17:38.928799 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:38.928619    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4aab415-9a5c-457c-b5af-18a0ef16ab42-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-d92ps\" (UID: \"a4aab415-9a5c-457c-b5af-18a0ef16ab42\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d92ps"
Jul 26 18:17:38.928799 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:38.928647    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a4aab415-9a5c-457c-b5af-18a0ef16ab42-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-d92ps\" (UID: \"a4aab415-9a5c-457c-b5af-18a0ef16ab42\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d92ps"
Jul 26 18:17:38.928924 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:38.928807    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a4aab415-9a5c-457c-b5af-18a0ef16ab42-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-d92ps\" (UID: \"a4aab415-9a5c-457c-b5af-18a0ef16ab42\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d92ps"
Jul 26 18:17:38.928924 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:38.928847    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a4aab415-9a5c-457c-b5af-18a0ef16ab42-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-d92ps\" (UID: \"a4aab415-9a5c-457c-b5af-18a0ef16ab42\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d92ps"
Jul 26 18:17:38.928924 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:38.928899    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a4aab415-9a5c-457c-b5af-18a0ef16ab42-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-d92ps\" (UID: \"a4aab415-9a5c-457c-b5af-18a0ef16ab42\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d92ps"
Jul 26 18:17:38.929067 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:38.928948    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snpcd\" (UniqueName: \"kubernetes.io/projected/a4aab415-9a5c-457c-b5af-18a0ef16ab42-kube-api-access-snpcd\") pod \"istiod-openshift-gateway-5677ccb49b-d92ps\" (UID: \"a4aab415-9a5c-457c-b5af-18a0ef16ab42\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d92ps"
Jul 26 18:17:38.929817 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:38.929789    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:38.961877 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:38.961838    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-d92ps"]
Jul 26 18:17:39.033164 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:39.033129    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4aab415-9a5c-457c-b5af-18a0ef16ab42-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-d92ps\" (UID: \"a4aab415-9a5c-457c-b5af-18a0ef16ab42\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d92ps"
Jul 26 18:17:39.033164 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:39.033164    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a4aab415-9a5c-457c-b5af-18a0ef16ab42-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-d92ps\" (UID: \"a4aab415-9a5c-457c-b5af-18a0ef16ab42\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d92ps"
Jul 26 18:17:39.033409 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:39.033228    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a4aab415-9a5c-457c-b5af-18a0ef16ab42-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-d92ps\" (UID: \"a4aab415-9a5c-457c-b5af-18a0ef16ab42\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d92ps"
Jul 26 18:17:39.033409 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:39.033244    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a4aab415-9a5c-457c-b5af-18a0ef16ab42-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-d92ps\" (UID: \"a4aab415-9a5c-457c-b5af-18a0ef16ab42\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d92ps"
Jul 26 18:17:39.033409 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:39.033285    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a4aab415-9a5c-457c-b5af-18a0ef16ab42-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-d92ps\" (UID: \"a4aab415-9a5c-457c-b5af-18a0ef16ab42\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d92ps"
Jul 26 18:17:39.033409 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:39.033339    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snpcd\" (UniqueName: \"kubernetes.io/projected/a4aab415-9a5c-457c-b5af-18a0ef16ab42-kube-api-access-snpcd\") pod \"istiod-openshift-gateway-5677ccb49b-d92ps\" (UID: \"a4aab415-9a5c-457c-b5af-18a0ef16ab42\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d92ps"
Jul 26 18:17:39.033409 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:39.033391    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4aab415-9a5c-457c-b5af-18a0ef16ab42-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-d92ps\" (UID: \"a4aab415-9a5c-457c-b5af-18a0ef16ab42\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d92ps"
Jul 26 18:17:39.034261 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:39.034197    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a4aab415-9a5c-457c-b5af-18a0ef16ab42-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-d92ps\" (UID: \"a4aab415-9a5c-457c-b5af-18a0ef16ab42\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d92ps"
Jul 26 18:17:39.036430 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:39.036387    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a4aab415-9a5c-457c-b5af-18a0ef16ab42-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-d92ps\" (UID: \"a4aab415-9a5c-457c-b5af-18a0ef16ab42\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d92ps"
Jul 26 18:17:39.036527 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:39.036451    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4aab415-9a5c-457c-b5af-18a0ef16ab42-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-d92ps\" (UID: \"a4aab415-9a5c-457c-b5af-18a0ef16ab42\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d92ps"
Jul 26 18:17:39.037107 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:39.037080    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a4aab415-9a5c-457c-b5af-18a0ef16ab42-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-d92ps\" (UID: \"a4aab415-9a5c-457c-b5af-18a0ef16ab42\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d92ps"
Jul 26 18:17:39.037316 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:39.037299    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a4aab415-9a5c-457c-b5af-18a0ef16ab42-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-d92ps\" (UID: \"a4aab415-9a5c-457c-b5af-18a0ef16ab42\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d92ps"
Jul 26 18:17:39.050979 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:39.050955    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4aab415-9a5c-457c-b5af-18a0ef16ab42-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-d92ps\" (UID: \"a4aab415-9a5c-457c-b5af-18a0ef16ab42\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d92ps"
Jul 26 18:17:39.068028 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:39.067991    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snpcd\" (UniqueName: \"kubernetes.io/projected/a4aab415-9a5c-457c-b5af-18a0ef16ab42-kube-api-access-snpcd\") pod \"istiod-openshift-gateway-5677ccb49b-d92ps\" (UID: \"a4aab415-9a5c-457c-b5af-18a0ef16ab42\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d92ps"
Jul 26 18:17:39.155780 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:39.155670    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:39.166031 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:39.165986    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hwn8m" event={"ID":"3128f681-a36c-4e7e-9c9b-a594b32e1567","Type":"ContainerStarted","Data":"ea06db055d627efd2da54d19b525f0faca681af24669690d2b0480d0d4e24454"}
Jul 26 18:17:39.204569 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:39.204533    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d92ps"
Jul 26 18:17:39.481004 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:39.480979    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-d92ps"]
Jul 26 18:17:39.579525 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:17:39.579486    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4aab415_9a5c_457c_b5af_18a0ef16ab42.slice/crio-bd97a54f7b4135223f475b8a10d83d161a2feb3811a0fa25a0cc474c5ed529b4 WatchSource:0}: Error finding container bd97a54f7b4135223f475b8a10d83d161a2feb3811a0fa25a0cc474c5ed529b4: Status 404 returned error can't find the container with id bd97a54f7b4135223f475b8a10d83d161a2feb3811a0fa25a0cc474c5ed529b4
Jul 26 18:17:40.174076 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:40.174039    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d92ps" event={"ID":"a4aab415-9a5c-457c-b5af-18a0ef16ab42","Type":"ContainerStarted","Data":"bd97a54f7b4135223f475b8a10d83d161a2feb3811a0fa25a0cc474c5ed529b4"}
Jul 26 18:17:40.175653 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:40.175629    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5c7c67d4f6-wg28w" event={"ID":"b2611181-ba1a-488b-be78-c6f4459823e5","Type":"ContainerStarted","Data":"b8bc322bf70270d56d830f8521c7be6794931ee7ee9510b843ca49960c11d5dd"}
Jul 26 18:17:40.245944 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:40.245882    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5c7c67d4f6-wg28w" podStartSLOduration=1.325133206 podStartE2EDuration="4.245857559s" totalImagesPullingTime="2.920724353s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:17:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:17:40.243563235 +0000 UTC m=+345.950970295" watchObservedRunningTime="2026-07-26 18:17:40.245857559 +0000 UTC m=+345.953264619"
Jul 26 18:17:40.353669 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:40.353610    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:42.139040 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:42.139003    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-ssbqj"
Jul 26 18:17:42.141419 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:42.141399    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-hcvtf"
Jul 26 18:17:42.888927 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:42.888894    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:17:42.888991 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:42.888962    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:17:43.190261 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:43.190225    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xrbql" event={"ID":"2473a76b-1a39-4732-b37c-f92386120068","Type":"ContainerStarted","Data":"39a1bdc18b2d818b5dc91fe1ce7c38c26c1693c42410f73751f4043824d2df7a"}
Jul 26 18:17:43.190722 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:43.190359    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xrbql"
Jul 26 18:17:43.191989 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:43.191963    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hwn8m" event={"ID":"3128f681-a36c-4e7e-9c9b-a594b32e1567","Type":"ContainerStarted","Data":"4a0d6036b7d9ffe0575e868c8067eb3a833ac0c95354cf93910bc7e4a9ba7203"}
Jul 26 18:17:43.192100 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:43.192059    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hwn8m"
Jul 26 18:17:43.193541 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:43.193520    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d92ps" event={"ID":"a4aab415-9a5c-457c-b5af-18a0ef16ab42","Type":"ContainerStarted","Data":"17d681bd92edf662eeeacfba0f34feb06d5689009c1440f2d5e3242a5fa9480c"}
Jul 26 18:17:43.193724 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:43.193708    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d92ps"
Jul 26 18:17:43.195378 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:43.195355    2576 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-d92ps container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 26 18:17:43.195479 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:43.195404    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d92ps" podUID="a4aab415-9a5c-457c-b5af-18a0ef16ab42" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 26 18:17:43.212158 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:43.212110    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xrbql" podStartSLOduration=1.205272105 podStartE2EDuration="6.212095904s" totalImagesPullingTime="5.006823799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:17:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:17:43.210677263 +0000 UTC m=+348.918084322" watchObservedRunningTime="2026-07-26 18:17:43.212095904 +0000 UTC m=+348.919502963"
Jul 26 18:17:43.238410 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:43.238344    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hwn8m" podStartSLOduration=1.8398908889999999 podStartE2EDuration="6.238325436s" totalImagesPullingTime="4.398434547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:17:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:17:43.237863298 +0000 UTC m=+348.945270369" watchObservedRunningTime="2026-07-26 18:17:43.238325436 +0000 UTC m=+348.945732494"
Jul 26 18:17:43.264994 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:43.264933    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d92ps" podStartSLOduration=1.9591036640000001 podStartE2EDuration="5.26491313s" totalImagesPullingTime="3.305809466s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:17:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:17:43.263068638 +0000 UTC m=+348.970475699" watchObservedRunningTime="2026-07-26 18:17:43.26491313 +0000 UTC m=+348.972320187"
Jul 26 18:17:43.469913 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:43.469806    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:43.619039 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:43.618947    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:44.197988 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:44.197959    2576 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-d92ps container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 26 18:17:44.198351 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:44.198006    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d92ps" podUID="a4aab415-9a5c-457c-b5af-18a0ef16ab42" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 26 18:17:44.440119 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:44.440086    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:44.566876 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:44.566783    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:44.772044 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:44.771998    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:45.053961 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:45.053920    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:45.140559 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:45.140519    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:47.198175 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:47.198098    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-d92ps"
Jul 26 18:17:53.912631 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:53.912598    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:17:54.010894 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:17:54.010859    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:18:12.353433 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:18:12.353391    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:18:12.433910 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:18:12.433874    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:18:14.199104 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:18:14.199076    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xrbql"
Jul 26 18:18:14.202336 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:18:14.202314    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hwn8m"
Jul 26 18:18:26.702905 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:18:26.702853    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:18:26.814636 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:18:26.814592    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:18:27.045555 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:18:27.045462    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:18:54.563528 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:18:54.563494    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:18:55.194012 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:18:55.193966    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:18:59.857163 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:18:59.857130    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:19:01.865717 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:01.865683    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:19:02.013340 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:02.013309    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:19:02.835795 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:02.835764    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:19:04.350955 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:04.350915    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:19:05.607582 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:05.607548    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hfvpm"]
Jul 26 18:19:05.609675 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:05.609655    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hfvpm"
Jul 26 18:19:05.616961 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:05.616597    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 26 18:19:05.616961 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:05.616619    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-jx45p\""
Jul 26 18:19:05.616961 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:05.616861    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 26 18:19:05.634546 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:05.634517    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hfvpm"]
Jul 26 18:19:05.709847 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:05.709813    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trjx6\" (UniqueName: \"kubernetes.io/projected/e0f6a24a-e27f-4958-850e-ec282d487c01-kube-api-access-trjx6\") pod \"limitador-operator-controller-manager-859d7b78bd-hfvpm\" (UID: \"e0f6a24a-e27f-4958-850e-ec282d487c01\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hfvpm"
Jul 26 18:19:05.810466 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:05.810434    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trjx6\" (UniqueName: \"kubernetes.io/projected/e0f6a24a-e27f-4958-850e-ec282d487c01-kube-api-access-trjx6\") pod \"limitador-operator-controller-manager-859d7b78bd-hfvpm\" (UID: \"e0f6a24a-e27f-4958-850e-ec282d487c01\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hfvpm"
Jul 26 18:19:05.826018 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:05.825985    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trjx6\" (UniqueName: \"kubernetes.io/projected/e0f6a24a-e27f-4958-850e-ec282d487c01-kube-api-access-trjx6\") pod \"limitador-operator-controller-manager-859d7b78bd-hfvpm\" (UID: \"e0f6a24a-e27f-4958-850e-ec282d487c01\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hfvpm"
Jul 26 18:19:05.920774 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:05.920658    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hfvpm"
Jul 26 18:19:06.051982 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:06.051953    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hfvpm"]
Jul 26 18:19:06.055604 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:19:06.055578    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0f6a24a_e27f_4958_850e_ec282d487c01.slice/crio-983e4919a7c3ce60fc89fe8176c2fc84e7d1cd5dbc84fda0c5f7c163de606231 WatchSource:0}: Error finding container 983e4919a7c3ce60fc89fe8176c2fc84e7d1cd5dbc84fda0c5f7c163de606231: Status 404 returned error can't find the container with id 983e4919a7c3ce60fc89fe8176c2fc84e7d1cd5dbc84fda0c5f7c163de606231
Jul 26 18:19:06.183765 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:06.183718    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:19:06.522168 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:06.522093    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hfvpm" event={"ID":"e0f6a24a-e27f-4958-850e-ec282d487c01","Type":"ContainerStarted","Data":"983e4919a7c3ce60fc89fe8176c2fc84e7d1cd5dbc84fda0c5f7c163de606231"}
Jul 26 18:19:06.698617 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:06.698580    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:19:08.532027 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:08.531990    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hfvpm" event={"ID":"e0f6a24a-e27f-4958-850e-ec282d487c01","Type":"ContainerStarted","Data":"08576b134467a8f50112a4c850580943c077bf8e512af61caecfff5c63319797"}
Jul 26 18:19:08.532411 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:08.532133    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hfvpm"
Jul 26 18:19:08.576895 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:08.576842    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hfvpm" podStartSLOduration=2.02803405 podStartE2EDuration="3.576821357s" totalImagesPullingTime="1.548787307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:19:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:19:08.574969636 +0000 UTC m=+434.282376694" watchObservedRunningTime="2026-07-26 18:19:08.576821357 +0000 UTC m=+434.284228416"
Jul 26 18:19:09.909449 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:09.909415    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:19:11.782790 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:11.782718    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:19:11.883102 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:11.883067    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:19:12.183560 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:12.183528    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:19:12.312117 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:12.312078    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:19:15.775083 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:15.775043    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:19:15.876237 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:15.876197    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:19:18.715120 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:18.715031    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:19:19.177602 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:19.177569    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:19:19.538649 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:19.538574    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hfvpm"
Jul 26 18:19:19.820480 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:19.820391    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:19:19.920672 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:19.920637    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:19:23.388365 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:23.388326    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:19:23.595949 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:23.595914    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:19:29.972017 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:29.971983    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hfvpm"]
Jul 26 18:19:29.972420 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:29.972198    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hfvpm" podUID="e0f6a24a-e27f-4958-850e-ec282d487c01" containerName="manager" containerID="cri-o://08576b134467a8f50112a4c850580943c077bf8e512af61caecfff5c63319797" gracePeriod=2
Jul 26 18:19:29.996639 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:29.996610    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hfvpm"]
Jul 26 18:19:30.016928 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:19:30.016881    2576 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"e0f6a24a-e27f-4958-850e-ec282d487c01\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-hfvpm\": pods \"limitador-operator-controller-manager-859d7b78bd-hfvpm\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hfvpm"
Jul 26 18:19:30.019935 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:30.019911    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-867cs"]
Jul 26 18:19:30.020198 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:30.020187    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0f6a24a-e27f-4958-850e-ec282d487c01" containerName="manager"
Jul 26 18:19:30.020238 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:30.020200    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0f6a24a-e27f-4958-850e-ec282d487c01" containerName="manager"
Jul 26 18:19:30.020271 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:30.020247    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0f6a24a-e27f-4958-850e-ec282d487c01" containerName="manager"
Jul 26 18:19:30.022977 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:30.022958    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-867cs"
Jul 26 18:19:30.027523 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:19:30.027492    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hfvpm\" is forbidden: User \"system:node:ip-10-0-134-7.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-7.ec2.internal' and this object" podUID="e0f6a24a-e27f-4958-850e-ec282d487c01" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hfvpm"
Jul 26 18:19:30.053729 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:30.053692    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-867cs"]
Jul 26 18:19:30.122988 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:30.122959    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nd2g\" (UniqueName: \"kubernetes.io/projected/137b7c20-6067-4a91-a05d-50ee376714f5-kube-api-access-2nd2g\") pod \"limitador-operator-controller-manager-859d7b78bd-867cs\" (UID: \"137b7c20-6067-4a91-a05d-50ee376714f5\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-867cs"
Jul 26 18:19:30.204939 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:30.204913    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hfvpm"
Jul 26 18:19:30.207726 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:19:30.207699    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hfvpm\" is forbidden: User \"system:node:ip-10-0-134-7.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-7.ec2.internal' and this object" podUID="e0f6a24a-e27f-4958-850e-ec282d487c01" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hfvpm"
Jul 26 18:19:30.224018 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:30.223962    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-trjx6\" (UniqueName: \"kubernetes.io/projected/e0f6a24a-e27f-4958-850e-ec282d487c01-kube-api-access-trjx6\") pod \"e0f6a24a-e27f-4958-850e-ec282d487c01\" (UID: \"e0f6a24a-e27f-4958-850e-ec282d487c01\") "
Jul 26 18:19:30.224108 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:30.224094    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nd2g\" (UniqueName: \"kubernetes.io/projected/137b7c20-6067-4a91-a05d-50ee376714f5-kube-api-access-2nd2g\") pod \"limitador-operator-controller-manager-859d7b78bd-867cs\" (UID: \"137b7c20-6067-4a91-a05d-50ee376714f5\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-867cs"
Jul 26 18:19:30.226015 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:30.225991    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0f6a24a-e27f-4958-850e-ec282d487c01-kube-api-access-trjx6" (OuterVolumeSpecName: "kube-api-access-trjx6") pod "e0f6a24a-e27f-4958-850e-ec282d487c01" (UID: "e0f6a24a-e27f-4958-850e-ec282d487c01"). InnerVolumeSpecName "kube-api-access-trjx6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:19:30.239265 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:30.239240    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nd2g\" (UniqueName: \"kubernetes.io/projected/137b7c20-6067-4a91-a05d-50ee376714f5-kube-api-access-2nd2g\") pod \"limitador-operator-controller-manager-859d7b78bd-867cs\" (UID: \"137b7c20-6067-4a91-a05d-50ee376714f5\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-867cs"
Jul 26 18:19:30.324978 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:30.324951    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trjx6\" (UniqueName: \"kubernetes.io/projected/e0f6a24a-e27f-4958-850e-ec282d487c01-kube-api-access-trjx6\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:19:30.361012 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:30.360990    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-867cs"
Jul 26 18:19:30.504504 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:30.504421    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-867cs"]
Jul 26 18:19:30.509123 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:19:30.509096    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod137b7c20_6067_4a91_a05d_50ee376714f5.slice/crio-bb8791d7db581c9fdaac81423f1a51c51e817bdc2121442860e29831ffc44624 WatchSource:0}: Error finding container bb8791d7db581c9fdaac81423f1a51c51e817bdc2121442860e29831ffc44624: Status 404 returned error can't find the container with id bb8791d7db581c9fdaac81423f1a51c51e817bdc2121442860e29831ffc44624
Jul 26 18:19:30.621371 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:30.621337    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-867cs" event={"ID":"137b7c20-6067-4a91-a05d-50ee376714f5","Type":"ContainerStarted","Data":"3971fcd14848fa2d126f030f9c6a80efe9cc5769d62e3a6f55dd561d3d9e3bb0"}
Jul 26 18:19:30.621587 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:30.621395    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-867cs" event={"ID":"137b7c20-6067-4a91-a05d-50ee376714f5","Type":"ContainerStarted","Data":"bb8791d7db581c9fdaac81423f1a51c51e817bdc2121442860e29831ffc44624"}
Jul 26 18:19:30.621587 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:30.621425    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-867cs"
Jul 26 18:19:30.622574 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:30.622548    2576 generic.go:356] "Generic (PLEG): container finished" podID="e0f6a24a-e27f-4958-850e-ec282d487c01" containerID="08576b134467a8f50112a4c850580943c077bf8e512af61caecfff5c63319797" exitCode=0
Jul 26 18:19:30.622678 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:30.622585    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hfvpm"
Jul 26 18:19:30.622678 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:30.622623    2576 scope.go:117] "RemoveContainer" containerID="08576b134467a8f50112a4c850580943c077bf8e512af61caecfff5c63319797"
Jul 26 18:19:30.624133 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:19:30.624109    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hfvpm\" is forbidden: User \"system:node:ip-10-0-134-7.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-7.ec2.internal' and this object" podUID="e0f6a24a-e27f-4958-850e-ec282d487c01" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hfvpm"
Jul 26 18:19:30.630632 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:30.630615    2576 scope.go:117] "RemoveContainer" containerID="08576b134467a8f50112a4c850580943c077bf8e512af61caecfff5c63319797"
Jul 26 18:19:30.630972 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:19:30.630950    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08576b134467a8f50112a4c850580943c077bf8e512af61caecfff5c63319797\": container with ID starting with 08576b134467a8f50112a4c850580943c077bf8e512af61caecfff5c63319797 not found: ID does not exist" containerID="08576b134467a8f50112a4c850580943c077bf8e512af61caecfff5c63319797"
Jul 26 18:19:30.631041 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:30.630981    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08576b134467a8f50112a4c850580943c077bf8e512af61caecfff5c63319797"} err="failed to get container status \"08576b134467a8f50112a4c850580943c077bf8e512af61caecfff5c63319797\": rpc error: code = NotFound desc = could not find container \"08576b134467a8f50112a4c850580943c077bf8e512af61caecfff5c63319797\": container with ID starting with 08576b134467a8f50112a4c850580943c077bf8e512af61caecfff5c63319797 not found: ID does not exist"
Jul 26 18:19:30.648331 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:30.648284    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-867cs" podStartSLOduration=1.648269069 podStartE2EDuration="1.648269069s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:19:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:19:30.647812999 +0000 UTC m=+456.355220060" watchObservedRunningTime="2026-07-26 18:19:30.648269069 +0000 UTC m=+456.355676127"
Jul 26 18:19:30.650515 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:19:30.650488    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hfvpm\" is forbidden: User \"system:node:ip-10-0-134-7.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-7.ec2.internal' and this object" podUID="e0f6a24a-e27f-4958-850e-ec282d487c01" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hfvpm"
Jul 26 18:19:30.848867 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:30.848785    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0f6a24a-e27f-4958-850e-ec282d487c01" path="/var/lib/kubelet/pods/e0f6a24a-e27f-4958-850e-ec282d487c01/volumes"
Jul 26 18:19:31.423540 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:31.423500    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:19:31.523178 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:31.523136    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:19:32.007943 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:32.007908    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:19:32.116528 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:32.116494    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:19:34.224794 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:34.224759    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:19:34.319609 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:34.319555    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:19:39.591629 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:39.591592    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:19:39.700187 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:39.700147    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:19:41.629827 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:41.629792    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-867cs"
Jul 26 18:19:42.149683 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:42.149645    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:19:42.244070 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:42.244027    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:19:49.102010 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:49.101968    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:19:49.202813 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:19:49.202768    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:20:01.130047 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:20:01.130017    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:20:09.346321 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:20:09.346259    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:20:09.446519 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:20:09.446480    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:20:15.838610 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:20:15.838568    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:20:15.981608 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:20:15.981574    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:20:20.018285 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:20:20.018240    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:20:20.162572 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:20:20.162533    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:20:20.391544 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:20:20.391451    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:20:20.394492 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:20:20.394475    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"
Jul 26 18:20:20.397393 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:20:20.397363    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-rvvv5\""
Jul 26 18:20:20.397393 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:20:20.397365    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 26 18:20:20.406162 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:20:20.406139    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:20:20.456115 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:20:20.456082    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:20:20.527084 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:20:20.527052    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/31bcb2be-8928-438b-b59f-11414eff93ac-config-file\") pod \"limitador-limitador-54d6d988d5-pqvlq\" (UID: \"31bcb2be-8928-438b-b59f-11414eff93ac\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"
Jul 26 18:20:20.527237 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:20:20.527095    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xww44\" (UniqueName: \"kubernetes.io/projected/31bcb2be-8928-438b-b59f-11414eff93ac-kube-api-access-xww44\") pod \"limitador-limitador-54d6d988d5-pqvlq\" (UID: \"31bcb2be-8928-438b-b59f-11414eff93ac\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"
Jul 26 18:20:20.628063 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:20:20.628012    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/31bcb2be-8928-438b-b59f-11414eff93ac-config-file\") pod \"limitador-limitador-54d6d988d5-pqvlq\" (UID: \"31bcb2be-8928-438b-b59f-11414eff93ac\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"
Jul 26 18:20:20.628245 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:20:20.628081    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xww44\" (UniqueName: \"kubernetes.io/projected/31bcb2be-8928-438b-b59f-11414eff93ac-kube-api-access-xww44\") pod \"limitador-limitador-54d6d988d5-pqvlq\" (UID: \"31bcb2be-8928-438b-b59f-11414eff93ac\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"
Jul 26 18:20:20.628696 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:20:20.628675    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/31bcb2be-8928-438b-b59f-11414eff93ac-config-file\") pod \"limitador-limitador-54d6d988d5-pqvlq\" (UID: \"31bcb2be-8928-438b-b59f-11414eff93ac\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"
Jul 26 18:20:20.636829 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:20:20.636804    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xww44\" (UniqueName: \"kubernetes.io/projected/31bcb2be-8928-438b-b59f-11414eff93ac-kube-api-access-xww44\") pod \"limitador-limitador-54d6d988d5-pqvlq\" (UID: \"31bcb2be-8928-438b-b59f-11414eff93ac\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"
Jul 26 18:20:20.704918 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:20:20.704880    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"
Jul 26 18:20:20.833000 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:20:20.832968    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:20:20.836283 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:20:20.836251    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31bcb2be_8928_438b_b59f_11414eff93ac.slice/crio-abe57954af0b4b0b555ca90e3f16c1164b78418c99f8db472291c7292f54f681 WatchSource:0}: Error finding container abe57954af0b4b0b555ca90e3f16c1164b78418c99f8db472291c7292f54f681: Status 404 returned error can't find the container with id abe57954af0b4b0b555ca90e3f16c1164b78418c99f8db472291c7292f54f681
Jul 26 18:20:20.911700 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:20:20.911661    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:20:21.013788 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:20:21.013679    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:20:21.824443 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:20:21.824405    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-pqvlq" event={"ID":"31bcb2be-8928-438b-b59f-11414eff93ac","Type":"ContainerStarted","Data":"abe57954af0b4b0b555ca90e3f16c1164b78418c99f8db472291c7292f54f681"}
Jul 26 18:20:23.833701 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:20:23.833668    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-pqvlq" event={"ID":"31bcb2be-8928-438b-b59f-11414eff93ac","Type":"ContainerStarted","Data":"d53f391b1552251de5baded3990c6f8b1b1c0dd4e0709738d675f3a6cdf53580"}
Jul 26 18:20:23.834086 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:20:23.833787    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"
Jul 26 18:20:23.851291 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:20:23.851241    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-pqvlq" podStartSLOduration=1.49378889 podStartE2EDuration="3.851222352s" totalImagesPullingTime="2.357433462s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:20:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:20:23.850736136 +0000 UTC m=+509.558143196" watchObservedRunningTime="2026-07-26 18:20:23.851222352 +0000 UTC m=+509.558629411"
Jul 26 18:20:34.839635 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:20:34.839603    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"
Jul 26 18:20:39.506245 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:20:39.506206    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:20:39.618714 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:20:39.618681    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:20:39.974215 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:20:39.974181    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:20:52.997547 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:20:52.997460    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:20:53.138189 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:20:53.138152    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:21:00.246255 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:21:00.246215    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:21:00.375560 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:21:00.375521    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:21:00.539733 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:21:00.539640    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:21:04.610947 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:21:04.610918    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:21:04.715462 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:21:04.715425    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:21:12.378578 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:21:12.378544    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:21:12.480395 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:21:12.480358    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:21:27.332787 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:21:27.332733    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:21:27.400231 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:21:27.400193    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:21:36.350797 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:21:36.350739    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:21:37.587788 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:21:37.587728    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:21:37.695521 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:21:37.695464    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:21:37.783239 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:21:37.783200    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:21:37.876476 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:21:37.876403    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:21:48.793470 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:21:48.793436    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:21:48.891658 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:21:48.891624    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:21:59.928866 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:21:59.928831    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:22:00.399630 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:00.399545    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:22:08.174500 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:08.174459    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:22:09.433368 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:09.433328    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:22:09.994313 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:09.994279    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:22:24.388289 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:24.388205    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:22:24.455227 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:24.455183    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:22:26.116133 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:26.116097    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:22:27.095644 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:27.095609    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:22:37.646113 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:37.646067    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2"]
Jul 26 18:22:37.649216 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:37.649195    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2"
Jul 26 18:22:37.653147 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:37.653113    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-pxsw8\""
Jul 26 18:22:37.653147 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:37.653125    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 26 18:22:37.653317 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:37.653170    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 26 18:22:37.653317 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:37.653184    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 26 18:22:37.662681 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:37.662657    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2"]
Jul 26 18:22:37.672590 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:37.672563    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:22:37.744540 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:37.744508    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/994d0da6-95aa-48c0-94a3-50e3d0ade5a3-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2\" (UID: \"994d0da6-95aa-48c0-94a3-50e3d0ade5a3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2"
Jul 26 18:22:37.744705 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:37.744558    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/994d0da6-95aa-48c0-94a3-50e3d0ade5a3-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2\" (UID: \"994d0da6-95aa-48c0-94a3-50e3d0ade5a3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2"
Jul 26 18:22:37.744705 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:37.744592    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/994d0da6-95aa-48c0-94a3-50e3d0ade5a3-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2\" (UID: \"994d0da6-95aa-48c0-94a3-50e3d0ade5a3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2"
Jul 26 18:22:37.744705 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:37.744612    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5h99\" (UniqueName: \"kubernetes.io/projected/994d0da6-95aa-48c0-94a3-50e3d0ade5a3-kube-api-access-j5h99\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2\" (UID: \"994d0da6-95aa-48c0-94a3-50e3d0ade5a3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2"
Jul 26 18:22:37.744705 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:37.744654    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/994d0da6-95aa-48c0-94a3-50e3d0ade5a3-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2\" (UID: \"994d0da6-95aa-48c0-94a3-50e3d0ade5a3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2"
Jul 26 18:22:37.744705 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:37.744674    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/994d0da6-95aa-48c0-94a3-50e3d0ade5a3-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2\" (UID: \"994d0da6-95aa-48c0-94a3-50e3d0ade5a3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2"
Jul 26 18:22:37.845564 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:37.845533    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/994d0da6-95aa-48c0-94a3-50e3d0ade5a3-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2\" (UID: \"994d0da6-95aa-48c0-94a3-50e3d0ade5a3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2"
Jul 26 18:22:37.845731 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:37.845571    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/994d0da6-95aa-48c0-94a3-50e3d0ade5a3-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2\" (UID: \"994d0da6-95aa-48c0-94a3-50e3d0ade5a3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2"
Jul 26 18:22:37.845731 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:37.845599    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5h99\" (UniqueName: \"kubernetes.io/projected/994d0da6-95aa-48c0-94a3-50e3d0ade5a3-kube-api-access-j5h99\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2\" (UID: \"994d0da6-95aa-48c0-94a3-50e3d0ade5a3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2"
Jul 26 18:22:37.845731 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:37.845632    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/994d0da6-95aa-48c0-94a3-50e3d0ade5a3-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2\" (UID: \"994d0da6-95aa-48c0-94a3-50e3d0ade5a3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2"
Jul 26 18:22:37.845731 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:37.845661    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/994d0da6-95aa-48c0-94a3-50e3d0ade5a3-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2\" (UID: \"994d0da6-95aa-48c0-94a3-50e3d0ade5a3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2"
Jul 26 18:22:37.845731 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:37.845705    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/994d0da6-95aa-48c0-94a3-50e3d0ade5a3-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2\" (UID: \"994d0da6-95aa-48c0-94a3-50e3d0ade5a3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2"
Jul 26 18:22:37.846019 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:37.846001    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/994d0da6-95aa-48c0-94a3-50e3d0ade5a3-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2\" (UID: \"994d0da6-95aa-48c0-94a3-50e3d0ade5a3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2"
Jul 26 18:22:37.846076 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:37.846002    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/994d0da6-95aa-48c0-94a3-50e3d0ade5a3-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2\" (UID: \"994d0da6-95aa-48c0-94a3-50e3d0ade5a3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2"
Jul 26 18:22:37.850772 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:37.846830    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/994d0da6-95aa-48c0-94a3-50e3d0ade5a3-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2\" (UID: \"994d0da6-95aa-48c0-94a3-50e3d0ade5a3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2"
Jul 26 18:22:37.850772 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:37.848477    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/994d0da6-95aa-48c0-94a3-50e3d0ade5a3-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2\" (UID: \"994d0da6-95aa-48c0-94a3-50e3d0ade5a3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2"
Jul 26 18:22:37.850772 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:37.849034    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/994d0da6-95aa-48c0-94a3-50e3d0ade5a3-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2\" (UID: \"994d0da6-95aa-48c0-94a3-50e3d0ade5a3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2"
Jul 26 18:22:37.857141 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:37.857104    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5h99\" (UniqueName: \"kubernetes.io/projected/994d0da6-95aa-48c0-94a3-50e3d0ade5a3-kube-api-access-j5h99\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2\" (UID: \"994d0da6-95aa-48c0-94a3-50e3d0ade5a3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2"
Jul 26 18:22:37.958893 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:37.958857    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2"
Jul 26 18:22:38.090665 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:38.090641    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2"]
Jul 26 18:22:38.093245 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:22:38.093217    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod994d0da6_95aa_48c0_94a3_50e3d0ade5a3.slice/crio-fa8e8cf878a75325452fef90b95eb88d3913ae5b65f201fb4c12191d1de860f4 WatchSource:0}: Error finding container fa8e8cf878a75325452fef90b95eb88d3913ae5b65f201fb4c12191d1de860f4: Status 404 returned error can't find the container with id fa8e8cf878a75325452fef90b95eb88d3913ae5b65f201fb4c12191d1de860f4
Jul 26 18:22:38.094524 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:38.094504    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 18:22:38.362942 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:38.362850    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2" event={"ID":"994d0da6-95aa-48c0-94a3-50e3d0ade5a3","Type":"ContainerStarted","Data":"fa8e8cf878a75325452fef90b95eb88d3913ae5b65f201fb4c12191d1de860f4"}
Jul 26 18:22:39.194885 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:39.194849    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:22:39.368434 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:39.368350    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2" event={"ID":"994d0da6-95aa-48c0-94a3-50e3d0ade5a3","Type":"ContainerStarted","Data":"6cd76c378728c07fc791e476049a7c8664e6f27cf80a79f4c43bfe92b026a566"}
Jul 26 18:22:39.387675 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:39.387629    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2" podStartSLOduration=1.472191808 podStartE2EDuration="2.387612516s" totalImagesPullingTime="915.420708ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:22:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:22:39.387311196 +0000 UTC m=+645.094718250" watchObservedRunningTime="2026-07-26 18:22:39.387612516 +0000 UTC m=+645.095019552"
Jul 26 18:22:46.995739 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:46.995690    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:22:47.960007 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:47.959971    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2"
Jul 26 18:22:47.960174 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:47.960029    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2"
Jul 26 18:22:47.972588 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:47.972555    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2"
Jul 26 18:22:48.417153 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:48.417069    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2"
Jul 26 18:22:49.894876 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:49.894836    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:22:53.528188 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:22:53.528145    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:23:02.896668 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:23:02.896633    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:23:47.541804 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:23:47.541696    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:23:53.435561 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:23:53.435520    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:23:53.503914 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:23:53.503879    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:24:03.027118 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:24:03.027080    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:24:03.216299 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:24:03.216263    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:24:04.128719 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:24:04.128684    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:24:04.385982 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:24:04.385871    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:24:32.089883 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:24:32.089848    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:24:41.237678 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:24:41.237629    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:24:52.087252 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:24:52.087217    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:26:07.639833 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:26:07.639798    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:26:08.850530 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:26:08.850478    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:26:32.417481 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:26:32.417442    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:26:37.146949 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:26:37.146907    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:26:37.241526 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:26:37.241474    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:26:37.371570 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:26:37.371538    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:26:43.047823 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:26:43.047789    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:26:44.197275 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:26:44.197241    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:26:49.159180 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:26:49.159107    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:26:49.259304 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:26:49.259266    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:27:05.797225 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:27:05.797177    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:27:10.923737 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:27:10.923706    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:27:15.384127 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:27:15.384093    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:27:23.784387 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:27:23.784352    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:27:32.882314 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:27:32.882277    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:27:42.887302 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:27:42.887260    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:27:53.599915 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:27:53.599876    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:28:04.286218 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:28:04.286177    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:28:14.187504 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:28:14.187457    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:28:39.298321 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:28:39.298284    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:28:50.394259 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:28:50.394221    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:29:14.500489 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:29:14.500453    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:29:16.208045 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:29:16.208010    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:29:16.299970 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:29:16.299939    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:29:24.500250 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:29:24.500203    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:29:39.301071 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:29:39.301021    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:29:49.704708 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:29:49.704628    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:30:00.159786 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:30:00.159738    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751510-9xzn5"]
Jul 26 18:30:00.162830 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:30:00.162813    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751510-9xzn5"
Jul 26 18:30:00.169631 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:30:00.169610    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxnsx\""
Jul 26 18:30:00.171858 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:30:00.171842    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751510-9xzn5"
Jul 26 18:30:00.186425 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:30:00.186400    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751510-9xzn5"]
Jul 26 18:30:00.316331 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:30:00.316290    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751510-9xzn5"]
Jul 26 18:30:00.319225 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:30:00.319199    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85ebe06d_9b91_4aae_bcc6_5967e656c5f3.slice/crio-8487a553fda7357d386cdf343b2dce6a9dc92bcb94ad24022d54acf0874a9eb0 WatchSource:0}: Error finding container 8487a553fda7357d386cdf343b2dce6a9dc92bcb94ad24022d54acf0874a9eb0: Status 404 returned error can't find the container with id 8487a553fda7357d386cdf343b2dce6a9dc92bcb94ad24022d54acf0874a9eb0
Jul 26 18:30:00.321140 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:30:00.321124    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 18:30:01.021140 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:30:01.021098    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:30:01.144229 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:30:01.144196    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751510-9xzn5" event={"ID":"85ebe06d-9b91-4aae-bcc6-5967e656c5f3","Type":"ContainerStarted","Data":"8487a553fda7357d386cdf343b2dce6a9dc92bcb94ad24022d54acf0874a9eb0"}
Jul 26 18:30:04.160306 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:30:04.160276    2576 generic.go:356] "Generic (PLEG): container finished" podID="85ebe06d-9b91-4aae-bcc6-5967e656c5f3" containerID="43fbd1bebea0b70c0b6b96c6be3de3c4cfdecf1bff7fd677f1570dbe2d55d799" exitCode=0
Jul 26 18:30:04.160613 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:30:04.160314    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751510-9xzn5" event={"ID":"85ebe06d-9b91-4aae-bcc6-5967e656c5f3","Type":"ContainerDied","Data":"43fbd1bebea0b70c0b6b96c6be3de3c4cfdecf1bff7fd677f1570dbe2d55d799"}
Jul 26 18:30:05.285731 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:30:05.285706    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751510-9xzn5"
Jul 26 18:30:06.169157 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:30:06.169119    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751510-9xzn5" event={"ID":"85ebe06d-9b91-4aae-bcc6-5967e656c5f3","Type":"ContainerDied","Data":"8487a553fda7357d386cdf343b2dce6a9dc92bcb94ad24022d54acf0874a9eb0"}
Jul 26 18:30:06.169157 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:30:06.169146    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751510-9xzn5"
Jul 26 18:30:06.169437 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:30:06.169171    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8487a553fda7357d386cdf343b2dce6a9dc92bcb94ad24022d54acf0874a9eb0"
Jul 26 18:30:10.693689 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:30:10.693658    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:30:20.084850 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:30:20.084810    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:31:23.581029 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:31:23.580941    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:31:23.669470 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:31:23.669431    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:32:04.836882 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:32:04.836843    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:32:07.490619 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:32:07.490583    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:32:15.433879 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:32:15.433846    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:32:18.654064 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:32:18.654025    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:32:30.498946 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:32:30.498908    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:32:41.890787 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:32:41.890738    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:32:52.496596 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:32:52.496515    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:33:04.395516 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:33:04.395478    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:33:12.784294 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:33:12.784252    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:33:23.687655 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:33:23.687619    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:33:35.504472 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:33:35.504433    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:33:44.689975 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:33:44.689939    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:33:55.999604 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:33:55.999562    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:34:17.890477 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:17.890385    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:34:26.690194 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:26.690159    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:34:32.602216 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:32.602183    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m"]
Jul 26 18:34:32.602595 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:32.602496    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85ebe06d-9b91-4aae-bcc6-5967e656c5f3" containerName="cleanup"
Jul 26 18:34:32.602595 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:32.602508    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="85ebe06d-9b91-4aae-bcc6-5967e656c5f3" containerName="cleanup"
Jul 26 18:34:32.602595 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:32.602573    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="85ebe06d-9b91-4aae-bcc6-5967e656c5f3" containerName="cleanup"
Jul 26 18:34:32.604572 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:32.604553    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m"
Jul 26 18:34:32.607056 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:32.607029    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-d30d1f16-kserve-self-signed-certs\""
Jul 26 18:34:32.621499 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:32.621470    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m"]
Jul 26 18:34:32.625267 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:32.625228    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23a200e7-0b79-4609-a13a-49e2307277dc-dshm\") pod \"e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m\" (UID: \"23a200e7-0b79-4609-a13a-49e2307277dc\") " pod="llm/e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m"
Jul 26 18:34:32.625398 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:32.625283    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23a200e7-0b79-4609-a13a-49e2307277dc-model-cache\") pod \"e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m\" (UID: \"23a200e7-0b79-4609-a13a-49e2307277dc\") " pod="llm/e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m"
Jul 26 18:34:32.625398 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:32.625315    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23a200e7-0b79-4609-a13a-49e2307277dc-tls-certs\") pod \"e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m\" (UID: \"23a200e7-0b79-4609-a13a-49e2307277dc\") " pod="llm/e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m"
Jul 26 18:34:32.625398 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:32.625356    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76kgz\" (UniqueName: \"kubernetes.io/projected/23a200e7-0b79-4609-a13a-49e2307277dc-kube-api-access-76kgz\") pod \"e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m\" (UID: \"23a200e7-0b79-4609-a13a-49e2307277dc\") " pod="llm/e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m"
Jul 26 18:34:32.625501 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:32.625418    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23a200e7-0b79-4609-a13a-49e2307277dc-home\") pod \"e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m\" (UID: \"23a200e7-0b79-4609-a13a-49e2307277dc\") " pod="llm/e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m"
Jul 26 18:34:32.625501 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:32.625441    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23a200e7-0b79-4609-a13a-49e2307277dc-tmp-dir\") pod \"e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m\" (UID: \"23a200e7-0b79-4609-a13a-49e2307277dc\") " pod="llm/e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m"
Jul 26 18:34:32.668199 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:32.668160    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:34:32.726058 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:32.726025    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23a200e7-0b79-4609-a13a-49e2307277dc-home\") pod \"e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m\" (UID: \"23a200e7-0b79-4609-a13a-49e2307277dc\") " pod="llm/e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m"
Jul 26 18:34:32.726058 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:32.726057    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23a200e7-0b79-4609-a13a-49e2307277dc-tmp-dir\") pod \"e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m\" (UID: \"23a200e7-0b79-4609-a13a-49e2307277dc\") " pod="llm/e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m"
Jul 26 18:34:32.726290 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:32.726098    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23a200e7-0b79-4609-a13a-49e2307277dc-dshm\") pod \"e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m\" (UID: \"23a200e7-0b79-4609-a13a-49e2307277dc\") " pod="llm/e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m"
Jul 26 18:34:32.726290 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:32.726119    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23a200e7-0b79-4609-a13a-49e2307277dc-model-cache\") pod \"e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m\" (UID: \"23a200e7-0b79-4609-a13a-49e2307277dc\") " pod="llm/e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m"
Jul 26 18:34:32.726290 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:32.726145    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23a200e7-0b79-4609-a13a-49e2307277dc-tls-certs\") pod \"e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m\" (UID: \"23a200e7-0b79-4609-a13a-49e2307277dc\") " pod="llm/e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m"
Jul 26 18:34:32.726290 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:32.726174    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76kgz\" (UniqueName: \"kubernetes.io/projected/23a200e7-0b79-4609-a13a-49e2307277dc-kube-api-access-76kgz\") pod \"e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m\" (UID: \"23a200e7-0b79-4609-a13a-49e2307277dc\") " pod="llm/e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m"
Jul 26 18:34:32.726542 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:32.726480    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23a200e7-0b79-4609-a13a-49e2307277dc-tmp-dir\") pod \"e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m\" (UID: \"23a200e7-0b79-4609-a13a-49e2307277dc\") " pod="llm/e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m"
Jul 26 18:34:32.726542 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:32.726516    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23a200e7-0b79-4609-a13a-49e2307277dc-home\") pod \"e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m\" (UID: \"23a200e7-0b79-4609-a13a-49e2307277dc\") " pod="llm/e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m"
Jul 26 18:34:32.726650 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:32.726627    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23a200e7-0b79-4609-a13a-49e2307277dc-model-cache\") pod \"e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m\" (UID: \"23a200e7-0b79-4609-a13a-49e2307277dc\") " pod="llm/e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m"
Jul 26 18:34:32.728400 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:32.728379    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23a200e7-0b79-4609-a13a-49e2307277dc-dshm\") pod \"e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m\" (UID: \"23a200e7-0b79-4609-a13a-49e2307277dc\") " pod="llm/e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m"
Jul 26 18:34:32.728658 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:32.728637    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23a200e7-0b79-4609-a13a-49e2307277dc-tls-certs\") pod \"e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m\" (UID: \"23a200e7-0b79-4609-a13a-49e2307277dc\") " pod="llm/e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m"
Jul 26 18:34:32.735132 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:32.735108    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76kgz\" (UniqueName: \"kubernetes.io/projected/23a200e7-0b79-4609-a13a-49e2307277dc-kube-api-access-76kgz\") pod \"e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m\" (UID: \"23a200e7-0b79-4609-a13a-49e2307277dc\") " pod="llm/e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m"
Jul 26 18:34:32.914592 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:32.914496    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m"
Jul 26 18:34:33.050435 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:33.050396    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m"]
Jul 26 18:34:33.054248 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:34:33.054215    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod23a200e7_0b79_4609_a13a_49e2307277dc.slice/crio-aea0d9bdb24e41119c0f62ac676921ea490d5c830264aba84c8fdb9aae04f47e WatchSource:0}: Error finding container aea0d9bdb24e41119c0f62ac676921ea490d5c830264aba84c8fdb9aae04f47e: Status 404 returned error can't find the container with id aea0d9bdb24e41119c0f62ac676921ea490d5c830264aba84c8fdb9aae04f47e
Jul 26 18:34:33.261142 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:33.261105    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m" event={"ID":"23a200e7-0b79-4609-a13a-49e2307277dc","Type":"ContainerStarted","Data":"fa7a1b9b8ced01f44dcc27720327319178fef4f9a207db5de9040bef18c587bc"}
Jul 26 18:34:33.261142 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:33.261146    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m" event={"ID":"23a200e7-0b79-4609-a13a-49e2307277dc","Type":"ContainerStarted","Data":"aea0d9bdb24e41119c0f62ac676921ea490d5c830264aba84c8fdb9aae04f47e"}
Jul 26 18:34:33.281036 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:33.280987    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m" podStartSLOduration=1.280971487 podStartE2EDuration="1.280971487s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:34:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:34:33.279605028 +0000 UTC m=+1358.987012086" watchObservedRunningTime="2026-07-26 18:34:33.280971487 +0000 UTC m=+1358.988378546"
Jul 26 18:34:37.290969 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:37.290931    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:34:41.224929 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:41.224889    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:34:42.915724 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:42.915625    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m"
Jul 26 18:34:42.915724 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:42.915679    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m"
Jul 26 18:34:42.928278 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:42.928246    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m"
Jul 26 18:34:43.315896 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:43.315868    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m"
Jul 26 18:34:52.791104 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:34:52.791066    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:35:16.720116 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:35:16.720082    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m"]
Jul 26 18:35:16.741397 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:35:16.741358    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:35:18.346560 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:35:18.346513    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:35:19.286940 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:35:19.286905    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:35:31.739141 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:35:31.739091    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m" podUID="23a200e7-0b79-4609-a13a-49e2307277dc" containerName="main" containerID="cri-o://fa7a1b9b8ced01f44dcc27720327319178fef4f9a207db5de9040bef18c587bc" gracePeriod=45
Jul 26 18:35:31.993294 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:35:31.993230    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m"
Jul 26 18:35:32.121218 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:35:32.121175    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23a200e7-0b79-4609-a13a-49e2307277dc-home\") pod \"23a200e7-0b79-4609-a13a-49e2307277dc\" (UID: \"23a200e7-0b79-4609-a13a-49e2307277dc\") "
Jul 26 18:35:32.121417 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:35:32.121237    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23a200e7-0b79-4609-a13a-49e2307277dc-tmp-dir\") pod \"23a200e7-0b79-4609-a13a-49e2307277dc\" (UID: \"23a200e7-0b79-4609-a13a-49e2307277dc\") "
Jul 26 18:35:32.121417 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:35:32.121265    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23a200e7-0b79-4609-a13a-49e2307277dc-tls-certs\") pod \"23a200e7-0b79-4609-a13a-49e2307277dc\" (UID: \"23a200e7-0b79-4609-a13a-49e2307277dc\") "
Jul 26 18:35:32.121417 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:35:32.121286    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23a200e7-0b79-4609-a13a-49e2307277dc-model-cache\") pod \"23a200e7-0b79-4609-a13a-49e2307277dc\" (UID: \"23a200e7-0b79-4609-a13a-49e2307277dc\") "
Jul 26 18:35:32.121417 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:35:32.121318    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23a200e7-0b79-4609-a13a-49e2307277dc-dshm\") pod \"23a200e7-0b79-4609-a13a-49e2307277dc\" (UID: \"23a200e7-0b79-4609-a13a-49e2307277dc\") "
Jul 26 18:35:32.121417 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:35:32.121368    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76kgz\" (UniqueName: \"kubernetes.io/projected/23a200e7-0b79-4609-a13a-49e2307277dc-kube-api-access-76kgz\") pod \"23a200e7-0b79-4609-a13a-49e2307277dc\" (UID: \"23a200e7-0b79-4609-a13a-49e2307277dc\") "
Jul 26 18:35:32.121669 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:35:32.121493    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23a200e7-0b79-4609-a13a-49e2307277dc-home" (OuterVolumeSpecName: "home") pod "23a200e7-0b79-4609-a13a-49e2307277dc" (UID: "23a200e7-0b79-4609-a13a-49e2307277dc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:35:32.121669 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:35:32.121548    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23a200e7-0b79-4609-a13a-49e2307277dc-model-cache" (OuterVolumeSpecName: "model-cache") pod "23a200e7-0b79-4609-a13a-49e2307277dc" (UID: "23a200e7-0b79-4609-a13a-49e2307277dc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:35:32.121669 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:35:32.121567    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23a200e7-0b79-4609-a13a-49e2307277dc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "23a200e7-0b79-4609-a13a-49e2307277dc" (UID: "23a200e7-0b79-4609-a13a-49e2307277dc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:35:32.121669 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:35:32.121583    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23a200e7-0b79-4609-a13a-49e2307277dc-home\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:35:32.123620 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:35:32.123598    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23a200e7-0b79-4609-a13a-49e2307277dc-dshm" (OuterVolumeSpecName: "dshm") pod "23a200e7-0b79-4609-a13a-49e2307277dc" (UID: "23a200e7-0b79-4609-a13a-49e2307277dc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:35:32.123697 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:35:32.123648    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23a200e7-0b79-4609-a13a-49e2307277dc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "23a200e7-0b79-4609-a13a-49e2307277dc" (UID: "23a200e7-0b79-4609-a13a-49e2307277dc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:35:32.123697 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:35:32.123673    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23a200e7-0b79-4609-a13a-49e2307277dc-kube-api-access-76kgz" (OuterVolumeSpecName: "kube-api-access-76kgz") pod "23a200e7-0b79-4609-a13a-49e2307277dc" (UID: "23a200e7-0b79-4609-a13a-49e2307277dc"). InnerVolumeSpecName "kube-api-access-76kgz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:35:32.222548 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:35:32.222512    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23a200e7-0b79-4609-a13a-49e2307277dc-dshm\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:35:32.222548 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:35:32.222544    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76kgz\" (UniqueName: \"kubernetes.io/projected/23a200e7-0b79-4609-a13a-49e2307277dc-kube-api-access-76kgz\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:35:32.222548 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:35:32.222557    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23a200e7-0b79-4609-a13a-49e2307277dc-tmp-dir\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:35:32.222806 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:35:32.222566    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23a200e7-0b79-4609-a13a-49e2307277dc-tls-certs\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:35:32.222806 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:35:32.222574    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23a200e7-0b79-4609-a13a-49e2307277dc-model-cache\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:35:32.512426 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:35:32.512389    2576 generic.go:356] "Generic (PLEG): container finished" podID="23a200e7-0b79-4609-a13a-49e2307277dc" containerID="fa7a1b9b8ced01f44dcc27720327319178fef4f9a207db5de9040bef18c587bc" exitCode=0
Jul 26 18:35:32.512596 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:35:32.512431    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m"
Jul 26 18:35:32.512596 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:35:32.512444    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m" event={"ID":"23a200e7-0b79-4609-a13a-49e2307277dc","Type":"ContainerDied","Data":"fa7a1b9b8ced01f44dcc27720327319178fef4f9a207db5de9040bef18c587bc"}
Jul 26 18:35:32.512596 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:35:32.512480    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m" event={"ID":"23a200e7-0b79-4609-a13a-49e2307277dc","Type":"ContainerDied","Data":"aea0d9bdb24e41119c0f62ac676921ea490d5c830264aba84c8fdb9aae04f47e"}
Jul 26 18:35:32.512596 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:35:32.512500    2576 scope.go:117] "RemoveContainer" containerID="fa7a1b9b8ced01f44dcc27720327319178fef4f9a207db5de9040bef18c587bc"
Jul 26 18:35:32.521008 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:35:32.520984    2576 scope.go:117] "RemoveContainer" containerID="fa7a1b9b8ced01f44dcc27720327319178fef4f9a207db5de9040bef18c587bc"
Jul 26 18:35:32.521284 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:35:32.521263    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa7a1b9b8ced01f44dcc27720327319178fef4f9a207db5de9040bef18c587bc\": container with ID starting with fa7a1b9b8ced01f44dcc27720327319178fef4f9a207db5de9040bef18c587bc not found: ID does not exist" containerID="fa7a1b9b8ced01f44dcc27720327319178fef4f9a207db5de9040bef18c587bc"
Jul 26 18:35:32.521343 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:35:32.521294    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa7a1b9b8ced01f44dcc27720327319178fef4f9a207db5de9040bef18c587bc"} err="failed to get container status \"fa7a1b9b8ced01f44dcc27720327319178fef4f9a207db5de9040bef18c587bc\": rpc error: code = NotFound desc = could not find container \"fa7a1b9b8ced01f44dcc27720327319178fef4f9a207db5de9040bef18c587bc\": container with ID starting with fa7a1b9b8ced01f44dcc27720327319178fef4f9a207db5de9040bef18c587bc not found: ID does not exist"
Jul 26 18:35:32.534758 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:35:32.534708    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m"]
Jul 26 18:35:32.538230 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:35:32.538203    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-d30d1f16-kserve-bb8b74cc9-2p29m"]
Jul 26 18:35:32.848303 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:35:32.848226    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23a200e7-0b79-4609-a13a-49e2307277dc" path="/var/lib/kubelet/pods/23a200e7-0b79-4609-a13a-49e2307277dc/volumes"
Jul 26 18:35:33.784508 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:35:33.784466    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:35:40.585647 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:35:40.585603    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:35:57.084203 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:35:57.084167    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:36:06.489953 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:36:06.489919    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:36:15.711341 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:36:15.711302    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:36:25.697415 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:36:25.697369    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:36:34.592842 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:36:34.592805    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:36:43.597350 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:36:43.597314    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:36:56.695204 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:36:56.695167    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:37:05.835812 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:37:05.835771    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:37:08.378809 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:37:08.378775    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:37:09.524922 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:37:09.524885    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:37:15.198414 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:37:15.198374    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:37:26.688121 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:37:26.688035    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:37:45.648661 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:37:45.648628    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:37:56.104874 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:37:56.104836    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:38:06.889896 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:06.889859    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:38:20.394565 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:20.394531    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:38:22.190554 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:22.190506    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:38:27.655655 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:27.655609    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:38:33.210434 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:33.210394    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:38:43.385446 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:43.385408    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:38:43.627311 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:43.627268    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:38:46.554602 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:46.554566    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:38:46.652414 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:46.652369    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:38:46.972855 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:46.972822    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:38:52.978064 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:52.978024    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:38:53.080248 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:53.080218    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:38:53.102921 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:53.102888    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:38:53.224601 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:53.224557    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:38:53.616297 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:53.616259    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:38:53.712008 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:53.711966    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:38:56.165203 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:56.165160    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:38:56.250899 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:56.250866    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:38:56.427288 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:56.427246    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-67813b6a-9f8947656-75789"]
Jul 26 18:38:56.427619 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:56.427602    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23a200e7-0b79-4609-a13a-49e2307277dc" containerName="main"
Jul 26 18:38:56.427695 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:56.427622    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="23a200e7-0b79-4609-a13a-49e2307277dc" containerName="main"
Jul 26 18:38:56.427768 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:56.427728    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="23a200e7-0b79-4609-a13a-49e2307277dc" containerName="main"
Jul 26 18:38:56.430562 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:56.430539    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67813b6a-9f8947656-75789"
Jul 26 18:38:56.433439 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:56.433416    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 18:38:56.433577 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:56.433535    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9n9b7\""
Jul 26 18:38:56.434483 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:56.434465    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-67813b6a-serving-cert\""
Jul 26 18:38:56.434566 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:56.434483    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 26 18:38:56.443291 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:56.443259    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-67813b6a-9f8947656-75789"]
Jul 26 18:38:56.464515 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:56.464488    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:38:56.603632 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:56.603599    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/224694f0-d627-459a-9aad-cd3a38392504-maas-api-tls\") pod \"maas-api-e2e-ait-67813b6a-9f8947656-75789\" (UID: \"224694f0-d627-459a-9aad-cd3a38392504\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67813b6a-9f8947656-75789"
Jul 26 18:38:56.603810 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:56.603645    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frv7s\" (UniqueName: \"kubernetes.io/projected/224694f0-d627-459a-9aad-cd3a38392504-kube-api-access-frv7s\") pod \"maas-api-e2e-ait-67813b6a-9f8947656-75789\" (UID: \"224694f0-d627-459a-9aad-cd3a38392504\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67813b6a-9f8947656-75789"
Jul 26 18:38:56.603810 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:56.603735    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/224694f0-d627-459a-9aad-cd3a38392504-openshift-service-ca\") pod \"maas-api-e2e-ait-67813b6a-9f8947656-75789\" (UID: \"224694f0-d627-459a-9aad-cd3a38392504\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67813b6a-9f8947656-75789"
Jul 26 18:38:56.631327 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:56.631294    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:38:56.705069 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:56.704992    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/224694f0-d627-459a-9aad-cd3a38392504-maas-api-tls\") pod \"maas-api-e2e-ait-67813b6a-9f8947656-75789\" (UID: \"224694f0-d627-459a-9aad-cd3a38392504\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67813b6a-9f8947656-75789"
Jul 26 18:38:56.705069 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:56.705035    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frv7s\" (UniqueName: \"kubernetes.io/projected/224694f0-d627-459a-9aad-cd3a38392504-kube-api-access-frv7s\") pod \"maas-api-e2e-ait-67813b6a-9f8947656-75789\" (UID: \"224694f0-d627-459a-9aad-cd3a38392504\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67813b6a-9f8947656-75789"
Jul 26 18:38:56.705232 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:56.705083    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/224694f0-d627-459a-9aad-cd3a38392504-openshift-service-ca\") pod \"maas-api-e2e-ait-67813b6a-9f8947656-75789\" (UID: \"224694f0-d627-459a-9aad-cd3a38392504\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67813b6a-9f8947656-75789"
Jul 26 18:38:56.705771 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:56.705724    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/224694f0-d627-459a-9aad-cd3a38392504-openshift-service-ca\") pod \"maas-api-e2e-ait-67813b6a-9f8947656-75789\" (UID: \"224694f0-d627-459a-9aad-cd3a38392504\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67813b6a-9f8947656-75789"
Jul 26 18:38:56.707407 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:56.707384    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/224694f0-d627-459a-9aad-cd3a38392504-maas-api-tls\") pod \"maas-api-e2e-ait-67813b6a-9f8947656-75789\" (UID: \"224694f0-d627-459a-9aad-cd3a38392504\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67813b6a-9f8947656-75789"
Jul 26 18:38:56.714407 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:56.714387    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frv7s\" (UniqueName: \"kubernetes.io/projected/224694f0-d627-459a-9aad-cd3a38392504-kube-api-access-frv7s\") pod \"maas-api-e2e-ait-67813b6a-9f8947656-75789\" (UID: \"224694f0-d627-459a-9aad-cd3a38392504\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67813b6a-9f8947656-75789"
Jul 26 18:38:56.743371 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:56.743346    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67813b6a-9f8947656-75789"
Jul 26 18:38:56.880574 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:56.880552    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-67813b6a-9f8947656-75789"]
Jul 26 18:38:56.882460 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:38:56.882430    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod224694f0_d627_459a_9aad_cd3a38392504.slice/crio-9ced1319e29a594c3518b409c927d5b25e0ff0871053b99a66dc1af821f43fbb WatchSource:0}: Error finding container 9ced1319e29a594c3518b409c927d5b25e0ff0871053b99a66dc1af821f43fbb: Status 404 returned error can't find the container with id 9ced1319e29a594c3518b409c927d5b25e0ff0871053b99a66dc1af821f43fbb
Jul 26 18:38:56.883822 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:56.883800    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 18:38:56.994117 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:56.994040    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:38:57.096902 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:57.096868    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:38:57.180996 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:57.180955    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-67813b6a-7897757bd8-625h6"]
Jul 26 18:38:57.185151 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:57.185135    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-67813b6a-7897757bd8-625h6"
Jul 26 18:38:57.188568 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:57.188546    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-67813b6a\""
Jul 26 18:38:57.188811 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:57.188796    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-67813b6a-dockercfg-rf8t9\""
Jul 26 18:38:57.204353 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:57.204332    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-67813b6a-7897757bd8-625h6"]
Jul 26 18:38:57.310408 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:57.310325    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tj52j\" (UniqueName: \"kubernetes.io/projected/95b9c869-fbab-4ba6-b45b-face5a53fa2e-kube-api-access-tj52j\") pod \"payload-pre-processing-e2e-ait-67813b6a-7897757bd8-625h6\" (UID: \"95b9c869-fbab-4ba6-b45b-face5a53fa2e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-67813b6a-7897757bd8-625h6"
Jul 26 18:38:57.310408 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:57.310394    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95b9c869-fbab-4ba6-b45b-face5a53fa2e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-67813b6a-7897757bd8-625h6\" (UID: \"95b9c869-fbab-4ba6-b45b-face5a53fa2e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-67813b6a-7897757bd8-625h6"
Jul 26 18:38:57.359538 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:57.359506    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67813b6a-9f8947656-75789" event={"ID":"224694f0-d627-459a-9aad-cd3a38392504","Type":"ContainerStarted","Data":"9ced1319e29a594c3518b409c927d5b25e0ff0871053b99a66dc1af821f43fbb"}
Jul 26 18:38:57.410988 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:57.410957    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tj52j\" (UniqueName: \"kubernetes.io/projected/95b9c869-fbab-4ba6-b45b-face5a53fa2e-kube-api-access-tj52j\") pod \"payload-pre-processing-e2e-ait-67813b6a-7897757bd8-625h6\" (UID: \"95b9c869-fbab-4ba6-b45b-face5a53fa2e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-67813b6a-7897757bd8-625h6"
Jul 26 18:38:57.411161 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:57.411032    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95b9c869-fbab-4ba6-b45b-face5a53fa2e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-67813b6a-7897757bd8-625h6\" (UID: \"95b9c869-fbab-4ba6-b45b-face5a53fa2e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-67813b6a-7897757bd8-625h6"
Jul 26 18:38:57.411839 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:57.411821    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95b9c869-fbab-4ba6-b45b-face5a53fa2e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-67813b6a-7897757bd8-625h6\" (UID: \"95b9c869-fbab-4ba6-b45b-face5a53fa2e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-67813b6a-7897757bd8-625h6"
Jul 26 18:38:57.420097 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:57.420079    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tj52j\" (UniqueName: \"kubernetes.io/projected/95b9c869-fbab-4ba6-b45b-face5a53fa2e-kube-api-access-tj52j\") pod \"payload-pre-processing-e2e-ait-67813b6a-7897757bd8-625h6\" (UID: \"95b9c869-fbab-4ba6-b45b-face5a53fa2e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-67813b6a-7897757bd8-625h6"
Jul 26 18:38:57.495910 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:57.495877    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-67813b6a-7897757bd8-625h6"
Jul 26 18:38:57.652985 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:57.652924    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-67813b6a-7897757bd8-625h6"]
Jul 26 18:38:57.655548 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:38:57.655464    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95b9c869_fbab_4ba6_b45b_face5a53fa2e.slice/crio-3ac418e9cdb54f71668e31cfd276e80cede7aa067547faa06d11668931d034c3 WatchSource:0}: Error finding container 3ac418e9cdb54f71668e31cfd276e80cede7aa067547faa06d11668931d034c3: Status 404 returned error can't find the container with id 3ac418e9cdb54f71668e31cfd276e80cede7aa067547faa06d11668931d034c3
Jul 26 18:38:58.366453 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:38:58.366415    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-67813b6a-7897757bd8-625h6" event={"ID":"95b9c869-fbab-4ba6-b45b-face5a53fa2e","Type":"ContainerStarted","Data":"3ac418e9cdb54f71668e31cfd276e80cede7aa067547faa06d11668931d034c3"}
Jul 26 18:39:01.380790 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:01.380731    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-67813b6a-7897757bd8-625h6" event={"ID":"95b9c869-fbab-4ba6-b45b-face5a53fa2e","Type":"ContainerStarted","Data":"1b65510b13b7f3981a0919063da9062df63ac711348207d7213594a715e5c3d1"}
Jul 26 18:39:01.381239 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:01.380820    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-67813b6a-7897757bd8-625h6"
Jul 26 18:39:01.382242 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:01.382213    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67813b6a-9f8947656-75789" event={"ID":"224694f0-d627-459a-9aad-cd3a38392504","Type":"ContainerStarted","Data":"1188c100657125cb499a36bd5afb55cbcf44012a202c6dc7c00518e0bb306092"}
Jul 26 18:39:01.382372 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:01.382357    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67813b6a-9f8947656-75789"
Jul 26 18:39:01.399438 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:01.399392    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-67813b6a-7897757bd8-625h6" podStartSLOduration=1.727517457 podStartE2EDuration="4.399377561s" totalImagesPullingTime="2.671860104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:38:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:39:01.398165696 +0000 UTC m=+1627.105572756" watchObservedRunningTime="2026-07-26 18:39:01.399377561 +0000 UTC m=+1627.106784599"
Jul 26 18:39:01.417255 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:01.417208    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67813b6a-9f8947656-75789" podStartSLOduration=2.014802108 podStartE2EDuration="5.417192776s" totalImagesPullingTime="3.402390668s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:38:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:39:01.415049001 +0000 UTC m=+1627.122456059" watchObservedRunningTime="2026-07-26 18:39:01.417192776 +0000 UTC m=+1627.124599834"
Jul 26 18:39:03.693529 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:03.693490    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:39:07.392145 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:07.392111    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67813b6a-9f8947656-75789"
Jul 26 18:39:12.388885 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:12.388851    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-67813b6a-7897757bd8-625h6"
Jul 26 18:39:21.599077 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:21.599039    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-67813b6a-mfp86"]
Jul 26 18:39:21.602556 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:21.602539    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-67813b6a-mfp86"
Jul 26 18:39:21.605316 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:21.605296    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxnsx\""
Jul 26 18:39:21.611102 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:21.611077    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-67813b6a-mfp86"]
Jul 26 18:39:21.690307 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:21.690271    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:39:21.696623 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:21.696600    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6533172-f127-4d07-b549-c04a35042488-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-67813b6a-mfp86\" (UID: \"e6533172-f127-4d07-b549-c04a35042488\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-67813b6a-mfp86"
Jul 26 18:39:21.797863 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:21.797827    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6533172-f127-4d07-b549-c04a35042488-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-67813b6a-mfp86\" (UID: \"e6533172-f127-4d07-b549-c04a35042488\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-67813b6a-mfp86"
Jul 26 18:39:21.798436 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:21.798415    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6533172-f127-4d07-b549-c04a35042488-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-67813b6a-mfp86\" (UID: \"e6533172-f127-4d07-b549-c04a35042488\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-67813b6a-mfp86"
Jul 26 18:39:21.914451 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:21.914362    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-67813b6a-mfp86"
Jul 26 18:39:22.251381 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:22.251354    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-67813b6a-mfp86"]
Jul 26 18:39:22.253691 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:39:22.253664    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6533172_f127_4d07_b549_c04a35042488.slice/crio-67c0bddf5d157df7236d3bb79bdc398924acfa473420c012a30d940a8f168285 WatchSource:0}: Error finding container 67c0bddf5d157df7236d3bb79bdc398924acfa473420c012a30d940a8f168285: Status 404 returned error can't find the container with id 67c0bddf5d157df7236d3bb79bdc398924acfa473420c012a30d940a8f168285
Jul 26 18:39:22.470812 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:22.470780    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-67813b6a-mfp86" event={"ID":"e6533172-f127-4d07-b549-c04a35042488","Type":"ContainerStarted","Data":"67c0bddf5d157df7236d3bb79bdc398924acfa473420c012a30d940a8f168285"}
Jul 26 18:39:23.476864 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:23.476811    2576 generic.go:356] "Generic (PLEG): container finished" podID="e6533172-f127-4d07-b549-c04a35042488" containerID="16a979f48e03287f34fe8b885fc0290d9d239f418c4e00415f53f6547c42d82b" exitCode=0
Jul 26 18:39:23.477285 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:23.476892    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-67813b6a-mfp86" event={"ID":"e6533172-f127-4d07-b549-c04a35042488","Type":"ContainerDied","Data":"16a979f48e03287f34fe8b885fc0290d9d239f418c4e00415f53f6547c42d82b"}
Jul 26 18:39:24.610243 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:24.610218    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-67813b6a-mfp86"
Jul 26 18:39:24.722144 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:24.722105    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6533172-f127-4d07-b549-c04a35042488-maas-api-service-ca\") pod \"e6533172-f127-4d07-b549-c04a35042488\" (UID: \"e6533172-f127-4d07-b549-c04a35042488\") "
Jul 26 18:39:24.722492 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:24.722468    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6533172-f127-4d07-b549-c04a35042488-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e6533172-f127-4d07-b549-c04a35042488" (UID: "e6533172-f127-4d07-b549-c04a35042488"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:39:24.822631 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:24.822557    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6533172-f127-4d07-b549-c04a35042488-maas-api-service-ca\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:39:25.485889 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:25.485854    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-67813b6a-mfp86"
Jul 26 18:39:25.486078 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:25.485884    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-67813b6a-mfp86" event={"ID":"e6533172-f127-4d07-b549-c04a35042488","Type":"ContainerDied","Data":"67c0bddf5d157df7236d3bb79bdc398924acfa473420c012a30d940a8f168285"}
Jul 26 18:39:25.486078 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:25.485941    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="67c0bddf5d157df7236d3bb79bdc398924acfa473420c012a30d940a8f168285"
Jul 26 18:39:32.111520 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.111472    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-67813b6a-9f8947656-75789"]
Jul 26 18:39:32.111924 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.111776    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67813b6a-9f8947656-75789" podUID="224694f0-d627-459a-9aad-cd3a38392504" containerName="maas-api" containerID="cri-o://1188c100657125cb499a36bd5afb55cbcf44012a202c6dc7c00518e0bb306092" gracePeriod=30
Jul 26 18:39:32.138239 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.138201    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:39:32.194436 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.194399    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-67813b6a-7897757bd8-625h6"]
Jul 26 18:39:32.194623 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.194601    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-67813b6a-7897757bd8-625h6" podUID="95b9c869-fbab-4ba6-b45b-face5a53fa2e" containerName="payload-pre-processing" containerID="cri-o://1b65510b13b7f3981a0919063da9062df63ac711348207d7213594a715e5c3d1" gracePeriod=30
Jul 26 18:39:32.224968 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.224930    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:39:32.325116 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.325084    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:39:32.387840 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.387652    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-ait-67813b6a-7897757bd8-625h6" podUID="95b9c869-fbab-4ba6-b45b-face5a53fa2e" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.29:9004: connect: connection refused"
Jul 26 18:39:32.451802 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.451774    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67813b6a-9f8947656-75789"
Jul 26 18:39:32.472548 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.472527    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-67813b6a-7897757bd8-625h6"
Jul 26 18:39:32.491068 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.491039    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/224694f0-d627-459a-9aad-cd3a38392504-maas-api-tls\") pod \"224694f0-d627-459a-9aad-cd3a38392504\" (UID: \"224694f0-d627-459a-9aad-cd3a38392504\") "
Jul 26 18:39:32.491194 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.491088    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/224694f0-d627-459a-9aad-cd3a38392504-openshift-service-ca\") pod \"224694f0-d627-459a-9aad-cd3a38392504\" (UID: \"224694f0-d627-459a-9aad-cd3a38392504\") "
Jul 26 18:39:32.491194 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.491125    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frv7s\" (UniqueName: \"kubernetes.io/projected/224694f0-d627-459a-9aad-cd3a38392504-kube-api-access-frv7s\") pod \"224694f0-d627-459a-9aad-cd3a38392504\" (UID: \"224694f0-d627-459a-9aad-cd3a38392504\") "
Jul 26 18:39:32.491461 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.491434    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/224694f0-d627-459a-9aad-cd3a38392504-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "224694f0-d627-459a-9aad-cd3a38392504" (UID: "224694f0-d627-459a-9aad-cd3a38392504"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:39:32.493316 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.493291    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/224694f0-d627-459a-9aad-cd3a38392504-kube-api-access-frv7s" (OuterVolumeSpecName: "kube-api-access-frv7s") pod "224694f0-d627-459a-9aad-cd3a38392504" (UID: "224694f0-d627-459a-9aad-cd3a38392504"). InnerVolumeSpecName "kube-api-access-frv7s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:39:32.493384 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.493324    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/224694f0-d627-459a-9aad-cd3a38392504-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "224694f0-d627-459a-9aad-cd3a38392504" (UID: "224694f0-d627-459a-9aad-cd3a38392504"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:39:32.524259 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.524228    2576 generic.go:356] "Generic (PLEG): container finished" podID="224694f0-d627-459a-9aad-cd3a38392504" containerID="1188c100657125cb499a36bd5afb55cbcf44012a202c6dc7c00518e0bb306092" exitCode=0
Jul 26 18:39:32.524454 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.524287    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67813b6a-9f8947656-75789"
Jul 26 18:39:32.524454 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.524314    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67813b6a-9f8947656-75789" event={"ID":"224694f0-d627-459a-9aad-cd3a38392504","Type":"ContainerDied","Data":"1188c100657125cb499a36bd5afb55cbcf44012a202c6dc7c00518e0bb306092"}
Jul 26 18:39:32.524454 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.524391    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67813b6a-9f8947656-75789" event={"ID":"224694f0-d627-459a-9aad-cd3a38392504","Type":"ContainerDied","Data":"9ced1319e29a594c3518b409c927d5b25e0ff0871053b99a66dc1af821f43fbb"}
Jul 26 18:39:32.524454 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.524418    2576 scope.go:117] "RemoveContainer" containerID="1188c100657125cb499a36bd5afb55cbcf44012a202c6dc7c00518e0bb306092"
Jul 26 18:39:32.525726 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.525701    2576 generic.go:356] "Generic (PLEG): container finished" podID="95b9c869-fbab-4ba6-b45b-face5a53fa2e" containerID="1b65510b13b7f3981a0919063da9062df63ac711348207d7213594a715e5c3d1" exitCode=0
Jul 26 18:39:32.525849 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.525725    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-67813b6a-7897757bd8-625h6" event={"ID":"95b9c869-fbab-4ba6-b45b-face5a53fa2e","Type":"ContainerDied","Data":"1b65510b13b7f3981a0919063da9062df63ac711348207d7213594a715e5c3d1"}
Jul 26 18:39:32.525849 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.525785    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-67813b6a-7897757bd8-625h6" event={"ID":"95b9c869-fbab-4ba6-b45b-face5a53fa2e","Type":"ContainerDied","Data":"3ac418e9cdb54f71668e31cfd276e80cede7aa067547faa06d11668931d034c3"}
Jul 26 18:39:32.525849 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.525790    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-67813b6a-7897757bd8-625h6"
Jul 26 18:39:32.533272 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.533257    2576 scope.go:117] "RemoveContainer" containerID="1188c100657125cb499a36bd5afb55cbcf44012a202c6dc7c00518e0bb306092"
Jul 26 18:39:32.533508 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:39:32.533492    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1188c100657125cb499a36bd5afb55cbcf44012a202c6dc7c00518e0bb306092\": container with ID starting with 1188c100657125cb499a36bd5afb55cbcf44012a202c6dc7c00518e0bb306092 not found: ID does not exist" containerID="1188c100657125cb499a36bd5afb55cbcf44012a202c6dc7c00518e0bb306092"
Jul 26 18:39:32.533572 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.533514    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1188c100657125cb499a36bd5afb55cbcf44012a202c6dc7c00518e0bb306092"} err="failed to get container status \"1188c100657125cb499a36bd5afb55cbcf44012a202c6dc7c00518e0bb306092\": rpc error: code = NotFound desc = could not find container \"1188c100657125cb499a36bd5afb55cbcf44012a202c6dc7c00518e0bb306092\": container with ID starting with 1188c100657125cb499a36bd5afb55cbcf44012a202c6dc7c00518e0bb306092 not found: ID does not exist"
Jul 26 18:39:32.533572 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.533529    2576 scope.go:117] "RemoveContainer" containerID="1b65510b13b7f3981a0919063da9062df63ac711348207d7213594a715e5c3d1"
Jul 26 18:39:32.540737 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.540719    2576 scope.go:117] "RemoveContainer" containerID="1b65510b13b7f3981a0919063da9062df63ac711348207d7213594a715e5c3d1"
Jul 26 18:39:32.541049 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:39:32.541026    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b65510b13b7f3981a0919063da9062df63ac711348207d7213594a715e5c3d1\": container with ID starting with 1b65510b13b7f3981a0919063da9062df63ac711348207d7213594a715e5c3d1 not found: ID does not exist" containerID="1b65510b13b7f3981a0919063da9062df63ac711348207d7213594a715e5c3d1"
Jul 26 18:39:32.541132 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.541057    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b65510b13b7f3981a0919063da9062df63ac711348207d7213594a715e5c3d1"} err="failed to get container status \"1b65510b13b7f3981a0919063da9062df63ac711348207d7213594a715e5c3d1\": rpc error: code = NotFound desc = could not find container \"1b65510b13b7f3981a0919063da9062df63ac711348207d7213594a715e5c3d1\": container with ID starting with 1b65510b13b7f3981a0919063da9062df63ac711348207d7213594a715e5c3d1 not found: ID does not exist"
Jul 26 18:39:32.577438 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.577405    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-67813b6a-9f8947656-75789"]
Jul 26 18:39:32.583681 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.583657    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-67813b6a-9f8947656-75789"]
Jul 26 18:39:32.592575 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.592551    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tj52j\" (UniqueName: \"kubernetes.io/projected/95b9c869-fbab-4ba6-b45b-face5a53fa2e-kube-api-access-tj52j\") pod \"95b9c869-fbab-4ba6-b45b-face5a53fa2e\" (UID: \"95b9c869-fbab-4ba6-b45b-face5a53fa2e\") "
Jul 26 18:39:32.592681 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.592594    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95b9c869-fbab-4ba6-b45b-face5a53fa2e-plugins-config-volume\") pod \"95b9c869-fbab-4ba6-b45b-face5a53fa2e\" (UID: \"95b9c869-fbab-4ba6-b45b-face5a53fa2e\") "
Jul 26 18:39:32.592824 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.592808    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/224694f0-d627-459a-9aad-cd3a38392504-maas-api-tls\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:39:32.592882 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.592825    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/224694f0-d627-459a-9aad-cd3a38392504-openshift-service-ca\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:39:32.592882 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.592836    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frv7s\" (UniqueName: \"kubernetes.io/projected/224694f0-d627-459a-9aad-cd3a38392504-kube-api-access-frv7s\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:39:32.592993 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.592965    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95b9c869-fbab-4ba6-b45b-face5a53fa2e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "95b9c869-fbab-4ba6-b45b-face5a53fa2e" (UID: "95b9c869-fbab-4ba6-b45b-face5a53fa2e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:39:32.594712 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.594691    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95b9c869-fbab-4ba6-b45b-face5a53fa2e-kube-api-access-tj52j" (OuterVolumeSpecName: "kube-api-access-tj52j") pod "95b9c869-fbab-4ba6-b45b-face5a53fa2e" (UID: "95b9c869-fbab-4ba6-b45b-face5a53fa2e"). InnerVolumeSpecName "kube-api-access-tj52j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:39:32.693825 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.693789    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tj52j\" (UniqueName: \"kubernetes.io/projected/95b9c869-fbab-4ba6-b45b-face5a53fa2e-kube-api-access-tj52j\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:39:32.693825 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.693821    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95b9c869-fbab-4ba6-b45b-face5a53fa2e-plugins-config-volume\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:39:32.849083 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.849049    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="224694f0-d627-459a-9aad-cd3a38392504" path="/var/lib/kubelet/pods/224694f0-d627-459a-9aad-cd3a38392504/volumes"
Jul 26 18:39:32.849310 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.849296    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-67813b6a-7897757bd8-625h6"]
Jul 26 18:39:32.853182 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:32.853154    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-67813b6a-7897757bd8-625h6"]
Jul 26 18:39:34.848356 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:34.848327    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95b9c869-fbab-4ba6-b45b-face5a53fa2e" path="/var/lib/kubelet/pods/95b9c869-fbab-4ba6-b45b-face5a53fa2e/volumes"
Jul 26 18:39:34.898978 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:34.898944    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:39:35.388287 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:35.388243    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67813b6a-9f8947656-75789" podUID="224694f0-d627-459a-9aad-cd3a38392504" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.28:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 26 18:39:36.701520 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:36.701475    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-67813b6a-mfp86"]
Jul 26 18:39:36.704216 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:36.704190    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-67813b6a-mfp86"]
Jul 26 18:39:36.712424 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:36.712403    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:39:36.849627 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:36.849591    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6533172-f127-4d07-b549-c04a35042488" path="/var/lib/kubelet/pods/e6533172-f127-4d07-b549-c04a35042488/volumes"
Jul 26 18:39:36.868404 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:36.868370    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:39:37.103963 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:37.103875    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:39:37.370427 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:37.370351    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:39:37.466675 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:37.466641    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:39:37.568555 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:37.568521    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:39:37.863417 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:37.863384    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"]
Jul 26 18:39:37.863885 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:37.863702    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95b9c869-fbab-4ba6-b45b-face5a53fa2e" containerName="payload-pre-processing"
Jul 26 18:39:37.863885 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:37.863716    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="95b9c869-fbab-4ba6-b45b-face5a53fa2e" containerName="payload-pre-processing"
Jul 26 18:39:37.863885 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:37.863741    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="224694f0-d627-459a-9aad-cd3a38392504" containerName="maas-api"
Jul 26 18:39:37.863885 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:37.863760    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="224694f0-d627-459a-9aad-cd3a38392504" containerName="maas-api"
Jul 26 18:39:37.863885 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:37.863766    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6533172-f127-4d07-b549-c04a35042488" containerName="revoke-keys"
Jul 26 18:39:37.863885 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:37.863772    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6533172-f127-4d07-b549-c04a35042488" containerName="revoke-keys"
Jul 26 18:39:37.863885 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:37.863825    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="224694f0-d627-459a-9aad-cd3a38392504" containerName="maas-api"
Jul 26 18:39:37.863885 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:37.863834    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6533172-f127-4d07-b549-c04a35042488" containerName="revoke-keys"
Jul 26 18:39:37.863885 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:37.863842    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="95b9c869-fbab-4ba6-b45b-face5a53fa2e" containerName="payload-pre-processing"
Jul 26 18:39:37.870009 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:37.869986    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:39:37.873056 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:37.873039    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-94b43cc4-gw-openshift-default-dockercfg-thxnr\""
Jul 26 18:39:37.882765 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:37.882725    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"]
Jul 26 18:39:37.891194 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:37.891162    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:39:37.938645 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:37.938613    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/165c983b-7a01-4b47-aae0-7b364c7e6ec0-credential-socket\") pod \"e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9\" (UID: \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\") " pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:39:37.938891 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:37.938654    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/165c983b-7a01-4b47-aae0-7b364c7e6ec0-istiod-ca-cert\") pod \"e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9\" (UID: \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\") " pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:39:37.938891 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:37.938689    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/165c983b-7a01-4b47-aae0-7b364c7e6ec0-workload-certs\") pod \"e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9\" (UID: \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\") " pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:39:37.938891 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:37.938718    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/165c983b-7a01-4b47-aae0-7b364c7e6ec0-workload-socket\") pod \"e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9\" (UID: \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\") " pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:39:37.938891 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:37.938759    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/165c983b-7a01-4b47-aae0-7b364c7e6ec0-istio-podinfo\") pod \"e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9\" (UID: \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\") " pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:39:37.938891 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:37.938788    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/165c983b-7a01-4b47-aae0-7b364c7e6ec0-istio-token\") pod \"e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9\" (UID: \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\") " pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:39:37.938891 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:37.938862    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/165c983b-7a01-4b47-aae0-7b364c7e6ec0-istio-envoy\") pod \"e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9\" (UID: \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\") " pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:39:37.938891 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:37.938883    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/165c983b-7a01-4b47-aae0-7b364c7e6ec0-istio-data\") pod \"e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9\" (UID: \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\") " pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:39:37.939170 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:37.938905    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lvzq\" (UniqueName: \"kubernetes.io/projected/165c983b-7a01-4b47-aae0-7b364c7e6ec0-kube-api-access-6lvzq\") pod \"e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9\" (UID: \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\") " pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:39:38.009415 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:38.009380    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:39:38.039607 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:38.039570    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/165c983b-7a01-4b47-aae0-7b364c7e6ec0-istiod-ca-cert\") pod \"e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9\" (UID: \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\") " pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:39:38.039607 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:38.039607    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/165c983b-7a01-4b47-aae0-7b364c7e6ec0-workload-certs\") pod \"e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9\" (UID: \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\") " pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:39:38.039871 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:38.039628    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/165c983b-7a01-4b47-aae0-7b364c7e6ec0-workload-socket\") pod \"e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9\" (UID: \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\") " pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:39:38.039929 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:38.039899    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/165c983b-7a01-4b47-aae0-7b364c7e6ec0-istio-podinfo\") pod \"e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9\" (UID: \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\") " pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:39:38.039985 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:38.039937    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/165c983b-7a01-4b47-aae0-7b364c7e6ec0-istio-token\") pod \"e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9\" (UID: \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\") " pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:39:38.040043 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:38.039999    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/165c983b-7a01-4b47-aae0-7b364c7e6ec0-istio-envoy\") pod \"e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9\" (UID: \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\") " pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:39:38.040167 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:38.040139    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/165c983b-7a01-4b47-aae0-7b364c7e6ec0-workload-socket\") pod \"e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9\" (UID: \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\") " pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:39:38.040313 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:38.040193    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/165c983b-7a01-4b47-aae0-7b364c7e6ec0-workload-certs\") pod \"e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9\" (UID: \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\") " pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:39:38.040421 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:38.040363    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/165c983b-7a01-4b47-aae0-7b364c7e6ec0-istio-data\") pod \"e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9\" (UID: \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\") " pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:39:38.040502 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:38.040432    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/165c983b-7a01-4b47-aae0-7b364c7e6ec0-istiod-ca-cert\") pod \"e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9\" (UID: \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\") " pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:39:38.040502 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:38.040442    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lvzq\" (UniqueName: \"kubernetes.io/projected/165c983b-7a01-4b47-aae0-7b364c7e6ec0-kube-api-access-6lvzq\") pod \"e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9\" (UID: \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\") " pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:39:38.040600 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:38.040506    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/165c983b-7a01-4b47-aae0-7b364c7e6ec0-credential-socket\") pod \"e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9\" (UID: \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\") " pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:39:38.040600 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:38.040529    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/165c983b-7a01-4b47-aae0-7b364c7e6ec0-istio-data\") pod \"e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9\" (UID: \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\") " pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:39:38.040784 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:38.040739    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/165c983b-7a01-4b47-aae0-7b364c7e6ec0-credential-socket\") pod \"e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9\" (UID: \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\") " pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:39:38.050693 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:38.049217    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/165c983b-7a01-4b47-aae0-7b364c7e6ec0-istio-podinfo\") pod \"e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9\" (UID: \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\") " pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:39:38.050693 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:38.050311    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lvzq\" (UniqueName: \"kubernetes.io/projected/165c983b-7a01-4b47-aae0-7b364c7e6ec0-kube-api-access-6lvzq\") pod \"e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9\" (UID: \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\") " pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:39:38.050924 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:38.050874    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/165c983b-7a01-4b47-aae0-7b364c7e6ec0-istio-envoy\") pod \"e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9\" (UID: \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\") " pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:39:38.052417 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:38.052387    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/165c983b-7a01-4b47-aae0-7b364c7e6ec0-istio-token\") pod \"e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9\" (UID: \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\") " pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:39:38.180845 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:38.180811    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:39:38.317796 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:38.317690    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"]
Jul 26 18:39:38.320399 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:39:38.320362    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod165c983b_7a01_4b47_aae0_7b364c7e6ec0.slice/crio-d7b31ac0f492c3d27a0ffa85db81912dc343840492dd0c786b07923918f4d779 WatchSource:0}: Error finding container d7b31ac0f492c3d27a0ffa85db81912dc343840492dd0c786b07923918f4d779: Status 404 returned error can't find the container with id d7b31ac0f492c3d27a0ffa85db81912dc343840492dd0c786b07923918f4d779
Jul 26 18:39:38.557774 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:38.557664    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9" event={"ID":"165c983b-7a01-4b47-aae0-7b364c7e6ec0","Type":"ContainerStarted","Data":"d7b31ac0f492c3d27a0ffa85db81912dc343840492dd0c786b07923918f4d779"}
Jul 26 18:39:40.979590 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:40.979552    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:39:40.979988 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:40.979640    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:39:40.979988 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:40.979682    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:39:41.572820 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:41.572773    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9" event={"ID":"165c983b-7a01-4b47-aae0-7b364c7e6ec0","Type":"ContainerStarted","Data":"7039ec62ad558d05e4dc267fe4a90aafe99f2e1e0d653db8b57e7ebc44c249a7"}
Jul 26 18:39:41.595755 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:41.595701    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9" podStartSLOduration=1.938878811 podStartE2EDuration="4.59566256s" totalImagesPullingTime="2.656783749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:39:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:39:41.594001909 +0000 UTC m=+1667.301408969" watchObservedRunningTime="2026-07-26 18:39:41.59566256 +0000 UTC m=+1667.303069618"
Jul 26 18:39:42.181990 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:42.181952    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:39:42.186357 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:42.186329    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:39:42.577569 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:42.577478    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:39:42.578538 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:42.578514    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:39:43.054682 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:43.054636    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:39:43.145484 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:43.145448    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-94b43cc4-57469d867c-7htvf"]
Jul 26 18:39:43.147862 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:43.147835    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-94b43cc4-57469d867c-7htvf"
Jul 26 18:39:43.150442 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:43.150422    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-94b43cc4-serving-cert\""
Jul 26 18:39:43.150556 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:43.150449    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 18:39:43.151429 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:43.151414    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 26 18:39:43.151522 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:43.151417    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9n9b7\""
Jul 26 18:39:43.162042 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:43.162013    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-94b43cc4-57469d867c-7htvf"]
Jul 26 18:39:43.186722 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:43.186692    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:39:43.285905 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:43.285847    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68b11ab7-9f16-49b8-948e-c28a7c3942ed-openshift-service-ca\") pod \"maas-api-e2e-migrate-94b43cc4-57469d867c-7htvf\" (UID: \"68b11ab7-9f16-49b8-948e-c28a7c3942ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-94b43cc4-57469d867c-7htvf"
Jul 26 18:39:43.286093 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:43.285914    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68b11ab7-9f16-49b8-948e-c28a7c3942ed-maas-api-tls\") pod \"maas-api-e2e-migrate-94b43cc4-57469d867c-7htvf\" (UID: \"68b11ab7-9f16-49b8-948e-c28a7c3942ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-94b43cc4-57469d867c-7htvf"
Jul 26 18:39:43.286093 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:43.285943    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h96h4\" (UniqueName: \"kubernetes.io/projected/68b11ab7-9f16-49b8-948e-c28a7c3942ed-kube-api-access-h96h4\") pod \"maas-api-e2e-migrate-94b43cc4-57469d867c-7htvf\" (UID: \"68b11ab7-9f16-49b8-948e-c28a7c3942ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-94b43cc4-57469d867c-7htvf"
Jul 26 18:39:43.386588 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:43.386497    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68b11ab7-9f16-49b8-948e-c28a7c3942ed-openshift-service-ca\") pod \"maas-api-e2e-migrate-94b43cc4-57469d867c-7htvf\" (UID: \"68b11ab7-9f16-49b8-948e-c28a7c3942ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-94b43cc4-57469d867c-7htvf"
Jul 26 18:39:43.386588 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:43.386545    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68b11ab7-9f16-49b8-948e-c28a7c3942ed-maas-api-tls\") pod \"maas-api-e2e-migrate-94b43cc4-57469d867c-7htvf\" (UID: \"68b11ab7-9f16-49b8-948e-c28a7c3942ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-94b43cc4-57469d867c-7htvf"
Jul 26 18:39:43.386588 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:43.386572    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h96h4\" (UniqueName: \"kubernetes.io/projected/68b11ab7-9f16-49b8-948e-c28a7c3942ed-kube-api-access-h96h4\") pod \"maas-api-e2e-migrate-94b43cc4-57469d867c-7htvf\" (UID: \"68b11ab7-9f16-49b8-948e-c28a7c3942ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-94b43cc4-57469d867c-7htvf"
Jul 26 18:39:43.387112 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:43.387087    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:39:43.387179 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:43.387169    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68b11ab7-9f16-49b8-948e-c28a7c3942ed-openshift-service-ca\") pod \"maas-api-e2e-migrate-94b43cc4-57469d867c-7htvf\" (UID: \"68b11ab7-9f16-49b8-948e-c28a7c3942ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-94b43cc4-57469d867c-7htvf"
Jul 26 18:39:43.389022 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:43.389005    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68b11ab7-9f16-49b8-948e-c28a7c3942ed-maas-api-tls\") pod \"maas-api-e2e-migrate-94b43cc4-57469d867c-7htvf\" (UID: \"68b11ab7-9f16-49b8-948e-c28a7c3942ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-94b43cc4-57469d867c-7htvf"
Jul 26 18:39:43.399005 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:43.398979    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h96h4\" (UniqueName: \"kubernetes.io/projected/68b11ab7-9f16-49b8-948e-c28a7c3942ed-kube-api-access-h96h4\") pod \"maas-api-e2e-migrate-94b43cc4-57469d867c-7htvf\" (UID: \"68b11ab7-9f16-49b8-948e-c28a7c3942ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-94b43cc4-57469d867c-7htvf"
Jul 26 18:39:43.457452 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:43.457420    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-94b43cc4-57469d867c-7htvf"
Jul 26 18:39:43.617242 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:43.617214    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-94b43cc4-57469d867c-7htvf"]
Jul 26 18:39:43.617358 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:39:43.617315    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68b11ab7_9f16_49b8_948e_c28a7c3942ed.slice/crio-1db1878a9d43982fc0f2eafbb8b5bffedc221e6719e17386bab985e52c0eafa9 WatchSource:0}: Error finding container 1db1878a9d43982fc0f2eafbb8b5bffedc221e6719e17386bab985e52c0eafa9: Status 404 returned error can't find the container with id 1db1878a9d43982fc0f2eafbb8b5bffedc221e6719e17386bab985e52c0eafa9
Jul 26 18:39:43.814413 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:43.814378    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:39:43.909401 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:43.909364    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:39:44.587485 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:44.587442    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-94b43cc4-57469d867c-7htvf" event={"ID":"68b11ab7-9f16-49b8-948e-c28a7c3942ed","Type":"ContainerStarted","Data":"7048e350523084b78b376dabbf3960408a75643ec9ed8f9165ec7e42b754a436"}
Jul 26 18:39:44.587948 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:44.587511    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-94b43cc4-57469d867c-7htvf" event={"ID":"68b11ab7-9f16-49b8-948e-c28a7c3942ed","Type":"ContainerStarted","Data":"1db1878a9d43982fc0f2eafbb8b5bffedc221e6719e17386bab985e52c0eafa9"}
Jul 26 18:39:44.587948 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:44.587627    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-94b43cc4-57469d867c-7htvf"
Jul 26 18:39:44.606698 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:44.606651    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-94b43cc4-57469d867c-7htvf" podStartSLOduration=1.214896271 podStartE2EDuration="1.606634801s" totalImagesPullingTime="391.73853ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:39:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:39:44.605494757 +0000 UTC m=+1670.312901816" watchObservedRunningTime="2026-07-26 18:39:44.606634801 +0000 UTC m=+1670.314041853"
Jul 26 18:39:47.305649 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:47.305615    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:39:50.597150 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:50.597124    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-94b43cc4-57469d867c-7htvf"
Jul 26 18:39:55.638366 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:55.638255    2576 generic.go:356] "Generic (PLEG): container finished" podID="165c983b-7a01-4b47-aae0-7b364c7e6ec0" containerID="7039ec62ad558d05e4dc267fe4a90aafe99f2e1e0d653db8b57e7ebc44c249a7" exitCode=0
Jul 26 18:39:55.638366 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:55.638325    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9" event={"ID":"165c983b-7a01-4b47-aae0-7b364c7e6ec0","Type":"ContainerDied","Data":"7039ec62ad558d05e4dc267fe4a90aafe99f2e1e0d653db8b57e7ebc44c249a7"}
Jul 26 18:39:55.638921 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:55.638790    2576 scope.go:117] "RemoveContainer" containerID="7039ec62ad558d05e4dc267fe4a90aafe99f2e1e0d653db8b57e7ebc44c249a7"
Jul 26 18:39:55.639925 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:55.639891    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:39:55.640026 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:55.639975    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:39:55.640073 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:55.640023    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:39:56.646313 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:56.646266    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9" event={"ID":"165c983b-7a01-4b47-aae0-7b364c7e6ec0","Type":"ContainerStarted","Data":"e134a68110e154fe64c86ac7a6b18891e9842bf15afe1f51fc944eb49fa68587"}
Jul 26 18:39:57.181526 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:57.181483    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:39:57.182571 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:57.182542    2576 patch_prober.go:28] interesting pod/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Jul 26 18:39:57.182687 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:57.182596    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9" podUID="165c983b-7a01-4b47-aae0-7b364c7e6ec0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Jul 26 18:39:57.577629 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:57.577537    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:39:58.181265 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:58.181230    2576 patch_prober.go:28] interesting pod/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Jul 26 18:39:58.181658 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:58.181290    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9" podUID="165c983b-7a01-4b47-aae0-7b364c7e6ec0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Jul 26 18:39:59.181666 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:59.181632    2576 patch_prober.go:28] interesting pod/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Jul 26 18:39:59.182067 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:39:59.181700    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9" podUID="165c983b-7a01-4b47-aae0-7b364c7e6ec0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Jul 26 18:40:00.182207 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:00.182169    2576 patch_prober.go:28] interesting pod/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Jul 26 18:40:00.182589 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:00.182262    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9" podUID="165c983b-7a01-4b47-aae0-7b364c7e6ec0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Jul 26 18:40:01.185158 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:01.185127    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:40:01.186190 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:01.186170    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:40:04.986263 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:04.986222    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:40:18.597036 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:18.596831    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-94b43cc4-57469d867c-7htvf"]
Jul 26 18:40:18.597643 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:18.597607    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-94b43cc4-57469d867c-7htvf" podUID="68b11ab7-9f16-49b8-948e-c28a7c3942ed" containerName="maas-api" containerID="cri-o://7048e350523084b78b376dabbf3960408a75643ec9ed8f9165ec7e42b754a436" gracePeriod=30
Jul 26 18:40:18.629170 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:18.629147    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:40:18.728984 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:18.728953    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:40:18.746846 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:18.746812    2576 generic.go:356] "Generic (PLEG): container finished" podID="68b11ab7-9f16-49b8-948e-c28a7c3942ed" containerID="7048e350523084b78b376dabbf3960408a75643ec9ed8f9165ec7e42b754a436" exitCode=0
Jul 26 18:40:18.747006 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:18.746884    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-94b43cc4-57469d867c-7htvf" event={"ID":"68b11ab7-9f16-49b8-948e-c28a7c3942ed","Type":"ContainerDied","Data":"7048e350523084b78b376dabbf3960408a75643ec9ed8f9165ec7e42b754a436"}
Jul 26 18:40:18.828892 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:18.828854    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:40:18.861164 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:18.861095    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-94b43cc4-57469d867c-7htvf"
Jul 26 18:40:18.882563 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:18.882531    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h96h4\" (UniqueName: \"kubernetes.io/projected/68b11ab7-9f16-49b8-948e-c28a7c3942ed-kube-api-access-h96h4\") pod \"68b11ab7-9f16-49b8-948e-c28a7c3942ed\" (UID: \"68b11ab7-9f16-49b8-948e-c28a7c3942ed\") "
Jul 26 18:40:18.882756 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:18.882581    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68b11ab7-9f16-49b8-948e-c28a7c3942ed-openshift-service-ca\") pod \"68b11ab7-9f16-49b8-948e-c28a7c3942ed\" (UID: \"68b11ab7-9f16-49b8-948e-c28a7c3942ed\") "
Jul 26 18:40:18.882756 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:18.882650    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68b11ab7-9f16-49b8-948e-c28a7c3942ed-maas-api-tls\") pod \"68b11ab7-9f16-49b8-948e-c28a7c3942ed\" (UID: \"68b11ab7-9f16-49b8-948e-c28a7c3942ed\") "
Jul 26 18:40:18.883028 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:18.883001    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68b11ab7-9f16-49b8-948e-c28a7c3942ed-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "68b11ab7-9f16-49b8-948e-c28a7c3942ed" (UID: "68b11ab7-9f16-49b8-948e-c28a7c3942ed"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:40:18.884928 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:18.884895    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68b11ab7-9f16-49b8-948e-c28a7c3942ed-kube-api-access-h96h4" (OuterVolumeSpecName: "kube-api-access-h96h4") pod "68b11ab7-9f16-49b8-948e-c28a7c3942ed" (UID: "68b11ab7-9f16-49b8-948e-c28a7c3942ed"). InnerVolumeSpecName "kube-api-access-h96h4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:40:18.885101 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:18.885082    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68b11ab7-9f16-49b8-948e-c28a7c3942ed-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "68b11ab7-9f16-49b8-948e-c28a7c3942ed" (UID: "68b11ab7-9f16-49b8-948e-c28a7c3942ed"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:40:18.983550 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:18.983515    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h96h4\" (UniqueName: \"kubernetes.io/projected/68b11ab7-9f16-49b8-948e-c28a7c3942ed-kube-api-access-h96h4\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:40:18.983550 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:18.983545    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68b11ab7-9f16-49b8-948e-c28a7c3942ed-openshift-service-ca\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:40:18.983550 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:18.983555    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68b11ab7-9f16-49b8-948e-c28a7c3942ed-maas-api-tls\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:40:19.752762 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:19.752727    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-94b43cc4-57469d867c-7htvf"
Jul 26 18:40:19.753169 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:19.752776    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-94b43cc4-57469d867c-7htvf" event={"ID":"68b11ab7-9f16-49b8-948e-c28a7c3942ed","Type":"ContainerDied","Data":"1db1878a9d43982fc0f2eafbb8b5bffedc221e6719e17386bab985e52c0eafa9"}
Jul 26 18:40:19.753169 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:19.752837    2576 scope.go:117] "RemoveContainer" containerID="7048e350523084b78b376dabbf3960408a75643ec9ed8f9165ec7e42b754a436"
Jul 26 18:40:19.775028 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:19.775003    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-94b43cc4-57469d867c-7htvf"]
Jul 26 18:40:19.778981 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:19.778959    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-94b43cc4-57469d867c-7htvf"]
Jul 26 18:40:20.848847 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:20.848815    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68b11ab7-9f16-49b8-948e-c28a7c3942ed" path="/var/lib/kubelet/pods/68b11ab7-9f16-49b8-948e-c28a7c3942ed/volumes"
Jul 26 18:40:21.534797 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:21.534759    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:40:23.137532 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:23.137494    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:40:23.268257 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:23.268222    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"]
Jul 26 18:40:23.268473 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:23.268449    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9" podUID="165c983b-7a01-4b47-aae0-7b364c7e6ec0" containerName="istio-proxy" containerID="cri-o://e134a68110e154fe64c86ac7a6b18891e9842bf15afe1f51fc944eb49fa68587" gracePeriod=30
Jul 26 18:40:23.297735 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:23.297709    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:40:23.522577 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:23.522547    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:40:23.758896 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:23.758866    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:40:24.049855 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.049821    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"]
Jul 26 18:40:24.050292 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.050277    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68b11ab7-9f16-49b8-948e-c28a7c3942ed" containerName="maas-api"
Jul 26 18:40:24.050340 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.050296    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="68b11ab7-9f16-49b8-948e-c28a7c3942ed" containerName="maas-api"
Jul 26 18:40:24.050396 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.050385    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="68b11ab7-9f16-49b8-948e-c28a7c3942ed" containerName="maas-api"
Jul 26 18:40:24.054930 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.054905    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:40:24.057831 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.057807    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-48526096-openshift-default-dockercfg-mj8qw\""
Jul 26 18:40:24.069096 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.069072    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"]
Jul 26 18:40:24.080029 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.080006    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:40:24.122817 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.122786    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cd12450-7306-4378-a3ec-f1de4735672b-istio-envoy\") pod \"e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk\" (UID: \"1cd12450-7306-4378-a3ec-f1de4735672b\") " pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:40:24.122962 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.122817    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cd12450-7306-4378-a3ec-f1de4735672b-istio-podinfo\") pod \"e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk\" (UID: \"1cd12450-7306-4378-a3ec-f1de4735672b\") " pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:40:24.122962 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.122844    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cd12450-7306-4378-a3ec-f1de4735672b-credential-socket\") pod \"e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk\" (UID: \"1cd12450-7306-4378-a3ec-f1de4735672b\") " pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:40:24.122962 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.122941    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cd12450-7306-4378-a3ec-f1de4735672b-istio-data\") pod \"e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk\" (UID: \"1cd12450-7306-4378-a3ec-f1de4735672b\") " pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:40:24.123131 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.122997    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cd12450-7306-4378-a3ec-f1de4735672b-workload-socket\") pod \"e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk\" (UID: \"1cd12450-7306-4378-a3ec-f1de4735672b\") " pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:40:24.123131 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.123034    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cd12450-7306-4378-a3ec-f1de4735672b-istiod-ca-cert\") pod \"e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk\" (UID: \"1cd12450-7306-4378-a3ec-f1de4735672b\") " pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:40:24.123131 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.123066    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cd12450-7306-4378-a3ec-f1de4735672b-workload-certs\") pod \"e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk\" (UID: \"1cd12450-7306-4378-a3ec-f1de4735672b\") " pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:40:24.123131 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.123092    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cd12450-7306-4378-a3ec-f1de4735672b-istio-token\") pod \"e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk\" (UID: \"1cd12450-7306-4378-a3ec-f1de4735672b\") " pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:40:24.123131 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.123117    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kv29\" (UniqueName: \"kubernetes.io/projected/1cd12450-7306-4378-a3ec-f1de4735672b-kube-api-access-7kv29\") pod \"e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk\" (UID: \"1cd12450-7306-4378-a3ec-f1de4735672b\") " pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:40:24.224012 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.223981    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cd12450-7306-4378-a3ec-f1de4735672b-istio-data\") pod \"e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk\" (UID: \"1cd12450-7306-4378-a3ec-f1de4735672b\") " pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:40:24.224487 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.224029    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cd12450-7306-4378-a3ec-f1de4735672b-workload-socket\") pod \"e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk\" (UID: \"1cd12450-7306-4378-a3ec-f1de4735672b\") " pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:40:24.224487 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.224050    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cd12450-7306-4378-a3ec-f1de4735672b-istiod-ca-cert\") pod \"e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk\" (UID: \"1cd12450-7306-4378-a3ec-f1de4735672b\") " pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:40:24.224487 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.224218    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cd12450-7306-4378-a3ec-f1de4735672b-workload-certs\") pod \"e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk\" (UID: \"1cd12450-7306-4378-a3ec-f1de4735672b\") " pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:40:24.224487 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.224251    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cd12450-7306-4378-a3ec-f1de4735672b-istio-token\") pod \"e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk\" (UID: \"1cd12450-7306-4378-a3ec-f1de4735672b\") " pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:40:24.224487 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.224276    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kv29\" (UniqueName: \"kubernetes.io/projected/1cd12450-7306-4378-a3ec-f1de4735672b-kube-api-access-7kv29\") pod \"e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk\" (UID: \"1cd12450-7306-4378-a3ec-f1de4735672b\") " pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:40:24.224487 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.224350    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cd12450-7306-4378-a3ec-f1de4735672b-istio-envoy\") pod \"e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk\" (UID: \"1cd12450-7306-4378-a3ec-f1de4735672b\") " pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:40:24.224487 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.224375    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cd12450-7306-4378-a3ec-f1de4735672b-istio-podinfo\") pod \"e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk\" (UID: \"1cd12450-7306-4378-a3ec-f1de4735672b\") " pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:40:24.224487 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.224417    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cd12450-7306-4378-a3ec-f1de4735672b-credential-socket\") pod \"e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk\" (UID: \"1cd12450-7306-4378-a3ec-f1de4735672b\") " pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:40:24.224487 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.224437    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cd12450-7306-4378-a3ec-f1de4735672b-istio-data\") pod \"e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk\" (UID: \"1cd12450-7306-4378-a3ec-f1de4735672b\") " pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:40:24.224487 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.224474    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cd12450-7306-4378-a3ec-f1de4735672b-workload-socket\") pod \"e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk\" (UID: \"1cd12450-7306-4378-a3ec-f1de4735672b\") " pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:40:24.224998 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.224761    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cd12450-7306-4378-a3ec-f1de4735672b-workload-certs\") pod \"e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk\" (UID: \"1cd12450-7306-4378-a3ec-f1de4735672b\") " pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:40:24.224998 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.224928    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cd12450-7306-4378-a3ec-f1de4735672b-credential-socket\") pod \"e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk\" (UID: \"1cd12450-7306-4378-a3ec-f1de4735672b\") " pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:40:24.225206 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.225182    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cd12450-7306-4378-a3ec-f1de4735672b-istiod-ca-cert\") pod \"e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk\" (UID: \"1cd12450-7306-4378-a3ec-f1de4735672b\") " pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:40:24.226827 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.226805    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cd12450-7306-4378-a3ec-f1de4735672b-istio-envoy\") pod \"e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk\" (UID: \"1cd12450-7306-4378-a3ec-f1de4735672b\") " pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:40:24.227056 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.227037    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cd12450-7306-4378-a3ec-f1de4735672b-istio-podinfo\") pod \"e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk\" (UID: \"1cd12450-7306-4378-a3ec-f1de4735672b\") " pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:40:24.232795 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.232736    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cd12450-7306-4378-a3ec-f1de4735672b-istio-token\") pod \"e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk\" (UID: \"1cd12450-7306-4378-a3ec-f1de4735672b\") " pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:40:24.232947 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.232929    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kv29\" (UniqueName: \"kubernetes.io/projected/1cd12450-7306-4378-a3ec-f1de4735672b-kube-api-access-7kv29\") pod \"e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk\" (UID: \"1cd12450-7306-4378-a3ec-f1de4735672b\") " pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:40:24.323042 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.322962    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:40:24.364830 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.364795    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:40:24.505235 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.505202    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"]
Jul 26 18:40:24.509026 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:40:24.508998    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cd12450_7306_4378_a3ec_f1de4735672b.slice/crio-1a0fcc83d37c3d1ab4b50751d98e54b3c6e98833e9c567fb3bc84660525b3afa WatchSource:0}: Error finding container 1a0fcc83d37c3d1ab4b50751d98e54b3c6e98833e9c567fb3bc84660525b3afa: Status 404 returned error can't find the container with id 1a0fcc83d37c3d1ab4b50751d98e54b3c6e98833e9c567fb3bc84660525b3afa
Jul 26 18:40:24.511080 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.511047    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:40:24.511180 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.511115    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:40:24.511180 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.511143    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:40:24.775367 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.775335    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk" event={"ID":"1cd12450-7306-4378-a3ec-f1de4735672b","Type":"ContainerStarted","Data":"5bba08019cc04b99ba513ca5aa9a36997b140c14b7d5268baf53d2a73fc2c510"}
Jul 26 18:40:24.775534 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.775384    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk" event={"ID":"1cd12450-7306-4378-a3ec-f1de4735672b","Type":"ContainerStarted","Data":"1a0fcc83d37c3d1ab4b50751d98e54b3c6e98833e9c567fb3bc84660525b3afa"}
Jul 26 18:40:24.802563 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:24.802511    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk" podStartSLOduration=0.802495546 podStartE2EDuration="802.495546ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:40:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:40:24.799501345 +0000 UTC m=+1710.506908405" watchObservedRunningTime="2026-07-26 18:40:24.802495546 +0000 UTC m=+1710.509902604"
Jul 26 18:40:25.364970 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:25.364929    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:40:25.369581 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:25.369558    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:40:25.780080 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:25.780049    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:40:25.780951 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:25.780931    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:40:26.168343 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:26.168264    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:40:26.297388 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:26.297356    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:40:26.491937 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:26.491900    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-48526096-8f8765478-hlqv5"]
Jul 26 18:40:26.494129 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:26.494110    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48526096-8f8765478-hlqv5"
Jul 26 18:40:26.501218 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:26.501200    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-48526096-serving-cert\""
Jul 26 18:40:26.501339 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:26.501229    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 26 18:40:26.501720 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:26.501698    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9n9b7\""
Jul 26 18:40:26.502460 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:26.502445    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 18:40:26.534328 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:26.534291    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-48526096-8f8765478-hlqv5"]
Jul 26 18:40:26.539555 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:26.539529    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:40:26.544840 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:26.544813    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w95hr\" (UniqueName: \"kubernetes.io/projected/21cbf7e6-e752-4c41-99f1-7d6cf6211a3a-kube-api-access-w95hr\") pod \"maas-api-e2e-ait-48526096-8f8765478-hlqv5\" (UID: \"21cbf7e6-e752-4c41-99f1-7d6cf6211a3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48526096-8f8765478-hlqv5"
Jul 26 18:40:26.544940 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:26.544880    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21cbf7e6-e752-4c41-99f1-7d6cf6211a3a-maas-api-tls\") pod \"maas-api-e2e-ait-48526096-8f8765478-hlqv5\" (UID: \"21cbf7e6-e752-4c41-99f1-7d6cf6211a3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48526096-8f8765478-hlqv5"
Jul 26 18:40:26.545001 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:26.544978    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21cbf7e6-e752-4c41-99f1-7d6cf6211a3a-openshift-service-ca\") pod \"maas-api-e2e-ait-48526096-8f8765478-hlqv5\" (UID: \"21cbf7e6-e752-4c41-99f1-7d6cf6211a3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48526096-8f8765478-hlqv5"
Jul 26 18:40:26.645629 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:26.645589    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21cbf7e6-e752-4c41-99f1-7d6cf6211a3a-maas-api-tls\") pod \"maas-api-e2e-ait-48526096-8f8765478-hlqv5\" (UID: \"21cbf7e6-e752-4c41-99f1-7d6cf6211a3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48526096-8f8765478-hlqv5"
Jul 26 18:40:26.645857 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:26.645667    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21cbf7e6-e752-4c41-99f1-7d6cf6211a3a-openshift-service-ca\") pod \"maas-api-e2e-ait-48526096-8f8765478-hlqv5\" (UID: \"21cbf7e6-e752-4c41-99f1-7d6cf6211a3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48526096-8f8765478-hlqv5"
Jul 26 18:40:26.645857 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:26.645717    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w95hr\" (UniqueName: \"kubernetes.io/projected/21cbf7e6-e752-4c41-99f1-7d6cf6211a3a-kube-api-access-w95hr\") pod \"maas-api-e2e-ait-48526096-8f8765478-hlqv5\" (UID: \"21cbf7e6-e752-4c41-99f1-7d6cf6211a3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48526096-8f8765478-hlqv5"
Jul 26 18:40:26.646663 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:26.646632    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21cbf7e6-e752-4c41-99f1-7d6cf6211a3a-openshift-service-ca\") pod \"maas-api-e2e-ait-48526096-8f8765478-hlqv5\" (UID: \"21cbf7e6-e752-4c41-99f1-7d6cf6211a3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48526096-8f8765478-hlqv5"
Jul 26 18:40:26.648119 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:26.648099    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21cbf7e6-e752-4c41-99f1-7d6cf6211a3a-maas-api-tls\") pod \"maas-api-e2e-ait-48526096-8f8765478-hlqv5\" (UID: \"21cbf7e6-e752-4c41-99f1-7d6cf6211a3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48526096-8f8765478-hlqv5"
Jul 26 18:40:26.660561 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:26.660540    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w95hr\" (UniqueName: \"kubernetes.io/projected/21cbf7e6-e752-4c41-99f1-7d6cf6211a3a-kube-api-access-w95hr\") pod \"maas-api-e2e-ait-48526096-8f8765478-hlqv5\" (UID: \"21cbf7e6-e752-4c41-99f1-7d6cf6211a3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48526096-8f8765478-hlqv5"
Jul 26 18:40:26.702622 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:26.702588    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:40:26.804730 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:26.804649    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48526096-8f8765478-hlqv5"
Jul 26 18:40:26.936449 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:26.936407    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-48526096-8f8765478-hlqv5"]
Jul 26 18:40:26.943030 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:40:26.942941    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21cbf7e6_e752_4c41_99f1_7d6cf6211a3a.slice/crio-fe1d5f447c8552f7897db7e171f6700a55d46eaeb36cdb23dc614d94a0d80246 WatchSource:0}: Error finding container fe1d5f447c8552f7897db7e171f6700a55d46eaeb36cdb23dc614d94a0d80246: Status 404 returned error can't find the container with id fe1d5f447c8552f7897db7e171f6700a55d46eaeb36cdb23dc614d94a0d80246
Jul 26 18:40:27.060736 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:27.060653    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:40:27.155933 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:27.155891    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:40:27.790144 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:27.790106    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48526096-8f8765478-hlqv5" event={"ID":"21cbf7e6-e752-4c41-99f1-7d6cf6211a3a","Type":"ContainerStarted","Data":"98bd6ce32d30673aa8a74192fcacb7775539ea150288e8f1977252b0d0eb70fb"}
Jul 26 18:40:27.790603 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:27.790165    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48526096-8f8765478-hlqv5" event={"ID":"21cbf7e6-e752-4c41-99f1-7d6cf6211a3a","Type":"ContainerStarted","Data":"fe1d5f447c8552f7897db7e171f6700a55d46eaeb36cdb23dc614d94a0d80246"}
Jul 26 18:40:27.790603 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:27.790209    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48526096-8f8765478-hlqv5"
Jul 26 18:40:27.810013 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:27.809962    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48526096-8f8765478-hlqv5" podStartSLOduration=1.44205535 podStartE2EDuration="1.809945611s" totalImagesPullingTime="367.890261ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:40:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:40:27.808144267 +0000 UTC m=+1713.515551326" watchObservedRunningTime="2026-07-26 18:40:27.809945611 +0000 UTC m=+1713.517352669"
Jul 26 18:40:28.514564 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.514535    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:40:28.564059 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.563975    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/165c983b-7a01-4b47-aae0-7b364c7e6ec0-istiod-ca-cert\") pod \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\" (UID: \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\") "
Jul 26 18:40:28.564059 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.564014    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/165c983b-7a01-4b47-aae0-7b364c7e6ec0-workload-socket\") pod \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\" (UID: \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\") "
Jul 26 18:40:28.564059 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.564039    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/165c983b-7a01-4b47-aae0-7b364c7e6ec0-credential-socket\") pod \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\" (UID: \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\") "
Jul 26 18:40:28.564059 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.564057    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/165c983b-7a01-4b47-aae0-7b364c7e6ec0-istio-data\") pod \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\" (UID: \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\") "
Jul 26 18:40:28.564365 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.564079    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/165c983b-7a01-4b47-aae0-7b364c7e6ec0-istio-envoy\") pod \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\" (UID: \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\") "
Jul 26 18:40:28.564365 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.564128    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/165c983b-7a01-4b47-aae0-7b364c7e6ec0-istio-token\") pod \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\" (UID: \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\") "
Jul 26 18:40:28.564365 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.564219    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/165c983b-7a01-4b47-aae0-7b364c7e6ec0-workload-certs\") pod \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\" (UID: \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\") "
Jul 26 18:40:28.564365 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.564243    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/165c983b-7a01-4b47-aae0-7b364c7e6ec0-istio-podinfo\") pod \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\" (UID: \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\") "
Jul 26 18:40:28.564365 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.564267    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6lvzq\" (UniqueName: \"kubernetes.io/projected/165c983b-7a01-4b47-aae0-7b364c7e6ec0-kube-api-access-6lvzq\") pod \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\" (UID: \"165c983b-7a01-4b47-aae0-7b364c7e6ec0\") "
Jul 26 18:40:28.564365 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.564319    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/165c983b-7a01-4b47-aae0-7b364c7e6ec0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "165c983b-7a01-4b47-aae0-7b364c7e6ec0" (UID: "165c983b-7a01-4b47-aae0-7b364c7e6ec0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:40:28.564643 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.564400    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/165c983b-7a01-4b47-aae0-7b364c7e6ec0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "165c983b-7a01-4b47-aae0-7b364c7e6ec0" (UID: "165c983b-7a01-4b47-aae0-7b364c7e6ec0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:40:28.564643 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.564449    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/165c983b-7a01-4b47-aae0-7b364c7e6ec0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "165c983b-7a01-4b47-aae0-7b364c7e6ec0" (UID: "165c983b-7a01-4b47-aae0-7b364c7e6ec0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:40:28.564643 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.564614    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/165c983b-7a01-4b47-aae0-7b364c7e6ec0-istiod-ca-cert\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:40:28.564643 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.564635    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/165c983b-7a01-4b47-aae0-7b364c7e6ec0-workload-socket\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:40:28.564833 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.564640    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/165c983b-7a01-4b47-aae0-7b364c7e6ec0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "165c983b-7a01-4b47-aae0-7b364c7e6ec0" (UID: "165c983b-7a01-4b47-aae0-7b364c7e6ec0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:40:28.564833 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.564649    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/165c983b-7a01-4b47-aae0-7b364c7e6ec0-credential-socket\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:40:28.565132 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.565093    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/165c983b-7a01-4b47-aae0-7b364c7e6ec0-istio-data" (OuterVolumeSpecName: "istio-data") pod "165c983b-7a01-4b47-aae0-7b364c7e6ec0" (UID: "165c983b-7a01-4b47-aae0-7b364c7e6ec0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:40:28.566465 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.566439    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/165c983b-7a01-4b47-aae0-7b364c7e6ec0-kube-api-access-6lvzq" (OuterVolumeSpecName: "kube-api-access-6lvzq") pod "165c983b-7a01-4b47-aae0-7b364c7e6ec0" (UID: "165c983b-7a01-4b47-aae0-7b364c7e6ec0"). InnerVolumeSpecName "kube-api-access-6lvzq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:40:28.566564 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.566466    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/165c983b-7a01-4b47-aae0-7b364c7e6ec0-istio-token" (OuterVolumeSpecName: "istio-token") pod "165c983b-7a01-4b47-aae0-7b364c7e6ec0" (UID: "165c983b-7a01-4b47-aae0-7b364c7e6ec0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:40:28.566764 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.566736    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/165c983b-7a01-4b47-aae0-7b364c7e6ec0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "165c983b-7a01-4b47-aae0-7b364c7e6ec0" (UID: "165c983b-7a01-4b47-aae0-7b364c7e6ec0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:40:28.566824 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.566781    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/165c983b-7a01-4b47-aae0-7b364c7e6ec0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "165c983b-7a01-4b47-aae0-7b364c7e6ec0" (UID: "165c983b-7a01-4b47-aae0-7b364c7e6ec0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 26 18:40:28.665178 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.665141    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/165c983b-7a01-4b47-aae0-7b364c7e6ec0-istio-envoy\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:40:28.665178 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.665172    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/165c983b-7a01-4b47-aae0-7b364c7e6ec0-istio-token\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:40:28.665178 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.665184    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/165c983b-7a01-4b47-aae0-7b364c7e6ec0-workload-certs\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:40:28.665404 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.665192    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/165c983b-7a01-4b47-aae0-7b364c7e6ec0-istio-podinfo\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:40:28.665404 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.665200    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6lvzq\" (UniqueName: \"kubernetes.io/projected/165c983b-7a01-4b47-aae0-7b364c7e6ec0-kube-api-access-6lvzq\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:40:28.665404 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.665209    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/165c983b-7a01-4b47-aae0-7b364c7e6ec0-istio-data\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:40:28.795815 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.795777    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9" event={"ID":"165c983b-7a01-4b47-aae0-7b364c7e6ec0","Type":"ContainerDied","Data":"e134a68110e154fe64c86ac7a6b18891e9842bf15afe1f51fc944eb49fa68587"}
Jul 26 18:40:28.795815 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.795809    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"
Jul 26 18:40:28.796356 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.795839    2576 scope.go:117] "RemoveContainer" containerID="e134a68110e154fe64c86ac7a6b18891e9842bf15afe1f51fc944eb49fa68587"
Jul 26 18:40:28.796356 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.795739    2576 generic.go:356] "Generic (PLEG): container finished" podID="165c983b-7a01-4b47-aae0-7b364c7e6ec0" containerID="e134a68110e154fe64c86ac7a6b18891e9842bf15afe1f51fc944eb49fa68587" exitCode=0
Jul 26 18:40:28.796356 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.795962    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9" event={"ID":"165c983b-7a01-4b47-aae0-7b364c7e6ec0","Type":"ContainerDied","Data":"d7b31ac0f492c3d27a0ffa85db81912dc343840492dd0c786b07923918f4d779"}
Jul 26 18:40:28.804697 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.804681    2576 scope.go:117] "RemoveContainer" containerID="7039ec62ad558d05e4dc267fe4a90aafe99f2e1e0d653db8b57e7ebc44c249a7"
Jul 26 18:40:28.812619 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.812601    2576 scope.go:117] "RemoveContainer" containerID="e134a68110e154fe64c86ac7a6b18891e9842bf15afe1f51fc944eb49fa68587"
Jul 26 18:40:28.812898 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:40:28.812878    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e134a68110e154fe64c86ac7a6b18891e9842bf15afe1f51fc944eb49fa68587\": container with ID starting with e134a68110e154fe64c86ac7a6b18891e9842bf15afe1f51fc944eb49fa68587 not found: ID does not exist" containerID="e134a68110e154fe64c86ac7a6b18891e9842bf15afe1f51fc944eb49fa68587"
Jul 26 18:40:28.812986 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.812909    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e134a68110e154fe64c86ac7a6b18891e9842bf15afe1f51fc944eb49fa68587"} err="failed to get container status \"e134a68110e154fe64c86ac7a6b18891e9842bf15afe1f51fc944eb49fa68587\": rpc error: code = NotFound desc = could not find container \"e134a68110e154fe64c86ac7a6b18891e9842bf15afe1f51fc944eb49fa68587\": container with ID starting with e134a68110e154fe64c86ac7a6b18891e9842bf15afe1f51fc944eb49fa68587 not found: ID does not exist"
Jul 26 18:40:28.812986 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.812931    2576 scope.go:117] "RemoveContainer" containerID="7039ec62ad558d05e4dc267fe4a90aafe99f2e1e0d653db8b57e7ebc44c249a7"
Jul 26 18:40:28.813195 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:40:28.813177    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7039ec62ad558d05e4dc267fe4a90aafe99f2e1e0d653db8b57e7ebc44c249a7\": container with ID starting with 7039ec62ad558d05e4dc267fe4a90aafe99f2e1e0d653db8b57e7ebc44c249a7 not found: ID does not exist" containerID="7039ec62ad558d05e4dc267fe4a90aafe99f2e1e0d653db8b57e7ebc44c249a7"
Jul 26 18:40:28.813235 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.813202    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7039ec62ad558d05e4dc267fe4a90aafe99f2e1e0d653db8b57e7ebc44c249a7"} err="failed to get container status \"7039ec62ad558d05e4dc267fe4a90aafe99f2e1e0d653db8b57e7ebc44c249a7\": rpc error: code = NotFound desc = could not find container \"7039ec62ad558d05e4dc267fe4a90aafe99f2e1e0d653db8b57e7ebc44c249a7\": container with ID starting with 7039ec62ad558d05e4dc267fe4a90aafe99f2e1e0d653db8b57e7ebc44c249a7 not found: ID does not exist"
Jul 26 18:40:28.818277 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.818222    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"]
Jul 26 18:40:28.822924 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.822906    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-94b43cc4-gw-openshift-default-598d46f4cb-s5qp9"]
Jul 26 18:40:28.848812 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:28.848784    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="165c983b-7a01-4b47-aae0-7b364c7e6ec0" path="/var/lib/kubelet/pods/165c983b-7a01-4b47-aae0-7b364c7e6ec0/volumes"
Jul 26 18:40:33.404619 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:33.404586    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:40:33.802383 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:33.802356    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48526096-8f8765478-hlqv5"
Jul 26 18:40:40.972879 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:40.972827    2576 patch_prober.go:28] interesting pod/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": read tcp 10.133.0.2:36492->10.133.0.33:15021: read: connection reset by peer" start-of-body=
Jul 26 18:40:40.973288 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:40.972904    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk" podUID="1cd12450-7306-4378-a3ec-f1de4735672b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": read tcp 10.133.0.2:36492->10.133.0.33:15021: read: connection reset by peer"
Jul 26 18:40:41.854577 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:41.854547    2576 generic.go:356] "Generic (PLEG): container finished" podID="1cd12450-7306-4378-a3ec-f1de4735672b" containerID="5bba08019cc04b99ba513ca5aa9a36997b140c14b7d5268baf53d2a73fc2c510" exitCode=0
Jul 26 18:40:41.854764 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:41.854608    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk" event={"ID":"1cd12450-7306-4378-a3ec-f1de4735672b","Type":"ContainerDied","Data":"5bba08019cc04b99ba513ca5aa9a36997b140c14b7d5268baf53d2a73fc2c510"}
Jul 26 18:40:41.855045 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:41.855020    2576 scope.go:117] "RemoveContainer" containerID="5bba08019cc04b99ba513ca5aa9a36997b140c14b7d5268baf53d2a73fc2c510"
Jul 26 18:40:41.856064 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:41.856033    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:40:41.856166 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:41.856119    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:40:41.856166 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:41.856158    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:40:42.864190 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:42.864154    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk" event={"ID":"1cd12450-7306-4378-a3ec-f1de4735672b","Type":"ContainerStarted","Data":"d9816d4e461725aa3a0b44c8b805ef893c26f7fe72464ec425bc6b461018d178"}
Jul 26 18:40:43.365949 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:43.365908    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:40:43.366199 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:43.366176    2576 patch_prober.go:28] interesting pod/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Jul 26 18:40:43.366290 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:43.366239    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk" podUID="1cd12450-7306-4378-a3ec-f1de4735672b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Jul 26 18:40:44.366172 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:44.366130    2576 patch_prober.go:28] interesting pod/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Jul 26 18:40:44.366673 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:44.366191    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk" podUID="1cd12450-7306-4378-a3ec-f1de4735672b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Jul 26 18:40:45.366116 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:45.366076    2576 patch_prober.go:28] interesting pod/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Jul 26 18:40:45.366291 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:45.366138    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk" podUID="1cd12450-7306-4378-a3ec-f1de4735672b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Jul 26 18:40:46.365736 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:46.365686    2576 patch_prober.go:28] interesting pod/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Jul 26 18:40:46.365948 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:46.365772    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk" podUID="1cd12450-7306-4378-a3ec-f1de4735672b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Jul 26 18:40:47.368854 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:47.368827    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:40:47.369227 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:47.369073    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:40:47.369738 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:40:47.369724    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:41:02.442018 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:02.441981    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-48526096-8f8765478-hlqv5"]
Jul 26 18:41:02.442684 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:02.442296    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48526096-8f8765478-hlqv5" podUID="21cbf7e6-e752-4c41-99f1-7d6cf6211a3a" containerName="maas-api" containerID="cri-o://98bd6ce32d30673aa8a74192fcacb7775539ea150288e8f1977252b0d0eb70fb" gracePeriod=30
Jul 26 18:41:02.466656 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:02.466630    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:41:02.546483 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:02.546442    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:41:02.620473 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:02.620426    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:41:02.695580 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:02.695517    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48526096-8f8765478-hlqv5"
Jul 26 18:41:02.862980 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:02.862952    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w95hr\" (UniqueName: \"kubernetes.io/projected/21cbf7e6-e752-4c41-99f1-7d6cf6211a3a-kube-api-access-w95hr\") pod \"21cbf7e6-e752-4c41-99f1-7d6cf6211a3a\" (UID: \"21cbf7e6-e752-4c41-99f1-7d6cf6211a3a\") "
Jul 26 18:41:02.863147 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:02.862989    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21cbf7e6-e752-4c41-99f1-7d6cf6211a3a-maas-api-tls\") pod \"21cbf7e6-e752-4c41-99f1-7d6cf6211a3a\" (UID: \"21cbf7e6-e752-4c41-99f1-7d6cf6211a3a\") "
Jul 26 18:41:02.863147 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:02.863011    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21cbf7e6-e752-4c41-99f1-7d6cf6211a3a-openshift-service-ca\") pod \"21cbf7e6-e752-4c41-99f1-7d6cf6211a3a\" (UID: \"21cbf7e6-e752-4c41-99f1-7d6cf6211a3a\") "
Jul 26 18:41:02.863428 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:02.863404    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21cbf7e6-e752-4c41-99f1-7d6cf6211a3a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "21cbf7e6-e752-4c41-99f1-7d6cf6211a3a" (UID: "21cbf7e6-e752-4c41-99f1-7d6cf6211a3a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:41:02.865103 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:02.865078    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21cbf7e6-e752-4c41-99f1-7d6cf6211a3a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "21cbf7e6-e752-4c41-99f1-7d6cf6211a3a" (UID: "21cbf7e6-e752-4c41-99f1-7d6cf6211a3a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:41:02.865215 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:02.865145    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21cbf7e6-e752-4c41-99f1-7d6cf6211a3a-kube-api-access-w95hr" (OuterVolumeSpecName: "kube-api-access-w95hr") pod "21cbf7e6-e752-4c41-99f1-7d6cf6211a3a" (UID: "21cbf7e6-e752-4c41-99f1-7d6cf6211a3a"). InnerVolumeSpecName "kube-api-access-w95hr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:41:02.949929 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:02.949838    2576 generic.go:356] "Generic (PLEG): container finished" podID="21cbf7e6-e752-4c41-99f1-7d6cf6211a3a" containerID="98bd6ce32d30673aa8a74192fcacb7775539ea150288e8f1977252b0d0eb70fb" exitCode=0
Jul 26 18:41:02.949929 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:02.949888    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48526096-8f8765478-hlqv5" event={"ID":"21cbf7e6-e752-4c41-99f1-7d6cf6211a3a","Type":"ContainerDied","Data":"98bd6ce32d30673aa8a74192fcacb7775539ea150288e8f1977252b0d0eb70fb"}
Jul 26 18:41:02.949929 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:02.949892    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48526096-8f8765478-hlqv5"
Jul 26 18:41:02.949929 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:02.949930    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48526096-8f8765478-hlqv5" event={"ID":"21cbf7e6-e752-4c41-99f1-7d6cf6211a3a","Type":"ContainerDied","Data":"fe1d5f447c8552f7897db7e171f6700a55d46eaeb36cdb23dc614d94a0d80246"}
Jul 26 18:41:02.950266 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:02.949951    2576 scope.go:117] "RemoveContainer" containerID="98bd6ce32d30673aa8a74192fcacb7775539ea150288e8f1977252b0d0eb70fb"
Jul 26 18:41:02.958231 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:02.958200    2576 scope.go:117] "RemoveContainer" containerID="98bd6ce32d30673aa8a74192fcacb7775539ea150288e8f1977252b0d0eb70fb"
Jul 26 18:41:02.958490 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:41:02.958469    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98bd6ce32d30673aa8a74192fcacb7775539ea150288e8f1977252b0d0eb70fb\": container with ID starting with 98bd6ce32d30673aa8a74192fcacb7775539ea150288e8f1977252b0d0eb70fb not found: ID does not exist" containerID="98bd6ce32d30673aa8a74192fcacb7775539ea150288e8f1977252b0d0eb70fb"
Jul 26 18:41:02.958541 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:02.958499    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98bd6ce32d30673aa8a74192fcacb7775539ea150288e8f1977252b0d0eb70fb"} err="failed to get container status \"98bd6ce32d30673aa8a74192fcacb7775539ea150288e8f1977252b0d0eb70fb\": rpc error: code = NotFound desc = could not find container \"98bd6ce32d30673aa8a74192fcacb7775539ea150288e8f1977252b0d0eb70fb\": container with ID starting with 98bd6ce32d30673aa8a74192fcacb7775539ea150288e8f1977252b0d0eb70fb not found: ID does not exist"
Jul 26 18:41:02.963985 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:02.963967    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w95hr\" (UniqueName: \"kubernetes.io/projected/21cbf7e6-e752-4c41-99f1-7d6cf6211a3a-kube-api-access-w95hr\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:41:02.964054 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:02.963987    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21cbf7e6-e752-4c41-99f1-7d6cf6211a3a-maas-api-tls\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:41:02.964054 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:02.963996    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21cbf7e6-e752-4c41-99f1-7d6cf6211a3a-openshift-service-ca\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:41:02.971535 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:02.971512    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-48526096-8f8765478-hlqv5"]
Jul 26 18:41:02.976921 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:02.976903    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-48526096-8f8765478-hlqv5"]
Jul 26 18:41:04.854611 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:04.854574    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21cbf7e6-e752-4c41-99f1-7d6cf6211a3a" path="/var/lib/kubelet/pods/21cbf7e6-e752-4c41-99f1-7d6cf6211a3a/volumes"
Jul 26 18:41:05.389524 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:05.389489    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:41:06.973542 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:06.973512    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:41:08.143225 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:08.143190    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"]
Jul 26 18:41:08.143648 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:08.143442    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk" podUID="1cd12450-7306-4378-a3ec-f1de4735672b" containerName="istio-proxy" containerID="cri-o://d9816d4e461725aa3a0b44c8b805ef893c26f7fe72464ec425bc6b461018d178" gracePeriod=30
Jul 26 18:41:08.170912 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:08.170864    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:41:08.375890 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:08.375861    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:41:08.653826 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:08.653793    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:41:08.900618 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:08.900583    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"]
Jul 26 18:41:08.900932 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:08.900916    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="165c983b-7a01-4b47-aae0-7b364c7e6ec0" containerName="istio-proxy"
Jul 26 18:41:08.900999 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:08.900935    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="165c983b-7a01-4b47-aae0-7b364c7e6ec0" containerName="istio-proxy"
Jul 26 18:41:08.900999 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:08.900946    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="165c983b-7a01-4b47-aae0-7b364c7e6ec0" containerName="istio-proxy"
Jul 26 18:41:08.900999 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:08.900951    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="165c983b-7a01-4b47-aae0-7b364c7e6ec0" containerName="istio-proxy"
Jul 26 18:41:08.900999 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:08.900963    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21cbf7e6-e752-4c41-99f1-7d6cf6211a3a" containerName="maas-api"
Jul 26 18:41:08.900999 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:08.900969    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="21cbf7e6-e752-4c41-99f1-7d6cf6211a3a" containerName="maas-api"
Jul 26 18:41:08.901256 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:08.901017    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="165c983b-7a01-4b47-aae0-7b364c7e6ec0" containerName="istio-proxy"
Jul 26 18:41:08.901256 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:08.901029    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="21cbf7e6-e752-4c41-99f1-7d6cf6211a3a" containerName="maas-api"
Jul 26 18:41:08.905593 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:08.905517    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:08.909781 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:08.909760    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-83ff5c8f-openshift-default-dockercfg-n5q5z\""
Jul 26 18:41:08.923385 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:08.923360    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"]
Jul 26 18:41:08.935535 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:08.935507    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:41:09.013048 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:09.013008    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49479a04-e92f-4e0e-b7d5-ff31b1852394-credential-socket\") pod \"e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs\" (UID: \"49479a04-e92f-4e0e-b7d5-ff31b1852394\") " pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:09.013231 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:09.013078    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49479a04-e92f-4e0e-b7d5-ff31b1852394-istio-podinfo\") pod \"e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs\" (UID: \"49479a04-e92f-4e0e-b7d5-ff31b1852394\") " pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:09.013231 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:09.013198    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssql8\" (UniqueName: \"kubernetes.io/projected/49479a04-e92f-4e0e-b7d5-ff31b1852394-kube-api-access-ssql8\") pod \"e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs\" (UID: \"49479a04-e92f-4e0e-b7d5-ff31b1852394\") " pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:09.013347 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:09.013240    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49479a04-e92f-4e0e-b7d5-ff31b1852394-istio-token\") pod \"e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs\" (UID: \"49479a04-e92f-4e0e-b7d5-ff31b1852394\") " pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:09.013347 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:09.013278    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49479a04-e92f-4e0e-b7d5-ff31b1852394-istio-envoy\") pod \"e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs\" (UID: \"49479a04-e92f-4e0e-b7d5-ff31b1852394\") " pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:09.013347 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:09.013308    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49479a04-e92f-4e0e-b7d5-ff31b1852394-workload-socket\") pod \"e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs\" (UID: \"49479a04-e92f-4e0e-b7d5-ff31b1852394\") " pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:09.013464 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:09.013375    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49479a04-e92f-4e0e-b7d5-ff31b1852394-workload-certs\") pod \"e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs\" (UID: \"49479a04-e92f-4e0e-b7d5-ff31b1852394\") " pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:09.013464 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:09.013413    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49479a04-e92f-4e0e-b7d5-ff31b1852394-istio-data\") pod \"e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs\" (UID: \"49479a04-e92f-4e0e-b7d5-ff31b1852394\") " pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:09.013464 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:09.013447    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49479a04-e92f-4e0e-b7d5-ff31b1852394-istiod-ca-cert\") pod \"e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs\" (UID: \"49479a04-e92f-4e0e-b7d5-ff31b1852394\") " pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:09.114643 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:09.114604    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49479a04-e92f-4e0e-b7d5-ff31b1852394-istio-podinfo\") pod \"e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs\" (UID: \"49479a04-e92f-4e0e-b7d5-ff31b1852394\") " pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:09.114643 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:09.114645    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssql8\" (UniqueName: \"kubernetes.io/projected/49479a04-e92f-4e0e-b7d5-ff31b1852394-kube-api-access-ssql8\") pod \"e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs\" (UID: \"49479a04-e92f-4e0e-b7d5-ff31b1852394\") " pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:09.114939 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:09.114666    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49479a04-e92f-4e0e-b7d5-ff31b1852394-istio-token\") pod \"e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs\" (UID: \"49479a04-e92f-4e0e-b7d5-ff31b1852394\") " pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:09.114939 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:09.114692    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49479a04-e92f-4e0e-b7d5-ff31b1852394-istio-envoy\") pod \"e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs\" (UID: \"49479a04-e92f-4e0e-b7d5-ff31b1852394\") " pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:09.114939 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:09.114721    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49479a04-e92f-4e0e-b7d5-ff31b1852394-workload-socket\") pod \"e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs\" (UID: \"49479a04-e92f-4e0e-b7d5-ff31b1852394\") " pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:09.114939 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:09.114788    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49479a04-e92f-4e0e-b7d5-ff31b1852394-workload-certs\") pod \"e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs\" (UID: \"49479a04-e92f-4e0e-b7d5-ff31b1852394\") " pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:09.114939 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:09.114828    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49479a04-e92f-4e0e-b7d5-ff31b1852394-istio-data\") pod \"e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs\" (UID: \"49479a04-e92f-4e0e-b7d5-ff31b1852394\") " pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:09.114939 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:09.114863    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49479a04-e92f-4e0e-b7d5-ff31b1852394-istiod-ca-cert\") pod \"e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs\" (UID: \"49479a04-e92f-4e0e-b7d5-ff31b1852394\") " pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:09.114939 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:09.114902    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49479a04-e92f-4e0e-b7d5-ff31b1852394-credential-socket\") pod \"e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs\" (UID: \"49479a04-e92f-4e0e-b7d5-ff31b1852394\") " pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:09.115301 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:09.115221    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49479a04-e92f-4e0e-b7d5-ff31b1852394-workload-certs\") pod \"e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs\" (UID: \"49479a04-e92f-4e0e-b7d5-ff31b1852394\") " pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:09.115301 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:09.115275    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49479a04-e92f-4e0e-b7d5-ff31b1852394-istio-data\") pod \"e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs\" (UID: \"49479a04-e92f-4e0e-b7d5-ff31b1852394\") " pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:09.115509 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:09.115485    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49479a04-e92f-4e0e-b7d5-ff31b1852394-workload-socket\") pod \"e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs\" (UID: \"49479a04-e92f-4e0e-b7d5-ff31b1852394\") " pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:09.115730 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:09.115705    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49479a04-e92f-4e0e-b7d5-ff31b1852394-istiod-ca-cert\") pod \"e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs\" (UID: \"49479a04-e92f-4e0e-b7d5-ff31b1852394\") " pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:09.115812 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:09.115724    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49479a04-e92f-4e0e-b7d5-ff31b1852394-credential-socket\") pod \"e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs\" (UID: \"49479a04-e92f-4e0e-b7d5-ff31b1852394\") " pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:09.117087 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:09.117067    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49479a04-e92f-4e0e-b7d5-ff31b1852394-istio-envoy\") pod \"e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs\" (UID: \"49479a04-e92f-4e0e-b7d5-ff31b1852394\") " pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:09.117216 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:09.117200    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49479a04-e92f-4e0e-b7d5-ff31b1852394-istio-podinfo\") pod \"e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs\" (UID: \"49479a04-e92f-4e0e-b7d5-ff31b1852394\") " pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:09.122699 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:09.122679    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49479a04-e92f-4e0e-b7d5-ff31b1852394-istio-token\") pod \"e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs\" (UID: \"49479a04-e92f-4e0e-b7d5-ff31b1852394\") " pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:09.122811 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:09.122724    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssql8\" (UniqueName: \"kubernetes.io/projected/49479a04-e92f-4e0e-b7d5-ff31b1852394-kube-api-access-ssql8\") pod \"e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs\" (UID: \"49479a04-e92f-4e0e-b7d5-ff31b1852394\") " pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:09.216035 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:09.215995    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:09.357211 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:09.357126    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"]
Jul 26 18:41:09.360333 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:41:09.360302    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49479a04_e92f_4e0e_b7d5_ff31b1852394.slice/crio-6b47cf3e7b3a35ac7c190a883fe3e909c0dcf708c1463e0fa813b0ba41062d53 WatchSource:0}: Error finding container 6b47cf3e7b3a35ac7c190a883fe3e909c0dcf708c1463e0fa813b0ba41062d53: Status 404 returned error can't find the container with id 6b47cf3e7b3a35ac7c190a883fe3e909c0dcf708c1463e0fa813b0ba41062d53
Jul 26 18:41:09.362531 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:09.362498    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:41:09.362635 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:09.362574    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:41:09.362635 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:09.362617    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:41:09.594433 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:09.594334    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:41:09.982805 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:09.982765    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs" event={"ID":"49479a04-e92f-4e0e-b7d5-ff31b1852394","Type":"ContainerStarted","Data":"58b6308c05e385468c702bcc2f6e42e4669b23dc7a3f8baaff6f64f25820e6b1"}
Jul 26 18:41:09.982984 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:09.982849    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs" event={"ID":"49479a04-e92f-4e0e-b7d5-ff31b1852394","Type":"ContainerStarted","Data":"6b47cf3e7b3a35ac7c190a883fe3e909c0dcf708c1463e0fa813b0ba41062d53"}
Jul 26 18:41:10.005798 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:10.005723    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs" podStartSLOduration=2.005707023 podStartE2EDuration="2.005707023s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:41:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:41:10.004685439 +0000 UTC m=+1755.712092510" watchObservedRunningTime="2026-07-26 18:41:10.005707023 +0000 UTC m=+1755.713114082"
Jul 26 18:41:10.216373 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:10.216336    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:10.221134 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:10.221108    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:10.987791 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:10.987761    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:10.988582 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:10.988561    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:11.442892 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:11.442862    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:41:11.543083 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:11.543047    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-83ff5c8f-668ffff79d-phmxj"]
Jul 26 18:41:11.543625 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:11.543610    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="165c983b-7a01-4b47-aae0-7b364c7e6ec0" containerName="istio-proxy"
Jul 26 18:41:11.546857 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:11.546835    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-83ff5c8f-668ffff79d-phmxj"
Jul 26 18:41:11.549573 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:11.549540    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:41:11.549692 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:11.549674    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 18:41:11.549758 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:11.549723    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 26 18:41:11.550595 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:11.550577    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-83ff5c8f-serving-cert\""
Jul 26 18:41:11.550689 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:11.550601    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9n9b7\""
Jul 26 18:41:11.557508 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:11.557485    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-83ff5c8f-668ffff79d-phmxj"]
Jul 26 18:41:11.636933 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:11.636890    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0225ab1-14bd-462e-bbd9-44b74a4801ea-maas-api-tls\") pod \"maas-api-e2e-derive-83ff5c8f-668ffff79d-phmxj\" (UID: \"a0225ab1-14bd-462e-bbd9-44b74a4801ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-83ff5c8f-668ffff79d-phmxj"
Jul 26 18:41:11.636933 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:11.636939    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0225ab1-14bd-462e-bbd9-44b74a4801ea-openshift-service-ca\") pod \"maas-api-e2e-derive-83ff5c8f-668ffff79d-phmxj\" (UID: \"a0225ab1-14bd-462e-bbd9-44b74a4801ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-83ff5c8f-668ffff79d-phmxj"
Jul 26 18:41:11.637167 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:11.636973    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bz7x\" (UniqueName: \"kubernetes.io/projected/a0225ab1-14bd-462e-bbd9-44b74a4801ea-kube-api-access-5bz7x\") pod \"maas-api-e2e-derive-83ff5c8f-668ffff79d-phmxj\" (UID: \"a0225ab1-14bd-462e-bbd9-44b74a4801ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-83ff5c8f-668ffff79d-phmxj"
Jul 26 18:41:11.644587 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:11.644556    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:41:11.738267 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:11.738160    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0225ab1-14bd-462e-bbd9-44b74a4801ea-maas-api-tls\") pod \"maas-api-e2e-derive-83ff5c8f-668ffff79d-phmxj\" (UID: \"a0225ab1-14bd-462e-bbd9-44b74a4801ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-83ff5c8f-668ffff79d-phmxj"
Jul 26 18:41:11.738267 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:11.738207    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0225ab1-14bd-462e-bbd9-44b74a4801ea-openshift-service-ca\") pod \"maas-api-e2e-derive-83ff5c8f-668ffff79d-phmxj\" (UID: \"a0225ab1-14bd-462e-bbd9-44b74a4801ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-83ff5c8f-668ffff79d-phmxj"
Jul 26 18:41:11.738267 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:11.738238    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bz7x\" (UniqueName: \"kubernetes.io/projected/a0225ab1-14bd-462e-bbd9-44b74a4801ea-kube-api-access-5bz7x\") pod \"maas-api-e2e-derive-83ff5c8f-668ffff79d-phmxj\" (UID: \"a0225ab1-14bd-462e-bbd9-44b74a4801ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-83ff5c8f-668ffff79d-phmxj"
Jul 26 18:41:11.738885 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:11.738864    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0225ab1-14bd-462e-bbd9-44b74a4801ea-openshift-service-ca\") pod \"maas-api-e2e-derive-83ff5c8f-668ffff79d-phmxj\" (UID: \"a0225ab1-14bd-462e-bbd9-44b74a4801ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-83ff5c8f-668ffff79d-phmxj"
Jul 26 18:41:11.740538 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:11.740516    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0225ab1-14bd-462e-bbd9-44b74a4801ea-maas-api-tls\") pod \"maas-api-e2e-derive-83ff5c8f-668ffff79d-phmxj\" (UID: \"a0225ab1-14bd-462e-bbd9-44b74a4801ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-83ff5c8f-668ffff79d-phmxj"
Jul 26 18:41:11.745896 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:11.745866    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:41:11.747664 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:11.747640    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bz7x\" (UniqueName: \"kubernetes.io/projected/a0225ab1-14bd-462e-bbd9-44b74a4801ea-kube-api-access-5bz7x\") pod \"maas-api-e2e-derive-83ff5c8f-668ffff79d-phmxj\" (UID: \"a0225ab1-14bd-462e-bbd9-44b74a4801ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-83ff5c8f-668ffff79d-phmxj"
Jul 26 18:41:11.858557 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:11.858525    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-83ff5c8f-668ffff79d-phmxj"
Jul 26 18:41:11.988717 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:11.988631    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-83ff5c8f-668ffff79d-phmxj"]
Jul 26 18:41:11.992607 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:41:11.992570    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0225ab1_14bd_462e_bbd9_44b74a4801ea.slice/crio-a4d048816efaf962f4d2cad616bddca9335c995212b8da6d031c3114f0dff40d WatchSource:0}: Error finding container a4d048816efaf962f4d2cad616bddca9335c995212b8da6d031c3114f0dff40d: Status 404 returned error can't find the container with id a4d048816efaf962f4d2cad616bddca9335c995212b8da6d031c3114f0dff40d
Jul 26 18:41:12.264604 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:12.264532    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:41:12.372127 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:12.372091    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:41:12.999330 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:12.999293    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-83ff5c8f-668ffff79d-phmxj" event={"ID":"a0225ab1-14bd-462e-bbd9-44b74a4801ea","Type":"ContainerStarted","Data":"f920fd71130346b255a0c580aeeff4f075d97f9f1100dbe7c5984d3d5428989e"}
Jul 26 18:41:12.999730 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:12.999512    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-83ff5c8f-668ffff79d-phmxj"
Jul 26 18:41:12.999730 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:12.999556    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-83ff5c8f-668ffff79d-phmxj" event={"ID":"a0225ab1-14bd-462e-bbd9-44b74a4801ea","Type":"ContainerStarted","Data":"a4d048816efaf962f4d2cad616bddca9335c995212b8da6d031c3114f0dff40d"}
Jul 26 18:41:13.019446 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:13.019397    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-83ff5c8f-668ffff79d-phmxj" podStartSLOduration=1.559475239 podStartE2EDuration="2.01938319s" totalImagesPullingTime="459.907951ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:41:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:41:13.017393261 +0000 UTC m=+1758.724800319" watchObservedRunningTime="2026-07-26 18:41:13.01938319 +0000 UTC m=+1758.726790250"
Jul 26 18:41:13.391318 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:13.391293    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:41:13.554156 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:13.554074    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cd12450-7306-4378-a3ec-f1de4735672b-credential-socket\") pod \"1cd12450-7306-4378-a3ec-f1de4735672b\" (UID: \"1cd12450-7306-4378-a3ec-f1de4735672b\") "
Jul 26 18:41:13.554156 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:13.554111    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cd12450-7306-4378-a3ec-f1de4735672b-istiod-ca-cert\") pod \"1cd12450-7306-4378-a3ec-f1de4735672b\" (UID: \"1cd12450-7306-4378-a3ec-f1de4735672b\") "
Jul 26 18:41:13.554156 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:13.554141    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cd12450-7306-4378-a3ec-f1de4735672b-istio-data\") pod \"1cd12450-7306-4378-a3ec-f1de4735672b\" (UID: \"1cd12450-7306-4378-a3ec-f1de4735672b\") "
Jul 26 18:41:13.554429 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:13.554167    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cd12450-7306-4378-a3ec-f1de4735672b-workload-socket\") pod \"1cd12450-7306-4378-a3ec-f1de4735672b\" (UID: \"1cd12450-7306-4378-a3ec-f1de4735672b\") "
Jul 26 18:41:13.554429 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:13.554189    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cd12450-7306-4378-a3ec-f1de4735672b-istio-token\") pod \"1cd12450-7306-4378-a3ec-f1de4735672b\" (UID: \"1cd12450-7306-4378-a3ec-f1de4735672b\") "
Jul 26 18:41:13.554429 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:13.554243    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7kv29\" (UniqueName: \"kubernetes.io/projected/1cd12450-7306-4378-a3ec-f1de4735672b-kube-api-access-7kv29\") pod \"1cd12450-7306-4378-a3ec-f1de4735672b\" (UID: \"1cd12450-7306-4378-a3ec-f1de4735672b\") "
Jul 26 18:41:13.554429 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:13.554314    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cd12450-7306-4378-a3ec-f1de4735672b-istio-envoy\") pod \"1cd12450-7306-4378-a3ec-f1de4735672b\" (UID: \"1cd12450-7306-4378-a3ec-f1de4735672b\") "
Jul 26 18:41:13.554429 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:13.554357    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cd12450-7306-4378-a3ec-f1de4735672b-istio-podinfo\") pod \"1cd12450-7306-4378-a3ec-f1de4735672b\" (UID: \"1cd12450-7306-4378-a3ec-f1de4735672b\") "
Jul 26 18:41:13.554687 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:13.554462    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cd12450-7306-4378-a3ec-f1de4735672b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1cd12450-7306-4378-a3ec-f1de4735672b" (UID: "1cd12450-7306-4378-a3ec-f1de4735672b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:41:13.554687 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:13.554549    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cd12450-7306-4378-a3ec-f1de4735672b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1cd12450-7306-4378-a3ec-f1de4735672b" (UID: "1cd12450-7306-4378-a3ec-f1de4735672b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:41:13.554687 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:13.554572    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cd12450-7306-4378-a3ec-f1de4735672b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1cd12450-7306-4378-a3ec-f1de4735672b" (UID: "1cd12450-7306-4378-a3ec-f1de4735672b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:41:13.554687 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:13.554615    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cd12450-7306-4378-a3ec-f1de4735672b-workload-certs\") pod \"1cd12450-7306-4378-a3ec-f1de4735672b\" (UID: \"1cd12450-7306-4378-a3ec-f1de4735672b\") "
Jul 26 18:41:13.554964 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:13.554943    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cd12450-7306-4378-a3ec-f1de4735672b-credential-socket\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:41:13.555029 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:13.554975    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cd12450-7306-4378-a3ec-f1de4735672b-istiod-ca-cert\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:41:13.555029 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:13.554990    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cd12450-7306-4378-a3ec-f1de4735672b-workload-socket\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:41:13.555029 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:13.554944    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cd12450-7306-4378-a3ec-f1de4735672b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1cd12450-7306-4378-a3ec-f1de4735672b" (UID: "1cd12450-7306-4378-a3ec-f1de4735672b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:41:13.555180 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:13.555051    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cd12450-7306-4378-a3ec-f1de4735672b-istio-data" (OuterVolumeSpecName: "istio-data") pod "1cd12450-7306-4378-a3ec-f1de4735672b" (UID: "1cd12450-7306-4378-a3ec-f1de4735672b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:41:13.556534 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:13.556507    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cd12450-7306-4378-a3ec-f1de4735672b-kube-api-access-7kv29" (OuterVolumeSpecName: "kube-api-access-7kv29") pod "1cd12450-7306-4378-a3ec-f1de4735672b" (UID: "1cd12450-7306-4378-a3ec-f1de4735672b"). InnerVolumeSpecName "kube-api-access-7kv29". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:41:13.556642 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:13.556550    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1cd12450-7306-4378-a3ec-f1de4735672b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1cd12450-7306-4378-a3ec-f1de4735672b" (UID: "1cd12450-7306-4378-a3ec-f1de4735672b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 26 18:41:13.556642 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:13.556556    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cd12450-7306-4378-a3ec-f1de4735672b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1cd12450-7306-4378-a3ec-f1de4735672b" (UID: "1cd12450-7306-4378-a3ec-f1de4735672b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:41:13.556798 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:13.556783    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cd12450-7306-4378-a3ec-f1de4735672b-istio-token" (OuterVolumeSpecName: "istio-token") pod "1cd12450-7306-4378-a3ec-f1de4735672b" (UID: "1cd12450-7306-4378-a3ec-f1de4735672b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:41:13.656130 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:13.656091    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cd12450-7306-4378-a3ec-f1de4735672b-istio-envoy\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:41:13.656130 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:13.656123    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cd12450-7306-4378-a3ec-f1de4735672b-istio-podinfo\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:41:13.656130 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:13.656136    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cd12450-7306-4378-a3ec-f1de4735672b-workload-certs\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:41:13.656371 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:13.656146    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cd12450-7306-4378-a3ec-f1de4735672b-istio-data\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:41:13.656371 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:13.656157    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cd12450-7306-4378-a3ec-f1de4735672b-istio-token\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:41:13.656371 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:13.656168    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7kv29\" (UniqueName: \"kubernetes.io/projected/1cd12450-7306-4378-a3ec-f1de4735672b-kube-api-access-7kv29\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:41:14.005180 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:14.005141    2576 generic.go:356] "Generic (PLEG): container finished" podID="1cd12450-7306-4378-a3ec-f1de4735672b" containerID="d9816d4e461725aa3a0b44c8b805ef893c26f7fe72464ec425bc6b461018d178" exitCode=0
Jul 26 18:41:14.005635 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:14.005190    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"
Jul 26 18:41:14.005635 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:14.005231    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk" event={"ID":"1cd12450-7306-4378-a3ec-f1de4735672b","Type":"ContainerDied","Data":"d9816d4e461725aa3a0b44c8b805ef893c26f7fe72464ec425bc6b461018d178"}
Jul 26 18:41:14.005635 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:14.005292    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk" event={"ID":"1cd12450-7306-4378-a3ec-f1de4735672b","Type":"ContainerDied","Data":"1a0fcc83d37c3d1ab4b50751d98e54b3c6e98833e9c567fb3bc84660525b3afa"}
Jul 26 18:41:14.005635 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:14.005321    2576 scope.go:117] "RemoveContainer" containerID="d9816d4e461725aa3a0b44c8b805ef893c26f7fe72464ec425bc6b461018d178"
Jul 26 18:41:14.014361 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:14.014346    2576 scope.go:117] "RemoveContainer" containerID="5bba08019cc04b99ba513ca5aa9a36997b140c14b7d5268baf53d2a73fc2c510"
Jul 26 18:41:14.021741 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:14.021725    2576 scope.go:117] "RemoveContainer" containerID="d9816d4e461725aa3a0b44c8b805ef893c26f7fe72464ec425bc6b461018d178"
Jul 26 18:41:14.021999 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:41:14.021983    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9816d4e461725aa3a0b44c8b805ef893c26f7fe72464ec425bc6b461018d178\": container with ID starting with d9816d4e461725aa3a0b44c8b805ef893c26f7fe72464ec425bc6b461018d178 not found: ID does not exist" containerID="d9816d4e461725aa3a0b44c8b805ef893c26f7fe72464ec425bc6b461018d178"
Jul 26 18:41:14.022056 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:14.022005    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9816d4e461725aa3a0b44c8b805ef893c26f7fe72464ec425bc6b461018d178"} err="failed to get container status \"d9816d4e461725aa3a0b44c8b805ef893c26f7fe72464ec425bc6b461018d178\": rpc error: code = NotFound desc = could not find container \"d9816d4e461725aa3a0b44c8b805ef893c26f7fe72464ec425bc6b461018d178\": container with ID starting with d9816d4e461725aa3a0b44c8b805ef893c26f7fe72464ec425bc6b461018d178 not found: ID does not exist"
Jul 26 18:41:14.022056 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:14.022022    2576 scope.go:117] "RemoveContainer" containerID="5bba08019cc04b99ba513ca5aa9a36997b140c14b7d5268baf53d2a73fc2c510"
Jul 26 18:41:14.022240 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:41:14.022214    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bba08019cc04b99ba513ca5aa9a36997b140c14b7d5268baf53d2a73fc2c510\": container with ID starting with 5bba08019cc04b99ba513ca5aa9a36997b140c14b7d5268baf53d2a73fc2c510 not found: ID does not exist" containerID="5bba08019cc04b99ba513ca5aa9a36997b140c14b7d5268baf53d2a73fc2c510"
Jul 26 18:41:14.022408 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:14.022246    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bba08019cc04b99ba513ca5aa9a36997b140c14b7d5268baf53d2a73fc2c510"} err="failed to get container status \"5bba08019cc04b99ba513ca5aa9a36997b140c14b7d5268baf53d2a73fc2c510\": rpc error: code = NotFound desc = could not find container \"5bba08019cc04b99ba513ca5aa9a36997b140c14b7d5268baf53d2a73fc2c510\": container with ID starting with 5bba08019cc04b99ba513ca5aa9a36997b140c14b7d5268baf53d2a73fc2c510 not found: ID does not exist"
Jul 26 18:41:14.033047 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:14.033015    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"]
Jul 26 18:41:14.033220 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:14.033206    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-48526096-openshift-default-5d44b5df9b-28hbk"]
Jul 26 18:41:14.849304 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:14.849270    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cd12450-7306-4378-a3ec-f1de4735672b" path="/var/lib/kubelet/pods/1cd12450-7306-4378-a3ec-f1de4735672b/volumes"
Jul 26 18:41:18.597512 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:18.597473    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:41:19.010625 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:19.010597    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-83ff5c8f-668ffff79d-phmxj"
Jul 26 18:41:26.614380 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:26.613787    2576 patch_prober.go:28] interesting pod/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": read tcp 10.133.0.2:58272->10.133.0.35:15021: read: connection reset by peer" start-of-body=
Jul 26 18:41:26.614380 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:26.613873    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs" podUID="49479a04-e92f-4e0e-b7d5-ff31b1852394" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": read tcp 10.133.0.2:58272->10.133.0.35:15021: read: connection reset by peer"
Jul 26 18:41:27.067739 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:27.067702    2576 generic.go:356] "Generic (PLEG): container finished" podID="49479a04-e92f-4e0e-b7d5-ff31b1852394" containerID="58b6308c05e385468c702bcc2f6e42e4669b23dc7a3f8baaff6f64f25820e6b1" exitCode=0
Jul 26 18:41:27.067927 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:27.067774    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs" event={"ID":"49479a04-e92f-4e0e-b7d5-ff31b1852394","Type":"ContainerDied","Data":"58b6308c05e385468c702bcc2f6e42e4669b23dc7a3f8baaff6f64f25820e6b1"}
Jul 26 18:41:27.068182 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:27.068169    2576 scope.go:117] "RemoveContainer" containerID="58b6308c05e385468c702bcc2f6e42e4669b23dc7a3f8baaff6f64f25820e6b1"
Jul 26 18:41:27.069221 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:27.069195    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:41:27.069289 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:27.069261    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:41:27.069325 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:27.069295    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:41:28.074300 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:28.074269    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs" event={"ID":"49479a04-e92f-4e0e-b7d5-ff31b1852394","Type":"ContainerStarted","Data":"c7507f5a4e0d398aa58c7b217e9043e7ddb4efdc55dc0126d4582db64e7fb3d3"}
Jul 26 18:41:28.216141 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:28.216103    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:28.216418 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:28.216394    2576 patch_prober.go:28] interesting pod/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Jul 26 18:41:28.216510 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:28.216448    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs" podUID="49479a04-e92f-4e0e-b7d5-ff31b1852394" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Jul 26 18:41:29.217024 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:29.216989    2576 patch_prober.go:28] interesting pod/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Jul 26 18:41:29.217396 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:29.217039    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs" podUID="49479a04-e92f-4e0e-b7d5-ff31b1852394" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Jul 26 18:41:30.217203 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:30.217169    2576 patch_prober.go:28] interesting pod/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Jul 26 18:41:30.217584 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:30.217224    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs" podUID="49479a04-e92f-4e0e-b7d5-ff31b1852394" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Jul 26 18:41:31.217247 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:31.217214    2576 patch_prober.go:28] interesting pod/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Jul 26 18:41:31.217619 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:31.217270    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs" podUID="49479a04-e92f-4e0e-b7d5-ff31b1852394" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Jul 26 18:41:32.220333 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:32.220303    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:32.220695 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:32.220672    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:32.221423 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:32.221404    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:46.638561 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:46.638523    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-83ff5c8f-668ffff79d-phmxj"]
Jul 26 18:41:46.639188 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:46.638796    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-83ff5c8f-668ffff79d-phmxj" podUID="a0225ab1-14bd-462e-bbd9-44b74a4801ea" containerName="maas-api" containerID="cri-o://f920fd71130346b255a0c580aeeff4f075d97f9f1100dbe7c5984d3d5428989e" gracePeriod=30
Jul 26 18:41:46.670867 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:46.670822    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:41:46.747635 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:46.747589    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:41:46.824012 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:46.823979    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:41:46.894669 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:46.894592    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-83ff5c8f-668ffff79d-phmxj"
Jul 26 18:41:46.951037 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:46.951007    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0225ab1-14bd-462e-bbd9-44b74a4801ea-maas-api-tls\") pod \"a0225ab1-14bd-462e-bbd9-44b74a4801ea\" (UID: \"a0225ab1-14bd-462e-bbd9-44b74a4801ea\") "
Jul 26 18:41:46.951206 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:46.951094    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0225ab1-14bd-462e-bbd9-44b74a4801ea-openshift-service-ca\") pod \"a0225ab1-14bd-462e-bbd9-44b74a4801ea\" (UID: \"a0225ab1-14bd-462e-bbd9-44b74a4801ea\") "
Jul 26 18:41:46.951267 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:46.951244    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5bz7x\" (UniqueName: \"kubernetes.io/projected/a0225ab1-14bd-462e-bbd9-44b74a4801ea-kube-api-access-5bz7x\") pod \"a0225ab1-14bd-462e-bbd9-44b74a4801ea\" (UID: \"a0225ab1-14bd-462e-bbd9-44b74a4801ea\") "
Jul 26 18:41:46.951436 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:46.951412    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0225ab1-14bd-462e-bbd9-44b74a4801ea-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a0225ab1-14bd-462e-bbd9-44b74a4801ea" (UID: "a0225ab1-14bd-462e-bbd9-44b74a4801ea"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:41:46.951572 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:46.951553    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0225ab1-14bd-462e-bbd9-44b74a4801ea-openshift-service-ca\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:41:46.953108 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:46.953086    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0225ab1-14bd-462e-bbd9-44b74a4801ea-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a0225ab1-14bd-462e-bbd9-44b74a4801ea" (UID: "a0225ab1-14bd-462e-bbd9-44b74a4801ea"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:41:46.953198 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:46.953179    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0225ab1-14bd-462e-bbd9-44b74a4801ea-kube-api-access-5bz7x" (OuterVolumeSpecName: "kube-api-access-5bz7x") pod "a0225ab1-14bd-462e-bbd9-44b74a4801ea" (UID: "a0225ab1-14bd-462e-bbd9-44b74a4801ea"). InnerVolumeSpecName "kube-api-access-5bz7x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:41:47.052612 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:47.052532    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5bz7x\" (UniqueName: \"kubernetes.io/projected/a0225ab1-14bd-462e-bbd9-44b74a4801ea-kube-api-access-5bz7x\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:41:47.052612 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:47.052563    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0225ab1-14bd-462e-bbd9-44b74a4801ea-maas-api-tls\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:41:47.155418 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:47.155334    2576 generic.go:356] "Generic (PLEG): container finished" podID="a0225ab1-14bd-462e-bbd9-44b74a4801ea" containerID="f920fd71130346b255a0c580aeeff4f075d97f9f1100dbe7c5984d3d5428989e" exitCode=0
Jul 26 18:41:47.155418 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:47.155381    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-83ff5c8f-668ffff79d-phmxj"
Jul 26 18:41:47.155643 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:47.155410    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-83ff5c8f-668ffff79d-phmxj" event={"ID":"a0225ab1-14bd-462e-bbd9-44b74a4801ea","Type":"ContainerDied","Data":"f920fd71130346b255a0c580aeeff4f075d97f9f1100dbe7c5984d3d5428989e"}
Jul 26 18:41:47.155643 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:47.155466    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-83ff5c8f-668ffff79d-phmxj" event={"ID":"a0225ab1-14bd-462e-bbd9-44b74a4801ea","Type":"ContainerDied","Data":"a4d048816efaf962f4d2cad616bddca9335c995212b8da6d031c3114f0dff40d"}
Jul 26 18:41:47.155643 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:47.155488    2576 scope.go:117] "RemoveContainer" containerID="f920fd71130346b255a0c580aeeff4f075d97f9f1100dbe7c5984d3d5428989e"
Jul 26 18:41:47.164634 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:47.164618    2576 scope.go:117] "RemoveContainer" containerID="f920fd71130346b255a0c580aeeff4f075d97f9f1100dbe7c5984d3d5428989e"
Jul 26 18:41:47.164946 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:41:47.164924    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f920fd71130346b255a0c580aeeff4f075d97f9f1100dbe7c5984d3d5428989e\": container with ID starting with f920fd71130346b255a0c580aeeff4f075d97f9f1100dbe7c5984d3d5428989e not found: ID does not exist" containerID="f920fd71130346b255a0c580aeeff4f075d97f9f1100dbe7c5984d3d5428989e"
Jul 26 18:41:47.165033 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:47.164955    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f920fd71130346b255a0c580aeeff4f075d97f9f1100dbe7c5984d3d5428989e"} err="failed to get container status \"f920fd71130346b255a0c580aeeff4f075d97f9f1100dbe7c5984d3d5428989e\": rpc error: code = NotFound desc = could not find container \"f920fd71130346b255a0c580aeeff4f075d97f9f1100dbe7c5984d3d5428989e\": container with ID starting with f920fd71130346b255a0c580aeeff4f075d97f9f1100dbe7c5984d3d5428989e not found: ID does not exist"
Jul 26 18:41:47.179342 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:47.179312    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-83ff5c8f-668ffff79d-phmxj"]
Jul 26 18:41:47.181537 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:47.181512    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-83ff5c8f-668ffff79d-phmxj"]
Jul 26 18:41:48.849251 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:48.849219    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0225ab1-14bd-462e-bbd9-44b74a4801ea" path="/var/lib/kubelet/pods/a0225ab1-14bd-462e-bbd9-44b74a4801ea/volumes"
Jul 26 18:41:49.396885 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:49.396844    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:41:51.267163 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:51.267131    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:41:51.395330 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:51.395295    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"]
Jul 26 18:41:51.395521 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:51.395496    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs" podUID="49479a04-e92f-4e0e-b7d5-ff31b1852394" containerName="istio-proxy" containerID="cri-o://c7507f5a4e0d398aa58c7b217e9043e7ddb4efdc55dc0126d4582db64e7fb3d3" gracePeriod=30
Jul 26 18:41:51.431322 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:51.431286    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:41:51.674944 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:51.674906    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:41:52.131068 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:52.130979    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:41:52.225077 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:52.225048    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:41:52.324803 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:52.324769    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:41:53.806569 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:53.806525    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:41:56.643651 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:56.643625    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:56.738207 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:56.738172    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49479a04-e92f-4e0e-b7d5-ff31b1852394-workload-certs\") pod \"49479a04-e92f-4e0e-b7d5-ff31b1852394\" (UID: \"49479a04-e92f-4e0e-b7d5-ff31b1852394\") "
Jul 26 18:41:56.738407 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:56.738271    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49479a04-e92f-4e0e-b7d5-ff31b1852394-istio-data\") pod \"49479a04-e92f-4e0e-b7d5-ff31b1852394\" (UID: \"49479a04-e92f-4e0e-b7d5-ff31b1852394\") "
Jul 26 18:41:56.738407 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:56.738298    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49479a04-e92f-4e0e-b7d5-ff31b1852394-workload-socket\") pod \"49479a04-e92f-4e0e-b7d5-ff31b1852394\" (UID: \"49479a04-e92f-4e0e-b7d5-ff31b1852394\") "
Jul 26 18:41:56.738407 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:56.738362    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ssql8\" (UniqueName: \"kubernetes.io/projected/49479a04-e92f-4e0e-b7d5-ff31b1852394-kube-api-access-ssql8\") pod \"49479a04-e92f-4e0e-b7d5-ff31b1852394\" (UID: \"49479a04-e92f-4e0e-b7d5-ff31b1852394\") "
Jul 26 18:41:56.738407 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:56.738399    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49479a04-e92f-4e0e-b7d5-ff31b1852394-istio-podinfo\") pod \"49479a04-e92f-4e0e-b7d5-ff31b1852394\" (UID: \"49479a04-e92f-4e0e-b7d5-ff31b1852394\") "
Jul 26 18:41:56.738596 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:56.738431    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49479a04-e92f-4e0e-b7d5-ff31b1852394-credential-socket\") pod \"49479a04-e92f-4e0e-b7d5-ff31b1852394\" (UID: \"49479a04-e92f-4e0e-b7d5-ff31b1852394\") "
Jul 26 18:41:56.738596 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:56.738437    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49479a04-e92f-4e0e-b7d5-ff31b1852394-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "49479a04-e92f-4e0e-b7d5-ff31b1852394" (UID: "49479a04-e92f-4e0e-b7d5-ff31b1852394"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:41:56.738596 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:56.738455    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49479a04-e92f-4e0e-b7d5-ff31b1852394-istio-token\") pod \"49479a04-e92f-4e0e-b7d5-ff31b1852394\" (UID: \"49479a04-e92f-4e0e-b7d5-ff31b1852394\") "
Jul 26 18:41:56.738596 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:56.738502    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49479a04-e92f-4e0e-b7d5-ff31b1852394-istio-envoy\") pod \"49479a04-e92f-4e0e-b7d5-ff31b1852394\" (UID: \"49479a04-e92f-4e0e-b7d5-ff31b1852394\") "
Jul 26 18:41:56.738596 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:56.738534    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49479a04-e92f-4e0e-b7d5-ff31b1852394-istiod-ca-cert\") pod \"49479a04-e92f-4e0e-b7d5-ff31b1852394\" (UID: \"49479a04-e92f-4e0e-b7d5-ff31b1852394\") "
Jul 26 18:41:56.738947 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:56.738621    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49479a04-e92f-4e0e-b7d5-ff31b1852394-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "49479a04-e92f-4e0e-b7d5-ff31b1852394" (UID: "49479a04-e92f-4e0e-b7d5-ff31b1852394"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:41:56.738947 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:56.738718    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49479a04-e92f-4e0e-b7d5-ff31b1852394-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "49479a04-e92f-4e0e-b7d5-ff31b1852394" (UID: "49479a04-e92f-4e0e-b7d5-ff31b1852394"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:41:56.738947 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:56.738871    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49479a04-e92f-4e0e-b7d5-ff31b1852394-workload-certs\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:41:56.738947 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:56.738886    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49479a04-e92f-4e0e-b7d5-ff31b1852394-workload-socket\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:41:56.738947 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:56.738895    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49479a04-e92f-4e0e-b7d5-ff31b1852394-credential-socket\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:41:56.739202 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:56.738971    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49479a04-e92f-4e0e-b7d5-ff31b1852394-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "49479a04-e92f-4e0e-b7d5-ff31b1852394" (UID: "49479a04-e92f-4e0e-b7d5-ff31b1852394"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:41:56.739337 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:56.739314    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49479a04-e92f-4e0e-b7d5-ff31b1852394-istio-data" (OuterVolumeSpecName: "istio-data") pod "49479a04-e92f-4e0e-b7d5-ff31b1852394" (UID: "49479a04-e92f-4e0e-b7d5-ff31b1852394"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:41:56.740684 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:56.740650    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/49479a04-e92f-4e0e-b7d5-ff31b1852394-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "49479a04-e92f-4e0e-b7d5-ff31b1852394" (UID: "49479a04-e92f-4e0e-b7d5-ff31b1852394"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 26 18:41:56.740850 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:56.740716    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49479a04-e92f-4e0e-b7d5-ff31b1852394-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "49479a04-e92f-4e0e-b7d5-ff31b1852394" (UID: "49479a04-e92f-4e0e-b7d5-ff31b1852394"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:41:56.740850 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:56.740770    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49479a04-e92f-4e0e-b7d5-ff31b1852394-istio-token" (OuterVolumeSpecName: "istio-token") pod "49479a04-e92f-4e0e-b7d5-ff31b1852394" (UID: "49479a04-e92f-4e0e-b7d5-ff31b1852394"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:41:56.740940 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:56.740894    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49479a04-e92f-4e0e-b7d5-ff31b1852394-kube-api-access-ssql8" (OuterVolumeSpecName: "kube-api-access-ssql8") pod "49479a04-e92f-4e0e-b7d5-ff31b1852394" (UID: "49479a04-e92f-4e0e-b7d5-ff31b1852394"). InnerVolumeSpecName "kube-api-access-ssql8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:41:56.839799 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:56.839737    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ssql8\" (UniqueName: \"kubernetes.io/projected/49479a04-e92f-4e0e-b7d5-ff31b1852394-kube-api-access-ssql8\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:41:56.839799 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:56.839794    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49479a04-e92f-4e0e-b7d5-ff31b1852394-istio-podinfo\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:41:56.839799 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:56.839804    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49479a04-e92f-4e0e-b7d5-ff31b1852394-istio-token\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:41:56.840029 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:56.839812    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49479a04-e92f-4e0e-b7d5-ff31b1852394-istio-envoy\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:41:56.840029 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:56.839820    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49479a04-e92f-4e0e-b7d5-ff31b1852394-istiod-ca-cert\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:41:56.840029 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:56.839827    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49479a04-e92f-4e0e-b7d5-ff31b1852394-istio-data\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:41:57.202716 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:57.202680    2576 generic.go:356] "Generic (PLEG): container finished" podID="49479a04-e92f-4e0e-b7d5-ff31b1852394" containerID="c7507f5a4e0d398aa58c7b217e9043e7ddb4efdc55dc0126d4582db64e7fb3d3" exitCode=0
Jul 26 18:41:57.202907 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:57.202726    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"
Jul 26 18:41:57.202907 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:57.202770    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs" event={"ID":"49479a04-e92f-4e0e-b7d5-ff31b1852394","Type":"ContainerDied","Data":"c7507f5a4e0d398aa58c7b217e9043e7ddb4efdc55dc0126d4582db64e7fb3d3"}
Jul 26 18:41:57.202907 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:57.202823    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs" event={"ID":"49479a04-e92f-4e0e-b7d5-ff31b1852394","Type":"ContainerDied","Data":"6b47cf3e7b3a35ac7c190a883fe3e909c0dcf708c1463e0fa813b0ba41062d53"}
Jul 26 18:41:57.202907 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:57.202846    2576 scope.go:117] "RemoveContainer" containerID="c7507f5a4e0d398aa58c7b217e9043e7ddb4efdc55dc0126d4582db64e7fb3d3"
Jul 26 18:41:57.211049 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:57.211031    2576 scope.go:117] "RemoveContainer" containerID="58b6308c05e385468c702bcc2f6e42e4669b23dc7a3f8baaff6f64f25820e6b1"
Jul 26 18:41:57.218234 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:57.218203    2576 scope.go:117] "RemoveContainer" containerID="c7507f5a4e0d398aa58c7b217e9043e7ddb4efdc55dc0126d4582db64e7fb3d3"
Jul 26 18:41:57.218496 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:41:57.218475    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7507f5a4e0d398aa58c7b217e9043e7ddb4efdc55dc0126d4582db64e7fb3d3\": container with ID starting with c7507f5a4e0d398aa58c7b217e9043e7ddb4efdc55dc0126d4582db64e7fb3d3 not found: ID does not exist" containerID="c7507f5a4e0d398aa58c7b217e9043e7ddb4efdc55dc0126d4582db64e7fb3d3"
Jul 26 18:41:57.218573 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:57.218509    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7507f5a4e0d398aa58c7b217e9043e7ddb4efdc55dc0126d4582db64e7fb3d3"} err="failed to get container status \"c7507f5a4e0d398aa58c7b217e9043e7ddb4efdc55dc0126d4582db64e7fb3d3\": rpc error: code = NotFound desc = could not find container \"c7507f5a4e0d398aa58c7b217e9043e7ddb4efdc55dc0126d4582db64e7fb3d3\": container with ID starting with c7507f5a4e0d398aa58c7b217e9043e7ddb4efdc55dc0126d4582db64e7fb3d3 not found: ID does not exist"
Jul 26 18:41:57.218573 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:57.218535    2576 scope.go:117] "RemoveContainer" containerID="58b6308c05e385468c702bcc2f6e42e4669b23dc7a3f8baaff6f64f25820e6b1"
Jul 26 18:41:57.218779 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:41:57.218740    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58b6308c05e385468c702bcc2f6e42e4669b23dc7a3f8baaff6f64f25820e6b1\": container with ID starting with 58b6308c05e385468c702bcc2f6e42e4669b23dc7a3f8baaff6f64f25820e6b1 not found: ID does not exist" containerID="58b6308c05e385468c702bcc2f6e42e4669b23dc7a3f8baaff6f64f25820e6b1"
Jul 26 18:41:57.218835 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:57.218785    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58b6308c05e385468c702bcc2f6e42e4669b23dc7a3f8baaff6f64f25820e6b1"} err="failed to get container status \"58b6308c05e385468c702bcc2f6e42e4669b23dc7a3f8baaff6f64f25820e6b1\": rpc error: code = NotFound desc = could not find container \"58b6308c05e385468c702bcc2f6e42e4669b23dc7a3f8baaff6f64f25820e6b1\": container with ID starting with 58b6308c05e385468c702bcc2f6e42e4669b23dc7a3f8baaff6f64f25820e6b1 not found: ID does not exist"
Jul 26 18:41:57.222163 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:57.222140    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"]
Jul 26 18:41:57.228408 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:57.228383    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-83ff5c8f-openshift-default-67fd4c87d4-n69xs"]
Jul 26 18:41:58.849122 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:41:58.849086    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49479a04-e92f-4e0e-b7d5-ff31b1852394" path="/var/lib/kubelet/pods/49479a04-e92f-4e0e-b7d5-ff31b1852394/volumes"
Jul 26 18:42:00.675913 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:42:00.675877    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:42:00.785591 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:42:00.785556    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:42:00.880841 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:42:00.880802    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:42:00.995063 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:42:00.994981    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:42:01.375905 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:42:01.375820    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:42:27.336742 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:42:27.336708    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:42:34.148940 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:42:34.148902    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:42:34.354825 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:42:34.354792    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:42:50.273119 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:42:50.273079    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:42:54.933718 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:42:54.933691    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:42:54.987180 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:42:54.987147    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:42:55.855336 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:42:55.855300    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:42:55.961543 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:42:55.961513    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:42:56.056394 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:42:56.056365    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:43:06.549991 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:06.549950    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:43:07.313006 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:07.312968    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:43:13.733263 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:13.733231    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:43:13.840371 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:13.840341    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:43:13.933807 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:13.933773    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:43:14.034703 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:14.034608    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:43:14.761201 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:14.761168    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:43:14.863247 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:14.863213    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:43:14.961002 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:14.960952    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:43:16.496759 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:16.496725    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:43:17.189024 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:17.188944    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:43:22.212340 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:22.212305    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:43:22.274788 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:22.274697    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:43:22.890368 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:22.890326    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:43:28.087838 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:28.087798    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:43:29.074870 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.074838    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:43:29.175299 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.175266    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:43:29.327083 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.326997    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:43:29.606535 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.606448    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"]
Jul 26 18:43:29.606946 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.606924    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49479a04-e92f-4e0e-b7d5-ff31b1852394" containerName="istio-proxy"
Jul 26 18:43:29.607008 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.606974    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="49479a04-e92f-4e0e-b7d5-ff31b1852394" containerName="istio-proxy"
Jul 26 18:43:29.607008 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.606988    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49479a04-e92f-4e0e-b7d5-ff31b1852394" containerName="istio-proxy"
Jul 26 18:43:29.607008 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.606997    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="49479a04-e92f-4e0e-b7d5-ff31b1852394" containerName="istio-proxy"
Jul 26 18:43:29.607111 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.607008    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cd12450-7306-4378-a3ec-f1de4735672b" containerName="istio-proxy"
Jul 26 18:43:29.607111 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.607019    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cd12450-7306-4378-a3ec-f1de4735672b" containerName="istio-proxy"
Jul 26 18:43:29.607111 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.607039    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0225ab1-14bd-462e-bbd9-44b74a4801ea" containerName="maas-api"
Jul 26 18:43:29.607111 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.607046    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0225ab1-14bd-462e-bbd9-44b74a4801ea" containerName="maas-api"
Jul 26 18:43:29.607280 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.607129    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="49479a04-e92f-4e0e-b7d5-ff31b1852394" containerName="istio-proxy"
Jul 26 18:43:29.607280 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.607140    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="49479a04-e92f-4e0e-b7d5-ff31b1852394" containerName="istio-proxy"
Jul 26 18:43:29.607280 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.607149    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cd12450-7306-4378-a3ec-f1de4735672b" containerName="istio-proxy"
Jul 26 18:43:29.607280 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.607158    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cd12450-7306-4378-a3ec-f1de4735672b" containerName="istio-proxy"
Jul 26 18:43:29.607280 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.607168    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0225ab1-14bd-462e-bbd9-44b74a4801ea" containerName="maas-api"
Jul 26 18:43:29.610291 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.610269    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:43:29.612986 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.612965    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-0dc66676-openshift-default-dockercfg-w6vpc\""
Jul 26 18:43:29.633870 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.633834    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"]
Jul 26 18:43:29.635431 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.635402    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:43:29.720000 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.719958    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57b9a95d-0eff-43e6-9140-97efd8e0bff0-workload-certs\") pod \"e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b\" (UID: \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\") " pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:43:29.720000 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.720004    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57b9a95d-0eff-43e6-9140-97efd8e0bff0-istio-envoy\") pod \"e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b\" (UID: \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\") " pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:43:29.720266 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.720089    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57b9a95d-0eff-43e6-9140-97efd8e0bff0-credential-socket\") pod \"e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b\" (UID: \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\") " pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:43:29.720266 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.720120    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57b9a95d-0eff-43e6-9140-97efd8e0bff0-istio-podinfo\") pod \"e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b\" (UID: \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\") " pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:43:29.720266 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.720137    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57b9a95d-0eff-43e6-9140-97efd8e0bff0-istiod-ca-cert\") pod \"e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b\" (UID: \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\") " pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:43:29.720266 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.720179    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57b9a95d-0eff-43e6-9140-97efd8e0bff0-istio-data\") pod \"e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b\" (UID: \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\") " pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:43:29.720266 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.720202    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57b9a95d-0eff-43e6-9140-97efd8e0bff0-workload-socket\") pod \"e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b\" (UID: \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\") " pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:43:29.720266 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.720230    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9rxc\" (UniqueName: \"kubernetes.io/projected/57b9a95d-0eff-43e6-9140-97efd8e0bff0-kube-api-access-s9rxc\") pod \"e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b\" (UID: \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\") " pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:43:29.720635 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.720272    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57b9a95d-0eff-43e6-9140-97efd8e0bff0-istio-token\") pod \"e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b\" (UID: \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\") " pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:43:29.812859 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.812823    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:43:29.821016 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.820990    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57b9a95d-0eff-43e6-9140-97efd8e0bff0-istio-data\") pod \"e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b\" (UID: \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\") " pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:43:29.821113 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.821024    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57b9a95d-0eff-43e6-9140-97efd8e0bff0-workload-socket\") pod \"e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b\" (UID: \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\") " pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:43:29.821113 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.821046    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9rxc\" (UniqueName: \"kubernetes.io/projected/57b9a95d-0eff-43e6-9140-97efd8e0bff0-kube-api-access-s9rxc\") pod \"e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b\" (UID: \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\") " pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:43:29.821113 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.821082    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57b9a95d-0eff-43e6-9140-97efd8e0bff0-istio-token\") pod \"e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b\" (UID: \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\") " pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:43:29.821113 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.821110    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57b9a95d-0eff-43e6-9140-97efd8e0bff0-workload-certs\") pod \"e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b\" (UID: \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\") " pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:43:29.821314 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.821130    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57b9a95d-0eff-43e6-9140-97efd8e0bff0-istio-envoy\") pod \"e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b\" (UID: \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\") " pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:43:29.821314 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.821155    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57b9a95d-0eff-43e6-9140-97efd8e0bff0-credential-socket\") pod \"e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b\" (UID: \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\") " pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:43:29.821314 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.821178    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57b9a95d-0eff-43e6-9140-97efd8e0bff0-istio-podinfo\") pod \"e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b\" (UID: \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\") " pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:43:29.821314 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.821203    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57b9a95d-0eff-43e6-9140-97efd8e0bff0-istiod-ca-cert\") pod \"e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b\" (UID: \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\") " pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:43:29.821522 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.821450    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57b9a95d-0eff-43e6-9140-97efd8e0bff0-istio-data\") pod \"e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b\" (UID: \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\") " pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:43:29.821522 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.821470    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57b9a95d-0eff-43e6-9140-97efd8e0bff0-workload-socket\") pod \"e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b\" (UID: \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\") " pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:43:29.821670 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.821646    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57b9a95d-0eff-43e6-9140-97efd8e0bff0-workload-certs\") pod \"e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b\" (UID: \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\") " pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:43:29.821930 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.821908    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57b9a95d-0eff-43e6-9140-97efd8e0bff0-credential-socket\") pod \"e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b\" (UID: \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\") " pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:43:29.821997 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.821929    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57b9a95d-0eff-43e6-9140-97efd8e0bff0-istiod-ca-cert\") pod \"e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b\" (UID: \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\") " pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:43:29.823426 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.823396    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57b9a95d-0eff-43e6-9140-97efd8e0bff0-istio-envoy\") pod \"e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b\" (UID: \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\") " pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:43:29.823548 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.823531    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57b9a95d-0eff-43e6-9140-97efd8e0bff0-istio-podinfo\") pod \"e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b\" (UID: \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\") " pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:43:29.829163 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.829137    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57b9a95d-0eff-43e6-9140-97efd8e0bff0-istio-token\") pod \"e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b\" (UID: \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\") " pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:43:29.829562 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.829539    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9rxc\" (UniqueName: \"kubernetes.io/projected/57b9a95d-0eff-43e6-9140-97efd8e0bff0-kube-api-access-s9rxc\") pod \"e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b\" (UID: \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\") " pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:43:29.922967 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:29.922943    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:43:30.067361 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:30.067325    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"]
Jul 26 18:43:30.070890 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:43:30.070859    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57b9a95d_0eff_43e6_9140_97efd8e0bff0.slice/crio-1dfcc0c60e735dd500620d1cf60da38e9747649c0cd22b66e9923c62c9db19b7 WatchSource:0}: Error finding container 1dfcc0c60e735dd500620d1cf60da38e9747649c0cd22b66e9923c62c9db19b7: Status 404 returned error can't find the container with id 1dfcc0c60e735dd500620d1cf60da38e9747649c0cd22b66e9923c62c9db19b7
Jul 26 18:43:30.072990 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:30.072959    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:43:30.073070 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:30.073029    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:43:30.073070 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:30.073059    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:43:30.592408 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:30.592362    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b" event={"ID":"57b9a95d-0eff-43e6-9140-97efd8e0bff0","Type":"ContainerStarted","Data":"c1e861dff3c86d21f5ea3bb5b282632fbacc850f3729b4dd345ee6ea466ec800"}
Jul 26 18:43:30.592841 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:30.592421    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b" event={"ID":"57b9a95d-0eff-43e6-9140-97efd8e0bff0","Type":"ContainerStarted","Data":"1dfcc0c60e735dd500620d1cf60da38e9747649c0cd22b66e9923c62c9db19b7"}
Jul 26 18:43:30.617448 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:30.617377    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b" podStartSLOduration=1.6173545 podStartE2EDuration="1.6173545s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:43:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:43:30.61670202 +0000 UTC m=+1896.324109082" watchObservedRunningTime="2026-07-26 18:43:30.6173545 +0000 UTC m=+1896.324761560"
Jul 26 18:43:30.923734 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:30.923708    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:43:31.928590 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:31.928552    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:43:32.519955 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:32.519918    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:43:32.601685 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:32.601652    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:43:32.602773 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:32.602726    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:43:32.609693 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:32.609672    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:43:32.700907 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:32.700869    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc"]
Jul 26 18:43:32.701363 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:32.701344    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cd12450-7306-4378-a3ec-f1de4735672b" containerName="istio-proxy"
Jul 26 18:43:32.701363 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:32.701362    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cd12450-7306-4378-a3ec-f1de4735672b" containerName="istio-proxy"
Jul 26 18:43:32.704640 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:32.704619    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc"
Jul 26 18:43:32.707359 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:32.707336    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9n9b7\""
Jul 26 18:43:32.707359 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:32.707352    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-0dc66676-serving-cert\""
Jul 26 18:43:32.707569 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:32.707436    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 18:43:32.707682 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:32.707651    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 26 18:43:32.723055 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:32.722054    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc"]
Jul 26 18:43:32.734031 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:32.733996    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:43:32.846944 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:32.846858    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0369b67-d9a2-4b47-aaed-b69a9a35fc4a-maas-api-tls\") pod \"maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc\" (UID: \"b0369b67-d9a2-4b47-aaed-b69a9a35fc4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc"
Jul 26 18:43:32.846944 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:32.846915    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0369b67-d9a2-4b47-aaed-b69a9a35fc4a-openshift-service-ca\") pod \"maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc\" (UID: \"b0369b67-d9a2-4b47-aaed-b69a9a35fc4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc"
Jul 26 18:43:32.847132 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:32.846947    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpjmz\" (UniqueName: \"kubernetes.io/projected/b0369b67-d9a2-4b47-aaed-b69a9a35fc4a-kube-api-access-gpjmz\") pod \"maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc\" (UID: \"b0369b67-d9a2-4b47-aaed-b69a9a35fc4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc"
Jul 26 18:43:32.947390 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:32.947354    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0369b67-d9a2-4b47-aaed-b69a9a35fc4a-openshift-service-ca\") pod \"maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc\" (UID: \"b0369b67-d9a2-4b47-aaed-b69a9a35fc4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc"
Jul 26 18:43:32.947390 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:32.947393    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpjmz\" (UniqueName: \"kubernetes.io/projected/b0369b67-d9a2-4b47-aaed-b69a9a35fc4a-kube-api-access-gpjmz\") pod \"maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc\" (UID: \"b0369b67-d9a2-4b47-aaed-b69a9a35fc4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc"
Jul 26 18:43:32.947939 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:32.947469    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0369b67-d9a2-4b47-aaed-b69a9a35fc4a-maas-api-tls\") pod \"maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc\" (UID: \"b0369b67-d9a2-4b47-aaed-b69a9a35fc4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc"
Jul 26 18:43:32.947939 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:43:32.947659    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-0dc66676-serving-cert: secret "maas-api-e2e-mt-0dc66676-serving-cert" not found
Jul 26 18:43:32.947939 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:43:32.947731    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0369b67-d9a2-4b47-aaed-b69a9a35fc4a-maas-api-tls podName:b0369b67-d9a2-4b47-aaed-b69a9a35fc4a nodeName:}" failed. No retries permitted until 2026-07-26 18:43:33.447711373 +0000 UTC m=+1899.155118413 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b0369b67-d9a2-4b47-aaed-b69a9a35fc4a-maas-api-tls") pod "maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc" (UID: "b0369b67-d9a2-4b47-aaed-b69a9a35fc4a") : secret "maas-api-e2e-mt-0dc66676-serving-cert" not found
Jul 26 18:43:32.948142 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:32.948121    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0369b67-d9a2-4b47-aaed-b69a9a35fc4a-openshift-service-ca\") pod \"maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc\" (UID: \"b0369b67-d9a2-4b47-aaed-b69a9a35fc4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc"
Jul 26 18:43:32.956840 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:32.956780    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpjmz\" (UniqueName: \"kubernetes.io/projected/b0369b67-d9a2-4b47-aaed-b69a9a35fc4a-kube-api-access-gpjmz\") pod \"maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc\" (UID: \"b0369b67-d9a2-4b47-aaed-b69a9a35fc4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc"
Jul 26 18:43:33.006680 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:33.006639    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:43:33.333064 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:33.333028    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:43:33.432654 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:33.432623    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:43:33.451895 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:33.451853    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0369b67-d9a2-4b47-aaed-b69a9a35fc4a-maas-api-tls\") pod \"maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc\" (UID: \"b0369b67-d9a2-4b47-aaed-b69a9a35fc4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc"
Jul 26 18:43:33.454576 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:33.454550    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0369b67-d9a2-4b47-aaed-b69a9a35fc4a-maas-api-tls\") pod \"maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc\" (UID: \"b0369b67-d9a2-4b47-aaed-b69a9a35fc4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc"
Jul 26 18:43:33.615722 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:33.615628    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc"
Jul 26 18:43:33.757417 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:33.757389    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc"]
Jul 26 18:43:33.760231 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:43:33.760206    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0369b67_d9a2_4b47_aaed_b69a9a35fc4a.slice/crio-c19fad9fd0d62f1fffd008696cc629da53d0dae5d386128ac9f36c3447aaf7c9 WatchSource:0}: Error finding container c19fad9fd0d62f1fffd008696cc629da53d0dae5d386128ac9f36c3447aaf7c9: Status 404 returned error can't find the container with id c19fad9fd0d62f1fffd008696cc629da53d0dae5d386128ac9f36c3447aaf7c9
Jul 26 18:43:34.610852 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:34.610739    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc" event={"ID":"b0369b67-d9a2-4b47-aaed-b69a9a35fc4a","Type":"ContainerStarted","Data":"9f1b0f182194572d8b87b190393f7d6402134931be8923cdaf1778348a942c94"}
Jul 26 18:43:34.610852 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:34.610843    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc"
Jul 26 18:43:34.611279 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:34.610882    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc" event={"ID":"b0369b67-d9a2-4b47-aaed-b69a9a35fc4a","Type":"ContainerStarted","Data":"c19fad9fd0d62f1fffd008696cc629da53d0dae5d386128ac9f36c3447aaf7c9"}
Jul 26 18:43:34.629554 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:34.629502    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc" podStartSLOduration=2.090775667 podStartE2EDuration="2.629488018s" totalImagesPullingTime="538.712351ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:43:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:43:34.628259897 +0000 UTC m=+1900.335666956" watchObservedRunningTime="2026-07-26 18:43:34.629488018 +0000 UTC m=+1900.336895078"
Jul 26 18:43:36.980155 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:36.980112    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:43:37.605060 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:37.605008    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:43:37.711440 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:37.711401    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:43:38.033613 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:38.033584    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:43:40.620359 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:40.620326    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc"
Jul 26 18:43:40.693551 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:40.693516    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:43:43.640970 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:43.640859    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:43:43.741466 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:43.741427    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:43:48.081210 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:48.080832    2576 patch_prober.go:28] interesting pod/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": read tcp 10.133.0.2:37120->10.133.0.37:15021: read: connection reset by peer" start-of-body=
Jul 26 18:43:48.081210 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:48.080908    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b" podUID="57b9a95d-0eff-43e6-9140-97efd8e0bff0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": read tcp 10.133.0.2:37120->10.133.0.37:15021: read: connection reset by peer"
Jul 26 18:43:48.679440 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:48.679405    2576 generic.go:356] "Generic (PLEG): container finished" podID="57b9a95d-0eff-43e6-9140-97efd8e0bff0" containerID="c1e861dff3c86d21f5ea3bb5b282632fbacc850f3729b4dd345ee6ea466ec800" exitCode=0
Jul 26 18:43:48.679612 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:48.679457    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b" event={"ID":"57b9a95d-0eff-43e6-9140-97efd8e0bff0","Type":"ContainerDied","Data":"c1e861dff3c86d21f5ea3bb5b282632fbacc850f3729b4dd345ee6ea466ec800"}
Jul 26 18:43:48.679860 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:48.679841    2576 scope.go:117] "RemoveContainer" containerID="c1e861dff3c86d21f5ea3bb5b282632fbacc850f3729b4dd345ee6ea466ec800"
Jul 26 18:43:48.680853 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:48.680823    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:43:48.680911 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:48.680895    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:43:48.680955 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:48.680923    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:43:49.686917 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:49.686885    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b" event={"ID":"57b9a95d-0eff-43e6-9140-97efd8e0bff0","Type":"ContainerStarted","Data":"04c5e0f9ea5446a233526a025430a3ae3710110339f7eb2b8b842365f50b4d7a"}
Jul 26 18:43:49.923988 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:49.923948    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:43:49.924166 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:49.924132    2576 patch_prober.go:28] interesting pod/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Jul 26 18:43:49.924224 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:49.924202    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b" podUID="57b9a95d-0eff-43e6-9140-97efd8e0bff0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Jul 26 18:43:50.924192 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:50.924142    2576 patch_prober.go:28] interesting pod/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Jul 26 18:43:50.924686 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:50.924212    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b" podUID="57b9a95d-0eff-43e6-9140-97efd8e0bff0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Jul 26 18:43:51.924175 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:51.924140    2576 patch_prober.go:28] interesting pod/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Jul 26 18:43:51.924364 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:51.924196    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b" podUID="57b9a95d-0eff-43e6-9140-97efd8e0bff0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Jul 26 18:43:52.923654 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:52.923622    2576 patch_prober.go:28] interesting pod/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Jul 26 18:43:52.923846 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:52.923674    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b" podUID="57b9a95d-0eff-43e6-9140-97efd8e0bff0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Jul 26 18:43:53.926886 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:53.926851    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:43:53.927346 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:53.927152    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:43:53.927841 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:53.927822    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:43:54.593592 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:54.593555    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc"]
Jul 26 18:43:54.593861 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:54.593823    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc" podUID="b0369b67-d9a2-4b47-aaed-b69a9a35fc4a" containerName="maas-api" containerID="cri-o://9f1b0f182194572d8b87b190393f7d6402134931be8923cdaf1778348a942c94" gracePeriod=30
Jul 26 18:43:54.626375 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:54.626346    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:43:54.695910 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:54.695879    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:43:54.714651 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:54.714620    2576 generic.go:356] "Generic (PLEG): container finished" podID="b0369b67-d9a2-4b47-aaed-b69a9a35fc4a" containerID="9f1b0f182194572d8b87b190393f7d6402134931be8923cdaf1778348a942c94" exitCode=0
Jul 26 18:43:54.714814 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:54.714699    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc" event={"ID":"b0369b67-d9a2-4b47-aaed-b69a9a35fc4a","Type":"ContainerDied","Data":"9f1b0f182194572d8b87b190393f7d6402134931be8923cdaf1778348a942c94"}
Jul 26 18:43:54.762090 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:54.762048    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:43:54.853863 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:54.853800    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc"
Jul 26 18:43:54.949738 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:54.949707    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gpjmz\" (UniqueName: \"kubernetes.io/projected/b0369b67-d9a2-4b47-aaed-b69a9a35fc4a-kube-api-access-gpjmz\") pod \"b0369b67-d9a2-4b47-aaed-b69a9a35fc4a\" (UID: \"b0369b67-d9a2-4b47-aaed-b69a9a35fc4a\") "
Jul 26 18:43:54.950224 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:54.949790    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0369b67-d9a2-4b47-aaed-b69a9a35fc4a-maas-api-tls\") pod \"b0369b67-d9a2-4b47-aaed-b69a9a35fc4a\" (UID: \"b0369b67-d9a2-4b47-aaed-b69a9a35fc4a\") "
Jul 26 18:43:54.950224 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:54.949813    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0369b67-d9a2-4b47-aaed-b69a9a35fc4a-openshift-service-ca\") pod \"b0369b67-d9a2-4b47-aaed-b69a9a35fc4a\" (UID: \"b0369b67-d9a2-4b47-aaed-b69a9a35fc4a\") "
Jul 26 18:43:54.950357 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:54.950261    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0369b67-d9a2-4b47-aaed-b69a9a35fc4a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b0369b67-d9a2-4b47-aaed-b69a9a35fc4a" (UID: "b0369b67-d9a2-4b47-aaed-b69a9a35fc4a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:43:54.951828 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:54.951802    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0369b67-d9a2-4b47-aaed-b69a9a35fc4a-kube-api-access-gpjmz" (OuterVolumeSpecName: "kube-api-access-gpjmz") pod "b0369b67-d9a2-4b47-aaed-b69a9a35fc4a" (UID: "b0369b67-d9a2-4b47-aaed-b69a9a35fc4a"). InnerVolumeSpecName "kube-api-access-gpjmz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:43:54.951932 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:54.951886    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0369b67-d9a2-4b47-aaed-b69a9a35fc4a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b0369b67-d9a2-4b47-aaed-b69a9a35fc4a" (UID: "b0369b67-d9a2-4b47-aaed-b69a9a35fc4a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:43:55.050754 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:55.050720    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0369b67-d9a2-4b47-aaed-b69a9a35fc4a-maas-api-tls\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:43:55.050943 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:55.050778    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0369b67-d9a2-4b47-aaed-b69a9a35fc4a-openshift-service-ca\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:43:55.050943 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:55.050787    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gpjmz\" (UniqueName: \"kubernetes.io/projected/b0369b67-d9a2-4b47-aaed-b69a9a35fc4a-kube-api-access-gpjmz\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:43:55.720741 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:55.720698    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc" event={"ID":"b0369b67-d9a2-4b47-aaed-b69a9a35fc4a","Type":"ContainerDied","Data":"c19fad9fd0d62f1fffd008696cc629da53d0dae5d386128ac9f36c3447aaf7c9"}
Jul 26 18:43:55.720947 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:55.720786    2576 scope.go:117] "RemoveContainer" containerID="9f1b0f182194572d8b87b190393f7d6402134931be8923cdaf1778348a942c94"
Jul 26 18:43:55.720947 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:55.720712    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc"
Jul 26 18:43:55.741833 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:55.741806    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc"]
Jul 26 18:43:55.745142 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:55.745119    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0dc66676-6dbcc4dff5-48hwc"]
Jul 26 18:43:56.849578 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:56.849543    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0369b67-d9a2-4b47-aaed-b69a9a35fc4a" path="/var/lib/kubelet/pods/b0369b67-d9a2-4b47-aaed-b69a9a35fc4a/volumes"
Jul 26 18:43:57.688387 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:57.688351    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:43:59.223846 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:59.223800    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:43:59.612777 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:43:59.612666    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:44:05.327154 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:05.327117    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:44:05.427841 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:05.427805    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:44:05.560046 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:05.560008    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"]
Jul 26 18:44:05.560268 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:05.560242    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b" podUID="57b9a95d-0eff-43e6-9140-97efd8e0bff0" containerName="istio-proxy" containerID="cri-o://04c5e0f9ea5446a233526a025430a3ae3710110339f7eb2b8b842365f50b4d7a" gracePeriod=30
Jul 26 18:44:05.585367 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:05.585265    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:44:05.799868 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:05.799836    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:44:05.916301 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:05.916224    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:44:06.014959 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:06.014901    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:44:06.128330 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:06.128296    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:44:08.929609 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:08.929565    2576 patch_prober.go:28] interesting pod/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Jul 26 18:44:08.930118 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:08.929629    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b" podUID="57b9a95d-0eff-43e6-9140-97efd8e0bff0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 26 18:44:10.785233 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:10.785201    2576 generic.go:356] "Generic (PLEG): container finished" podID="57b9a95d-0eff-43e6-9140-97efd8e0bff0" containerID="04c5e0f9ea5446a233526a025430a3ae3710110339f7eb2b8b842365f50b4d7a" exitCode=0
Jul 26 18:44:10.785649 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:10.785270    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b" event={"ID":"57b9a95d-0eff-43e6-9140-97efd8e0bff0","Type":"ContainerDied","Data":"04c5e0f9ea5446a233526a025430a3ae3710110339f7eb2b8b842365f50b4d7a"}
Jul 26 18:44:10.785649 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:10.785364    2576 scope.go:117] "RemoveContainer" containerID="c1e861dff3c86d21f5ea3bb5b282632fbacc850f3729b4dd345ee6ea466ec800"
Jul 26 18:44:10.818931 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:10.818899    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:44:10.888648 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:10.888548    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57b9a95d-0eff-43e6-9140-97efd8e0bff0-istiod-ca-cert\") pod \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\" (UID: \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\") "
Jul 26 18:44:10.888648 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:10.888615    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57b9a95d-0eff-43e6-9140-97efd8e0bff0-istio-envoy\") pod \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\" (UID: \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\") "
Jul 26 18:44:10.888648 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:10.888641    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57b9a95d-0eff-43e6-9140-97efd8e0bff0-credential-socket\") pod \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\" (UID: \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\") "
Jul 26 18:44:10.888948 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:10.888660    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57b9a95d-0eff-43e6-9140-97efd8e0bff0-istio-data\") pod \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\" (UID: \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\") "
Jul 26 18:44:10.888948 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:10.888681    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57b9a95d-0eff-43e6-9140-97efd8e0bff0-workload-certs\") pod \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\" (UID: \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\") "
Jul 26 18:44:10.888948 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:10.888707    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57b9a95d-0eff-43e6-9140-97efd8e0bff0-istio-token\") pod \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\" (UID: \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\") "
Jul 26 18:44:10.888948 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:10.888779    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57b9a95d-0eff-43e6-9140-97efd8e0bff0-workload-socket\") pod \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\" (UID: \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\") "
Jul 26 18:44:10.888948 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:10.888821    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s9rxc\" (UniqueName: \"kubernetes.io/projected/57b9a95d-0eff-43e6-9140-97efd8e0bff0-kube-api-access-s9rxc\") pod \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\" (UID: \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\") "
Jul 26 18:44:10.888948 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:10.888862    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57b9a95d-0eff-43e6-9140-97efd8e0bff0-istio-podinfo\") pod \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\" (UID: \"57b9a95d-0eff-43e6-9140-97efd8e0bff0\") "
Jul 26 18:44:10.889248 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:10.888997    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57b9a95d-0eff-43e6-9140-97efd8e0bff0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "57b9a95d-0eff-43e6-9140-97efd8e0bff0" (UID: "57b9a95d-0eff-43e6-9140-97efd8e0bff0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:44:10.889248 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:10.889072    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57b9a95d-0eff-43e6-9140-97efd8e0bff0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "57b9a95d-0eff-43e6-9140-97efd8e0bff0" (UID: "57b9a95d-0eff-43e6-9140-97efd8e0bff0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:44:10.889248 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:10.889103    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57b9a95d-0eff-43e6-9140-97efd8e0bff0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "57b9a95d-0eff-43e6-9140-97efd8e0bff0" (UID: "57b9a95d-0eff-43e6-9140-97efd8e0bff0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:44:10.889248 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:10.889128    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57b9a95d-0eff-43e6-9140-97efd8e0bff0-credential-socket\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:44:10.889473 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:10.889444    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57b9a95d-0eff-43e6-9140-97efd8e0bff0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "57b9a95d-0eff-43e6-9140-97efd8e0bff0" (UID: "57b9a95d-0eff-43e6-9140-97efd8e0bff0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:44:10.889855 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:10.889835    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57b9a95d-0eff-43e6-9140-97efd8e0bff0-istio-data" (OuterVolumeSpecName: "istio-data") pod "57b9a95d-0eff-43e6-9140-97efd8e0bff0" (UID: "57b9a95d-0eff-43e6-9140-97efd8e0bff0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:44:10.891457 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:10.891427    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57b9a95d-0eff-43e6-9140-97efd8e0bff0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "57b9a95d-0eff-43e6-9140-97efd8e0bff0" (UID: "57b9a95d-0eff-43e6-9140-97efd8e0bff0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:44:10.891551 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:10.891469    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57b9a95d-0eff-43e6-9140-97efd8e0bff0-kube-api-access-s9rxc" (OuterVolumeSpecName: "kube-api-access-s9rxc") pod "57b9a95d-0eff-43e6-9140-97efd8e0bff0" (UID: "57b9a95d-0eff-43e6-9140-97efd8e0bff0"). InnerVolumeSpecName "kube-api-access-s9rxc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:44:10.891551 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:10.891490    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/57b9a95d-0eff-43e6-9140-97efd8e0bff0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "57b9a95d-0eff-43e6-9140-97efd8e0bff0" (UID: "57b9a95d-0eff-43e6-9140-97efd8e0bff0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 26 18:44:10.891551 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:10.891514    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57b9a95d-0eff-43e6-9140-97efd8e0bff0-istio-token" (OuterVolumeSpecName: "istio-token") pod "57b9a95d-0eff-43e6-9140-97efd8e0bff0" (UID: "57b9a95d-0eff-43e6-9140-97efd8e0bff0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:44:10.990050 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:10.990009    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57b9a95d-0eff-43e6-9140-97efd8e0bff0-istio-podinfo\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:44:10.990050 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:10.990040    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57b9a95d-0eff-43e6-9140-97efd8e0bff0-istiod-ca-cert\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:44:10.990050 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:10.990048    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57b9a95d-0eff-43e6-9140-97efd8e0bff0-istio-envoy\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:44:10.990050 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:10.990057    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57b9a95d-0eff-43e6-9140-97efd8e0bff0-istio-data\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:44:10.990050 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:10.990064    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57b9a95d-0eff-43e6-9140-97efd8e0bff0-workload-certs\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:44:10.990358 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:10.990072    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57b9a95d-0eff-43e6-9140-97efd8e0bff0-istio-token\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:44:10.990358 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:10.990079    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57b9a95d-0eff-43e6-9140-97efd8e0bff0-workload-socket\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:44:10.990358 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:10.990087    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9rxc\" (UniqueName: \"kubernetes.io/projected/57b9a95d-0eff-43e6-9140-97efd8e0bff0-kube-api-access-s9rxc\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:44:11.790797 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:11.790763    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"
Jul 26 18:44:11.790797 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:11.790784    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b" event={"ID":"57b9a95d-0eff-43e6-9140-97efd8e0bff0","Type":"ContainerDied","Data":"1dfcc0c60e735dd500620d1cf60da38e9747649c0cd22b66e9923c62c9db19b7"}
Jul 26 18:44:11.791271 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:11.790844    2576 scope.go:117] "RemoveContainer" containerID="04c5e0f9ea5446a233526a025430a3ae3710110339f7eb2b8b842365f50b4d7a"
Jul 26 18:44:11.812080 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:11.812043    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"]
Jul 26 18:44:11.815616 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:11.815589    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-0dc66676-openshift-default-7b9f9cb6bd-kpx6b"]
Jul 26 18:44:11.894871 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:11.894842    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:44:12.026188 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:12.026149    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:44:12.114097 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:12.114009    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:44:12.219550 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:12.219519    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:44:12.848687 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:12.848641    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57b9a95d-0eff-43e6-9140-97efd8e0bff0" path="/var/lib/kubelet/pods/57b9a95d-0eff-43e6-9140-97efd8e0bff0/volumes"
Jul 26 18:44:18.052216 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:18.052173    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:44:18.153126 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:18.153096    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:44:24.726527 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:24.726492    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:44:25.051023 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.050944    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"]
Jul 26 18:44:25.051283 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.051270    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57b9a95d-0eff-43e6-9140-97efd8e0bff0" containerName="istio-proxy"
Jul 26 18:44:25.051338 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.051285    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="57b9a95d-0eff-43e6-9140-97efd8e0bff0" containerName="istio-proxy"
Jul 26 18:44:25.051338 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.051296    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57b9a95d-0eff-43e6-9140-97efd8e0bff0" containerName="istio-proxy"
Jul 26 18:44:25.051338 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.051301    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="57b9a95d-0eff-43e6-9140-97efd8e0bff0" containerName="istio-proxy"
Jul 26 18:44:25.051338 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.051317    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0369b67-d9a2-4b47-aaed-b69a9a35fc4a" containerName="maas-api"
Jul 26 18:44:25.051338 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.051322    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0369b67-d9a2-4b47-aaed-b69a9a35fc4a" containerName="maas-api"
Jul 26 18:44:25.051547 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.051368    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0369b67-d9a2-4b47-aaed-b69a9a35fc4a" containerName="maas-api"
Jul 26 18:44:25.051547 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.051376    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="57b9a95d-0eff-43e6-9140-97efd8e0bff0" containerName="istio-proxy"
Jul 26 18:44:25.051547 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.051383    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="57b9a95d-0eff-43e6-9140-97efd8e0bff0" containerName="istio-proxy"
Jul 26 18:44:25.055895 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.055873    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:44:25.058376 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.058352    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-1d6ece93-openshift-default-dockercfg-z4v6s\""
Jul 26 18:44:25.066060 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.066032    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"]
Jul 26 18:44:25.075409 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.075375    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:44:25.212859 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.212821    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bca3868b-37c8-438a-9360-7de85fe86f41-istio-token\") pod \"e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt\" (UID: \"bca3868b-37c8-438a-9360-7de85fe86f41\") " pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:44:25.213056 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.212866    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bca3868b-37c8-438a-9360-7de85fe86f41-workload-certs\") pod \"e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt\" (UID: \"bca3868b-37c8-438a-9360-7de85fe86f41\") " pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:44:25.213056 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.212921    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bca3868b-37c8-438a-9360-7de85fe86f41-istio-envoy\") pod \"e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt\" (UID: \"bca3868b-37c8-438a-9360-7de85fe86f41\") " pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:44:25.213056 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.212937    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bca3868b-37c8-438a-9360-7de85fe86f41-workload-socket\") pod \"e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt\" (UID: \"bca3868b-37c8-438a-9360-7de85fe86f41\") " pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:44:25.213056 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.212960    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p76mx\" (UniqueName: \"kubernetes.io/projected/bca3868b-37c8-438a-9360-7de85fe86f41-kube-api-access-p76mx\") pod \"e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt\" (UID: \"bca3868b-37c8-438a-9360-7de85fe86f41\") " pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:44:25.213056 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.212980    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bca3868b-37c8-438a-9360-7de85fe86f41-istiod-ca-cert\") pod \"e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt\" (UID: \"bca3868b-37c8-438a-9360-7de85fe86f41\") " pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:44:25.213056 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.213045    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bca3868b-37c8-438a-9360-7de85fe86f41-istio-data\") pod \"e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt\" (UID: \"bca3868b-37c8-438a-9360-7de85fe86f41\") " pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:44:25.213284 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.213091    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bca3868b-37c8-438a-9360-7de85fe86f41-credential-socket\") pod \"e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt\" (UID: \"bca3868b-37c8-438a-9360-7de85fe86f41\") " pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:44:25.213284 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.213106    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bca3868b-37c8-438a-9360-7de85fe86f41-istio-podinfo\") pod \"e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt\" (UID: \"bca3868b-37c8-438a-9360-7de85fe86f41\") " pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:44:25.314535 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.314425    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bca3868b-37c8-438a-9360-7de85fe86f41-istio-token\") pod \"e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt\" (UID: \"bca3868b-37c8-438a-9360-7de85fe86f41\") " pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:44:25.314535 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.314474    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bca3868b-37c8-438a-9360-7de85fe86f41-workload-certs\") pod \"e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt\" (UID: \"bca3868b-37c8-438a-9360-7de85fe86f41\") " pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:44:25.314535 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.314493    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bca3868b-37c8-438a-9360-7de85fe86f41-istio-envoy\") pod \"e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt\" (UID: \"bca3868b-37c8-438a-9360-7de85fe86f41\") " pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:44:25.314535 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.314511    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bca3868b-37c8-438a-9360-7de85fe86f41-workload-socket\") pod \"e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt\" (UID: \"bca3868b-37c8-438a-9360-7de85fe86f41\") " pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:44:25.314535 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.314543    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p76mx\" (UniqueName: \"kubernetes.io/projected/bca3868b-37c8-438a-9360-7de85fe86f41-kube-api-access-p76mx\") pod \"e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt\" (UID: \"bca3868b-37c8-438a-9360-7de85fe86f41\") " pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:44:25.314993 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.314672    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bca3868b-37c8-438a-9360-7de85fe86f41-istiod-ca-cert\") pod \"e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt\" (UID: \"bca3868b-37c8-438a-9360-7de85fe86f41\") " pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:44:25.314993 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.314777    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bca3868b-37c8-438a-9360-7de85fe86f41-istio-data\") pod \"e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt\" (UID: \"bca3868b-37c8-438a-9360-7de85fe86f41\") " pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:44:25.314993 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.314833    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bca3868b-37c8-438a-9360-7de85fe86f41-credential-socket\") pod \"e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt\" (UID: \"bca3868b-37c8-438a-9360-7de85fe86f41\") " pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:44:25.314993 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.314860    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bca3868b-37c8-438a-9360-7de85fe86f41-istio-podinfo\") pod \"e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt\" (UID: \"bca3868b-37c8-438a-9360-7de85fe86f41\") " pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:44:25.315199 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.314986    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bca3868b-37c8-438a-9360-7de85fe86f41-workload-socket\") pod \"e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt\" (UID: \"bca3868b-37c8-438a-9360-7de85fe86f41\") " pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:44:25.315267 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.315201    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bca3868b-37c8-438a-9360-7de85fe86f41-credential-socket\") pod \"e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt\" (UID: \"bca3868b-37c8-438a-9360-7de85fe86f41\") " pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:44:25.315267 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.315224    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bca3868b-37c8-438a-9360-7de85fe86f41-istio-data\") pod \"e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt\" (UID: \"bca3868b-37c8-438a-9360-7de85fe86f41\") " pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:44:25.315373 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.314955    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bca3868b-37c8-438a-9360-7de85fe86f41-workload-certs\") pod \"e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt\" (UID: \"bca3868b-37c8-438a-9360-7de85fe86f41\") " pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:44:25.315423 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.315387    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bca3868b-37c8-438a-9360-7de85fe86f41-istiod-ca-cert\") pod \"e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt\" (UID: \"bca3868b-37c8-438a-9360-7de85fe86f41\") " pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:44:25.316987 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.316960    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bca3868b-37c8-438a-9360-7de85fe86f41-istio-envoy\") pod \"e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt\" (UID: \"bca3868b-37c8-438a-9360-7de85fe86f41\") " pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:44:25.317246 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.317228    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bca3868b-37c8-438a-9360-7de85fe86f41-istio-podinfo\") pod \"e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt\" (UID: \"bca3868b-37c8-438a-9360-7de85fe86f41\") " pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:44:25.322235 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.322208    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bca3868b-37c8-438a-9360-7de85fe86f41-istio-token\") pod \"e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt\" (UID: \"bca3868b-37c8-438a-9360-7de85fe86f41\") " pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:44:25.322802 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.322780    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p76mx\" (UniqueName: \"kubernetes.io/projected/bca3868b-37c8-438a-9360-7de85fe86f41-kube-api-access-p76mx\") pod \"e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt\" (UID: \"bca3868b-37c8-438a-9360-7de85fe86f41\") " pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:44:25.366835 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.366792    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:44:25.416327 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.416293    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:44:25.507738 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.507709    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"]
Jul 26 18:44:25.509951 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:44:25.509920    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbca3868b_37c8_438a_9360_7de85fe86f41.slice/crio-3b313b8304728a77e19d0d4e23dcaffda99811a80a529aff7a4870525dce8fd8 WatchSource:0}: Error finding container 3b313b8304728a77e19d0d4e23dcaffda99811a80a529aff7a4870525dce8fd8: Status 404 returned error can't find the container with id 3b313b8304728a77e19d0d4e23dcaffda99811a80a529aff7a4870525dce8fd8
Jul 26 18:44:25.512363 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.512331    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:44:25.512455 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.512399    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:44:25.512455 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.512427    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:44:25.857331 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.857296    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt" event={"ID":"bca3868b-37c8-438a-9360-7de85fe86f41","Type":"ContainerStarted","Data":"5bf0826064a2e518096c6e97eeb35c97437d66df3e27d537b92f3ccd43297b3b"}
Jul 26 18:44:25.857762 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.857349    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt" event={"ID":"bca3868b-37c8-438a-9360-7de85fe86f41","Type":"ContainerStarted","Data":"3b313b8304728a77e19d0d4e23dcaffda99811a80a529aff7a4870525dce8fd8"}
Jul 26 18:44:25.880621 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:25.880564    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt" podStartSLOduration=0.880547408 podStartE2EDuration="880.547408ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:44:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:44:25.87868238 +0000 UTC m=+1951.586089449" watchObservedRunningTime="2026-07-26 18:44:25.880547408 +0000 UTC m=+1951.587954466"
Jul 26 18:44:26.367573 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:26.367530    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:44:26.372915 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:26.372881    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:44:26.861880 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:26.861850    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:44:26.862808 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:26.862785    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:44:27.724090 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:27.724045    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:44:27.824914 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:27.824874    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75"]
Jul 26 18:44:27.826478 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:27.826444    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:44:27.828575 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:27.828555    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75"
Jul 26 18:44:27.831449 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:27.831423    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-1d6ece93-serving-cert\""
Jul 26 18:44:27.831449 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:27.831442    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 26 18:44:27.831620 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:27.831589    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 18:44:27.832627 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:27.832605    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9n9b7\""
Jul 26 18:44:27.838535 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:27.838511    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75"]
Jul 26 18:44:27.892304 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:27.892265    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:44:27.938755 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:27.938708    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfca8d7b-20df-4534-abd0-a761c66f76f7-openshift-service-ca\") pod \"maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75\" (UID: \"dfca8d7b-20df-4534-abd0-a761c66f76f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75"
Jul 26 18:44:27.938935 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:27.938853    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p74j8\" (UniqueName: \"kubernetes.io/projected/dfca8d7b-20df-4534-abd0-a761c66f76f7-kube-api-access-p74j8\") pod \"maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75\" (UID: \"dfca8d7b-20df-4534-abd0-a761c66f76f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75"
Jul 26 18:44:27.938935 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:27.938893    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dfca8d7b-20df-4534-abd0-a761c66f76f7-maas-api-tls\") pod \"maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75\" (UID: \"dfca8d7b-20df-4534-abd0-a761c66f76f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75"
Jul 26 18:44:28.039616 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:28.039513    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p74j8\" (UniqueName: \"kubernetes.io/projected/dfca8d7b-20df-4534-abd0-a761c66f76f7-kube-api-access-p74j8\") pod \"maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75\" (UID: \"dfca8d7b-20df-4534-abd0-a761c66f76f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75"
Jul 26 18:44:28.039616 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:28.039562    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dfca8d7b-20df-4534-abd0-a761c66f76f7-maas-api-tls\") pod \"maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75\" (UID: \"dfca8d7b-20df-4534-abd0-a761c66f76f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75"
Jul 26 18:44:28.039898 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:44:28.039689    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-1d6ece93-serving-cert: secret "maas-api-e2e-mt-1d6ece93-serving-cert" not found
Jul 26 18:44:28.039898 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:28.039737    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfca8d7b-20df-4534-abd0-a761c66f76f7-openshift-service-ca\") pod \"maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75\" (UID: \"dfca8d7b-20df-4534-abd0-a761c66f76f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75"
Jul 26 18:44:28.039898 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:44:28.039796    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfca8d7b-20df-4534-abd0-a761c66f76f7-maas-api-tls podName:dfca8d7b-20df-4534-abd0-a761c66f76f7 nodeName:}" failed. No retries permitted until 2026-07-26 18:44:28.539741999 +0000 UTC m=+1954.247149050 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/dfca8d7b-20df-4534-abd0-a761c66f76f7-maas-api-tls") pod "maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75" (UID: "dfca8d7b-20df-4534-abd0-a761c66f76f7") : secret "maas-api-e2e-mt-1d6ece93-serving-cert" not found
Jul 26 18:44:28.040435 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:28.040416    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfca8d7b-20df-4534-abd0-a761c66f76f7-openshift-service-ca\") pod \"maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75\" (UID: \"dfca8d7b-20df-4534-abd0-a761c66f76f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75"
Jul 26 18:44:28.048698 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:28.048673    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p74j8\" (UniqueName: \"kubernetes.io/projected/dfca8d7b-20df-4534-abd0-a761c66f76f7-kube-api-access-p74j8\") pod \"maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75\" (UID: \"dfca8d7b-20df-4534-abd0-a761c66f76f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75"
Jul 26 18:44:28.285343 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:28.285311    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:44:28.383021 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:28.382926    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:44:28.471259 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:28.471227    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:44:28.545098 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:28.545047    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dfca8d7b-20df-4534-abd0-a761c66f76f7-maas-api-tls\") pod \"maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75\" (UID: \"dfca8d7b-20df-4534-abd0-a761c66f76f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75"
Jul 26 18:44:28.547581 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:28.547556    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dfca8d7b-20df-4534-abd0-a761c66f76f7-maas-api-tls\") pod \"maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75\" (UID: \"dfca8d7b-20df-4534-abd0-a761c66f76f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75"
Jul 26 18:44:28.739806 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:28.739722    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75"
Jul 26 18:44:28.877185 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:28.877156    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75"]
Jul 26 18:44:28.879516 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:44:28.879492    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfca8d7b_20df_4534_abd0_a761c66f76f7.slice/crio-598b049e9cf6879816d3bf91a002cff459266bef7f10dba9350210afd6e1485e WatchSource:0}: Error finding container 598b049e9cf6879816d3bf91a002cff459266bef7f10dba9350210afd6e1485e: Status 404 returned error can't find the container with id 598b049e9cf6879816d3bf91a002cff459266bef7f10dba9350210afd6e1485e
Jul 26 18:44:28.881102 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:28.881076    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 18:44:29.875951 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:29.875918    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75" event={"ID":"dfca8d7b-20df-4534-abd0-a761c66f76f7","Type":"ContainerStarted","Data":"afe1defefdab2eb15347f764bcd599c5e05f699abed73c9b30e0fc1dee950aa7"}
Jul 26 18:44:29.876361 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:29.875976    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75" event={"ID":"dfca8d7b-20df-4534-abd0-a761c66f76f7","Type":"ContainerStarted","Data":"598b049e9cf6879816d3bf91a002cff459266bef7f10dba9350210afd6e1485e"}
Jul 26 18:44:29.876361 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:29.876085    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75"
Jul 26 18:44:29.893573 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:29.893520    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75" podStartSLOduration=2.47020213 podStartE2EDuration="2.893504629s" totalImagesPullingTime="423.302499ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:44:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:44:29.891666285 +0000 UTC m=+1955.599073343" watchObservedRunningTime="2026-07-26 18:44:29.893504629 +0000 UTC m=+1955.600912082"
Jul 26 18:44:31.787056 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:31.787014    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:44:32.197814 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:32.197777    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:44:35.885951 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:35.885920    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75"
Jul 26 18:44:39.921802 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:39.921772    2576 generic.go:356] "Generic (PLEG): container finished" podID="bca3868b-37c8-438a-9360-7de85fe86f41" containerID="5bf0826064a2e518096c6e97eeb35c97437d66df3e27d537b92f3ccd43297b3b" exitCode=0
Jul 26 18:44:39.922197 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:39.921835    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt" event={"ID":"bca3868b-37c8-438a-9360-7de85fe86f41","Type":"ContainerDied","Data":"5bf0826064a2e518096c6e97eeb35c97437d66df3e27d537b92f3ccd43297b3b"}
Jul 26 18:44:39.922240 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:39.922208    2576 scope.go:117] "RemoveContainer" containerID="5bf0826064a2e518096c6e97eeb35c97437d66df3e27d537b92f3ccd43297b3b"
Jul 26 18:44:39.923184 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:39.923160    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:44:39.923239 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:39.923226    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:44:39.923274 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:39.923263    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:44:40.929189 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:40.929149    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt" event={"ID":"bca3868b-37c8-438a-9360-7de85fe86f41","Type":"ContainerStarted","Data":"96cc4173cedf2c87bd95be1b9052a94b71034ba28cfb02461d8d16268e3d7e35"}
Jul 26 18:44:41.367462 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:41.367348    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:44:41.368630 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:41.368594    2576 patch_prober.go:28] interesting pod/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Jul 26 18:44:41.368781 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:41.368678    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt" podUID="bca3868b-37c8-438a-9360-7de85fe86f41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Jul 26 18:44:41.862175 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:41.862124    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:44:42.368353 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:42.368300    2576 patch_prober.go:28] interesting pod/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Jul 26 18:44:42.368772 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:42.368398    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt" podUID="bca3868b-37c8-438a-9360-7de85fe86f41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Jul 26 18:44:43.367873 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:43.367828    2576 patch_prober.go:28] interesting pod/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Jul 26 18:44:43.368092 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:43.367896    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt" podUID="bca3868b-37c8-438a-9360-7de85fe86f41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Jul 26 18:44:44.367268 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:44.367229    2576 patch_prober.go:28] interesting pod/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Jul 26 18:44:44.367786 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:44.367297    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt" podUID="bca3868b-37c8-438a-9360-7de85fe86f41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Jul 26 18:44:45.372378 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:45.372347    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:44:45.373459 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:45.373441    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:44:49.528892 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:49.528809    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:44:52.688771 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:44:52.688720    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:45:08.102090 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:45:08.102035    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:45:26.384916 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:45:26.384883    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:45:27.744315 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:45:27.744273    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75"]
Jul 26 18:45:27.744847 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:45:27.744557    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75" podUID="dfca8d7b-20df-4534-abd0-a761c66f76f7" containerName="maas-api" containerID="cri-o://afe1defefdab2eb15347f764bcd599c5e05f699abed73c9b30e0fc1dee950aa7" gracePeriod=30
Jul 26 18:45:27.766247 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:45:27.766221    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:45:27.831371 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:45:27.831331    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:45:27.889130 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:45:27.889095    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:45:27.996541 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:45:27.996469    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75"
Jul 26 18:45:28.133810 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:45:28.133778    2576 generic.go:356] "Generic (PLEG): container finished" podID="dfca8d7b-20df-4534-abd0-a761c66f76f7" containerID="afe1defefdab2eb15347f764bcd599c5e05f699abed73c9b30e0fc1dee950aa7" exitCode=0
Jul 26 18:45:28.133986 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:45:28.133825    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75"
Jul 26 18:45:28.133986 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:45:28.133866    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75" event={"ID":"dfca8d7b-20df-4534-abd0-a761c66f76f7","Type":"ContainerDied","Data":"afe1defefdab2eb15347f764bcd599c5e05f699abed73c9b30e0fc1dee950aa7"}
Jul 26 18:45:28.133986 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:45:28.133926    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75" event={"ID":"dfca8d7b-20df-4534-abd0-a761c66f76f7","Type":"ContainerDied","Data":"598b049e9cf6879816d3bf91a002cff459266bef7f10dba9350210afd6e1485e"}
Jul 26 18:45:28.133986 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:45:28.133950    2576 scope.go:117] "RemoveContainer" containerID="afe1defefdab2eb15347f764bcd599c5e05f699abed73c9b30e0fc1dee950aa7"
Jul 26 18:45:28.141954 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:45:28.141933    2576 scope.go:117] "RemoveContainer" containerID="afe1defefdab2eb15347f764bcd599c5e05f699abed73c9b30e0fc1dee950aa7"
Jul 26 18:45:28.142253 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:45:28.142229    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afe1defefdab2eb15347f764bcd599c5e05f699abed73c9b30e0fc1dee950aa7\": container with ID starting with afe1defefdab2eb15347f764bcd599c5e05f699abed73c9b30e0fc1dee950aa7 not found: ID does not exist" containerID="afe1defefdab2eb15347f764bcd599c5e05f699abed73c9b30e0fc1dee950aa7"
Jul 26 18:45:28.142320 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:45:28.142266    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afe1defefdab2eb15347f764bcd599c5e05f699abed73c9b30e0fc1dee950aa7"} err="failed to get container status \"afe1defefdab2eb15347f764bcd599c5e05f699abed73c9b30e0fc1dee950aa7\": rpc error: code = NotFound desc = could not find container \"afe1defefdab2eb15347f764bcd599c5e05f699abed73c9b30e0fc1dee950aa7\": container with ID starting with afe1defefdab2eb15347f764bcd599c5e05f699abed73c9b30e0fc1dee950aa7 not found: ID does not exist"
Jul 26 18:45:28.159608 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:45:28.159571    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfca8d7b-20df-4534-abd0-a761c66f76f7-openshift-service-ca\") pod \"dfca8d7b-20df-4534-abd0-a761c66f76f7\" (UID: \"dfca8d7b-20df-4534-abd0-a761c66f76f7\") "
Jul 26 18:45:28.159796 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:45:28.159656    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p74j8\" (UniqueName: \"kubernetes.io/projected/dfca8d7b-20df-4534-abd0-a761c66f76f7-kube-api-access-p74j8\") pod \"dfca8d7b-20df-4534-abd0-a761c66f76f7\" (UID: \"dfca8d7b-20df-4534-abd0-a761c66f76f7\") "
Jul 26 18:45:28.159796 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:45:28.159685    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dfca8d7b-20df-4534-abd0-a761c66f76f7-maas-api-tls\") pod \"dfca8d7b-20df-4534-abd0-a761c66f76f7\" (UID: \"dfca8d7b-20df-4534-abd0-a761c66f76f7\") "
Jul 26 18:45:28.160047 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:45:28.160018    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dfca8d7b-20df-4534-abd0-a761c66f76f7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "dfca8d7b-20df-4534-abd0-a761c66f76f7" (UID: "dfca8d7b-20df-4534-abd0-a761c66f76f7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:45:28.161785 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:45:28.161756    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfca8d7b-20df-4534-abd0-a761c66f76f7-kube-api-access-p74j8" (OuterVolumeSpecName: "kube-api-access-p74j8") pod "dfca8d7b-20df-4534-abd0-a761c66f76f7" (UID: "dfca8d7b-20df-4534-abd0-a761c66f76f7"). InnerVolumeSpecName "kube-api-access-p74j8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:45:28.161892 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:45:28.161867    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfca8d7b-20df-4534-abd0-a761c66f76f7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "dfca8d7b-20df-4534-abd0-a761c66f76f7" (UID: "dfca8d7b-20df-4534-abd0-a761c66f76f7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:45:28.260659 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:45:28.260571    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfca8d7b-20df-4534-abd0-a761c66f76f7-openshift-service-ca\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:45:28.260659 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:45:28.260602    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p74j8\" (UniqueName: \"kubernetes.io/projected/dfca8d7b-20df-4534-abd0-a761c66f76f7-kube-api-access-p74j8\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:45:28.260659 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:45:28.260612    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dfca8d7b-20df-4534-abd0-a761c66f76f7-maas-api-tls\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:45:28.455021 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:45:28.454985    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75"]
Jul 26 18:45:28.458822 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:45:28.458794    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1d6ece93-ffc6b75d7-hxm75"]
Jul 26 18:45:28.849174 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:45:28.849143    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfca8d7b-20df-4534-abd0-a761c66f76f7" path="/var/lib/kubelet/pods/dfca8d7b-20df-4534-abd0-a761c66f76f7/volumes"
Jul 26 18:45:32.374924 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:45:32.374890    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:45:35.384247 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:45:35.384215    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:45:36.880535 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:45:36.880497    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:45:40.412317 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:45:40.412280    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:45:44.971224 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:45:44.971187    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:45:45.072837 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:45:45.072807    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:45:54.945409 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:45:54.945379    2576 scope.go:117] "RemoveContainer" containerID="16a979f48e03287f34fe8b885fc0290d9d239f418c4e00415f53f6547c42d82b"
Jul 26 18:46:17.085817 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:17.085724    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:46:17.388351 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:17.388270    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:46:22.858979 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:22.858940    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:46:28.443369 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:28.443322    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:46:28.506533 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:28.506495    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:46:28.697879 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:28.697791    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"]
Jul 26 18:46:28.698079 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:28.698054    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt" podUID="bca3868b-37c8-438a-9360-7de85fe86f41" containerName="istio-proxy" containerID="cri-o://96cc4173cedf2c87bd95be1b9052a94b71034ba28cfb02461d8d16268e3d7e35" gracePeriod=30
Jul 26 18:46:28.721226 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:28.721195    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:46:28.946667 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:28.946630    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:46:29.960932 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:29.960872    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:46:30.057555 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:30.057524    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:46:30.156849 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:30.156807    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:46:30.375667 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:30.375575    2576 patch_prober.go:28] interesting pod/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 26 18:46:30.375840 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:30.375662    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt" podUID="bca3868b-37c8-438a-9360-7de85fe86f41" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 26 18:46:30.858629 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:30.858593    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:46:30.950183 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:30.950148    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:46:31.049876 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:31.049840    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:46:32.365612 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:32.365582    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:46:32.706201 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:32.706162    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:46:33.959390 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:33.959325    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:46:34.130491 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.130445    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bca3868b-37c8-438a-9360-7de85fe86f41-istiod-ca-cert\") pod \"bca3868b-37c8-438a-9360-7de85fe86f41\" (UID: \"bca3868b-37c8-438a-9360-7de85fe86f41\") "
Jul 26 18:46:34.130491 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.130499    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bca3868b-37c8-438a-9360-7de85fe86f41-istio-token\") pod \"bca3868b-37c8-438a-9360-7de85fe86f41\" (UID: \"bca3868b-37c8-438a-9360-7de85fe86f41\") "
Jul 26 18:46:34.130795 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.130526    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bca3868b-37c8-438a-9360-7de85fe86f41-credential-socket\") pod \"bca3868b-37c8-438a-9360-7de85fe86f41\" (UID: \"bca3868b-37c8-438a-9360-7de85fe86f41\") "
Jul 26 18:46:34.130795 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.130654    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bca3868b-37c8-438a-9360-7de85fe86f41-istio-data\") pod \"bca3868b-37c8-438a-9360-7de85fe86f41\" (UID: \"bca3868b-37c8-438a-9360-7de85fe86f41\") "
Jul 26 18:46:34.130795 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.130698    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p76mx\" (UniqueName: \"kubernetes.io/projected/bca3868b-37c8-438a-9360-7de85fe86f41-kube-api-access-p76mx\") pod \"bca3868b-37c8-438a-9360-7de85fe86f41\" (UID: \"bca3868b-37c8-438a-9360-7de85fe86f41\") "
Jul 26 18:46:34.130795 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.130729    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bca3868b-37c8-438a-9360-7de85fe86f41-workload-socket\") pod \"bca3868b-37c8-438a-9360-7de85fe86f41\" (UID: \"bca3868b-37c8-438a-9360-7de85fe86f41\") "
Jul 26 18:46:34.131023 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.130795    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bca3868b-37c8-438a-9360-7de85fe86f41-istio-podinfo\") pod \"bca3868b-37c8-438a-9360-7de85fe86f41\" (UID: \"bca3868b-37c8-438a-9360-7de85fe86f41\") "
Jul 26 18:46:34.131023 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.130807    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bca3868b-37c8-438a-9360-7de85fe86f41-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bca3868b-37c8-438a-9360-7de85fe86f41" (UID: "bca3868b-37c8-438a-9360-7de85fe86f41"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:46:34.131023 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.130828    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bca3868b-37c8-438a-9360-7de85fe86f41-istio-envoy\") pod \"bca3868b-37c8-438a-9360-7de85fe86f41\" (UID: \"bca3868b-37c8-438a-9360-7de85fe86f41\") "
Jul 26 18:46:34.131023 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.130880    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bca3868b-37c8-438a-9360-7de85fe86f41-workload-certs\") pod \"bca3868b-37c8-438a-9360-7de85fe86f41\" (UID: \"bca3868b-37c8-438a-9360-7de85fe86f41\") "
Jul 26 18:46:34.131023 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.130950    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bca3868b-37c8-438a-9360-7de85fe86f41-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bca3868b-37c8-438a-9360-7de85fe86f41" (UID: "bca3868b-37c8-438a-9360-7de85fe86f41"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:46:34.131272 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.131093    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bca3868b-37c8-438a-9360-7de85fe86f41-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bca3868b-37c8-438a-9360-7de85fe86f41" (UID: "bca3868b-37c8-438a-9360-7de85fe86f41"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:46:34.131272 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.131186    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bca3868b-37c8-438a-9360-7de85fe86f41-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bca3868b-37c8-438a-9360-7de85fe86f41" (UID: "bca3868b-37c8-438a-9360-7de85fe86f41"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:46:34.131380 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.131326    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bca3868b-37c8-438a-9360-7de85fe86f41-credential-socket\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:46:34.131380 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.131342    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bca3868b-37c8-438a-9360-7de85fe86f41-workload-socket\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:46:34.131380 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.131359    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bca3868b-37c8-438a-9360-7de85fe86f41-workload-certs\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:46:34.131380 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.131371    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bca3868b-37c8-438a-9360-7de85fe86f41-istiod-ca-cert\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:46:34.131697 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.131669    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bca3868b-37c8-438a-9360-7de85fe86f41-istio-data" (OuterVolumeSpecName: "istio-data") pod "bca3868b-37c8-438a-9360-7de85fe86f41" (UID: "bca3868b-37c8-438a-9360-7de85fe86f41"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:46:34.132934 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.132900    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bca3868b-37c8-438a-9360-7de85fe86f41-istio-token" (OuterVolumeSpecName: "istio-token") pod "bca3868b-37c8-438a-9360-7de85fe86f41" (UID: "bca3868b-37c8-438a-9360-7de85fe86f41"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:46:34.133037 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.132930    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bca3868b-37c8-438a-9360-7de85fe86f41-kube-api-access-p76mx" (OuterVolumeSpecName: "kube-api-access-p76mx") pod "bca3868b-37c8-438a-9360-7de85fe86f41" (UID: "bca3868b-37c8-438a-9360-7de85fe86f41"). InnerVolumeSpecName "kube-api-access-p76mx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:46:34.133353 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.133330    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bca3868b-37c8-438a-9360-7de85fe86f41-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bca3868b-37c8-438a-9360-7de85fe86f41" (UID: "bca3868b-37c8-438a-9360-7de85fe86f41"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:46:34.133353 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.133337    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bca3868b-37c8-438a-9360-7de85fe86f41-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bca3868b-37c8-438a-9360-7de85fe86f41" (UID: "bca3868b-37c8-438a-9360-7de85fe86f41"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 26 18:46:34.232674 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.232572    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bca3868b-37c8-438a-9360-7de85fe86f41-istio-envoy\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:46:34.232674 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.232611    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bca3868b-37c8-438a-9360-7de85fe86f41-istio-token\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:46:34.232674 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.232620    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bca3868b-37c8-438a-9360-7de85fe86f41-istio-data\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:46:34.232674 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.232631    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p76mx\" (UniqueName: \"kubernetes.io/projected/bca3868b-37c8-438a-9360-7de85fe86f41-kube-api-access-p76mx\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:46:34.232674 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.232641    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bca3868b-37c8-438a-9360-7de85fe86f41-istio-podinfo\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:46:34.416674 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.416640    2576 generic.go:356] "Generic (PLEG): container finished" podID="bca3868b-37c8-438a-9360-7de85fe86f41" containerID="96cc4173cedf2c87bd95be1b9052a94b71034ba28cfb02461d8d16268e3d7e35" exitCode=0
Jul 26 18:46:34.416875 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.416694    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"
Jul 26 18:46:34.416875 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.416714    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt" event={"ID":"bca3868b-37c8-438a-9360-7de85fe86f41","Type":"ContainerDied","Data":"96cc4173cedf2c87bd95be1b9052a94b71034ba28cfb02461d8d16268e3d7e35"}
Jul 26 18:46:34.416875 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.416771    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt" event={"ID":"bca3868b-37c8-438a-9360-7de85fe86f41","Type":"ContainerDied","Data":"3b313b8304728a77e19d0d4e23dcaffda99811a80a529aff7a4870525dce8fd8"}
Jul 26 18:46:34.416875 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.416792    2576 scope.go:117] "RemoveContainer" containerID="96cc4173cedf2c87bd95be1b9052a94b71034ba28cfb02461d8d16268e3d7e35"
Jul 26 18:46:34.425452 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.425433    2576 scope.go:117] "RemoveContainer" containerID="5bf0826064a2e518096c6e97eeb35c97437d66df3e27d537b92f3ccd43297b3b"
Jul 26 18:46:34.433175 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.433152    2576 scope.go:117] "RemoveContainer" containerID="96cc4173cedf2c87bd95be1b9052a94b71034ba28cfb02461d8d16268e3d7e35"
Jul 26 18:46:34.433434 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:46:34.433413    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96cc4173cedf2c87bd95be1b9052a94b71034ba28cfb02461d8d16268e3d7e35\": container with ID starting with 96cc4173cedf2c87bd95be1b9052a94b71034ba28cfb02461d8d16268e3d7e35 not found: ID does not exist" containerID="96cc4173cedf2c87bd95be1b9052a94b71034ba28cfb02461d8d16268e3d7e35"
Jul 26 18:46:34.433487 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.433443    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96cc4173cedf2c87bd95be1b9052a94b71034ba28cfb02461d8d16268e3d7e35"} err="failed to get container status \"96cc4173cedf2c87bd95be1b9052a94b71034ba28cfb02461d8d16268e3d7e35\": rpc error: code = NotFound desc = could not find container \"96cc4173cedf2c87bd95be1b9052a94b71034ba28cfb02461d8d16268e3d7e35\": container with ID starting with 96cc4173cedf2c87bd95be1b9052a94b71034ba28cfb02461d8d16268e3d7e35 not found: ID does not exist"
Jul 26 18:46:34.433487 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.433463    2576 scope.go:117] "RemoveContainer" containerID="5bf0826064a2e518096c6e97eeb35c97437d66df3e27d537b92f3ccd43297b3b"
Jul 26 18:46:34.433681 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:46:34.433660    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bf0826064a2e518096c6e97eeb35c97437d66df3e27d537b92f3ccd43297b3b\": container with ID starting with 5bf0826064a2e518096c6e97eeb35c97437d66df3e27d537b92f3ccd43297b3b not found: ID does not exist" containerID="5bf0826064a2e518096c6e97eeb35c97437d66df3e27d537b92f3ccd43297b3b"
Jul 26 18:46:34.433723 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.433687    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bf0826064a2e518096c6e97eeb35c97437d66df3e27d537b92f3ccd43297b3b"} err="failed to get container status \"5bf0826064a2e518096c6e97eeb35c97437d66df3e27d537b92f3ccd43297b3b\": rpc error: code = NotFound desc = could not find container \"5bf0826064a2e518096c6e97eeb35c97437d66df3e27d537b92f3ccd43297b3b\": container with ID starting with 5bf0826064a2e518096c6e97eeb35c97437d66df3e27d537b92f3ccd43297b3b not found: ID does not exist"
Jul 26 18:46:34.440853 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.440830    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"]
Jul 26 18:46:34.446250 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.446225    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-1d6ece93-openshift-default-548f469f6d-x9wqt"]
Jul 26 18:46:34.849282 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:34.849243    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bca3868b-37c8-438a-9360-7de85fe86f41" path="/var/lib/kubelet/pods/bca3868b-37c8-438a-9360-7de85fe86f41/volumes"
Jul 26 18:46:38.242806 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:38.242734    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:46:38.343869 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:38.343835    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:46:38.887152 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:38.887114    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:46:44.691457 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:44.691421    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:46:44.887820 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:44.887789    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:46:44.996233 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:44.996134    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:46:45.094950 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:45.094917    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:46:51.572461 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:51.572425    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:46:55.563709 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:55.563666    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:46:55.915912 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:46:55.915824    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:47:16.832260 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:16.832223    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:47:23.129309 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.129277    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:47:23.234925 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.234893    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:47:23.459164 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.459129    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:47:23.739875 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.739786    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"]
Jul 26 18:47:23.740241 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.740226    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bca3868b-37c8-438a-9360-7de85fe86f41" containerName="istio-proxy"
Jul 26 18:47:23.740292 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.740242    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bca3868b-37c8-438a-9360-7de85fe86f41" containerName="istio-proxy"
Jul 26 18:47:23.740292 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.740257    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfca8d7b-20df-4534-abd0-a761c66f76f7" containerName="maas-api"
Jul 26 18:47:23.740292 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.740265    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfca8d7b-20df-4534-abd0-a761c66f76f7" containerName="maas-api"
Jul 26 18:47:23.740396 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.740340    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="bca3868b-37c8-438a-9360-7de85fe86f41" containerName="istio-proxy"
Jul 26 18:47:23.740396 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.740356    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="bca3868b-37c8-438a-9360-7de85fe86f41" containerName="istio-proxy"
Jul 26 18:47:23.740396 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.740369    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfca8d7b-20df-4534-abd0-a761c66f76f7" containerName="maas-api"
Jul 26 18:47:23.743574 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.743550    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:47:23.746452 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.746430    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-6329ea-openshift-default-dockercfg-vv4tb\""
Jul 26 18:47:23.756151 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.756126    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"]
Jul 26 18:47:23.770698 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.770664    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:47:23.859139 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.859106    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prldd\" (UniqueName: \"kubernetes.io/projected/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-kube-api-access-prldd\") pod \"e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg\" (UID: \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\") " pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:47:23.859327 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.859155    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-istio-podinfo\") pod \"e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg\" (UID: \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\") " pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:47:23.859327 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.859235    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-workload-certs\") pod \"e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg\" (UID: \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\") " pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:47:23.859327 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.859310    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-credential-socket\") pod \"e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg\" (UID: \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\") " pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:47:23.859457 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.859329    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-istio-envoy\") pod \"e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg\" (UID: \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\") " pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:47:23.859457 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.859352    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-istiod-ca-cert\") pod \"e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg\" (UID: \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\") " pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:47:23.859457 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.859391    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-workload-socket\") pod \"e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg\" (UID: \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\") " pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:47:23.859457 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.859410    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-istio-token\") pod \"e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg\" (UID: \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\") " pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:47:23.859457 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.859442    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-istio-data\") pod \"e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg\" (UID: \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\") " pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:47:23.883602 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.883573    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:47:23.960615 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.960574    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-istio-podinfo\") pod \"e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg\" (UID: \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\") " pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:47:23.960847 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.960626    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-workload-certs\") pod \"e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg\" (UID: \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\") " pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:47:23.960847 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.960679    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-credential-socket\") pod \"e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg\" (UID: \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\") " pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:47:23.960847 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.960701    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-istio-envoy\") pod \"e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg\" (UID: \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\") " pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:47:23.960847 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.960724    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-istiod-ca-cert\") pod \"e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg\" (UID: \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\") " pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:47:23.960847 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.960802    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-workload-socket\") pod \"e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg\" (UID: \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\") " pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:47:23.960847 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.960825    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-istio-token\") pod \"e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg\" (UID: \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\") " pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:47:23.961171 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.960859    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-istio-data\") pod \"e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg\" (UID: \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\") " pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:47:23.961171 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.960902    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prldd\" (UniqueName: \"kubernetes.io/projected/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-kube-api-access-prldd\") pod \"e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg\" (UID: \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\") " pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:47:23.961171 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.961135    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-credential-socket\") pod \"e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg\" (UID: \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\") " pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:47:23.961331 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.961172    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-workload-certs\") pod \"e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg\" (UID: \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\") " pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:47:23.961383 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.961342    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-istio-data\") pod \"e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg\" (UID: \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\") " pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:47:23.961466 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.961443    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-workload-socket\") pod \"e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg\" (UID: \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\") " pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:47:23.961684 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.961664    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-istiod-ca-cert\") pod \"e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg\" (UID: \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\") " pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:47:23.963212 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.963189    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-istio-envoy\") pod \"e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg\" (UID: \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\") " pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:47:23.963336 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.963321    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-istio-podinfo\") pod \"e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg\" (UID: \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\") " pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:47:23.970922 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.970902    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-istio-token\") pod \"e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg\" (UID: \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\") " pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:47:23.972986 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:23.972965    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prldd\" (UniqueName: \"kubernetes.io/projected/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-kube-api-access-prldd\") pod \"e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg\" (UID: \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\") " pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:47:24.054930 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:24.054840    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:47:24.195886 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:24.195865    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"]
Jul 26 18:47:24.198237 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:47:24.198206    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ad356e4_5b9a_4ec8_a803_1a2bf20d1e55.slice/crio-ff64f454c2e8081b30e8b5aea7a0f0bb287f1595c17deaa3532c18373b47084d WatchSource:0}: Error finding container ff64f454c2e8081b30e8b5aea7a0f0bb287f1595c17deaa3532c18373b47084d: Status 404 returned error can't find the container with id ff64f454c2e8081b30e8b5aea7a0f0bb287f1595c17deaa3532c18373b47084d
Jul 26 18:47:24.200479 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:24.200446    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:47:24.200550 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:24.200516    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:47:24.200603 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:24.200547    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:47:24.631259 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:24.631221    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg" event={"ID":"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55","Type":"ContainerStarted","Data":"77f6823a9de16946ed2d5e4eb70e25db38785989f13d564cb68d7cc29cdd31b7"}
Jul 26 18:47:24.631460 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:24.631286    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg" event={"ID":"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55","Type":"ContainerStarted","Data":"ff64f454c2e8081b30e8b5aea7a0f0bb287f1595c17deaa3532c18373b47084d"}
Jul 26 18:47:24.653675 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:24.653621    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg" podStartSLOduration=1.653604429 podStartE2EDuration="1.653604429s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:47:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:47:24.652207092 +0000 UTC m=+2130.359614153" watchObservedRunningTime="2026-07-26 18:47:24.653604429 +0000 UTC m=+2130.361011487"
Jul 26 18:47:25.055650 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:25.055607    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:47:25.060921 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:25.060888    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:47:25.638516 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:25.638477    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:47:25.639667 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:25.639647    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:47:26.628432 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:26.628402    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:47:26.728524 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:26.728469    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:47:26.782660 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:26.782623    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6329ea-864b49f685-nshzt"]
Jul 26 18:47:26.783147 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:26.783129    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bca3868b-37c8-438a-9360-7de85fe86f41" containerName="istio-proxy"
Jul 26 18:47:26.783214 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:26.783151    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bca3868b-37c8-438a-9360-7de85fe86f41" containerName="istio-proxy"
Jul 26 18:47:26.786534 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:26.786512    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6329ea-864b49f685-nshzt"
Jul 26 18:47:26.790177 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:26.790151    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-6329ea-serving-cert\""
Jul 26 18:47:26.790687 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:26.790396    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9n9b7\""
Jul 26 18:47:26.791386 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:26.791363    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 18:47:26.791506 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:26.791405    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 26 18:47:26.797883 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:26.797860    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6329ea-864b49f685-nshzt"]
Jul 26 18:47:26.831342 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:26.831308    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:47:26.888581 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:26.888496    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bfddd5e-726c-4cab-9da6-5d5ea004344e-openshift-service-ca\") pod \"maas-api-e2e-inf-a-6329ea-864b49f685-nshzt\" (UID: \"4bfddd5e-726c-4cab-9da6-5d5ea004344e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6329ea-864b49f685-nshzt"
Jul 26 18:47:26.888581 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:26.888556    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggbf6\" (UniqueName: \"kubernetes.io/projected/4bfddd5e-726c-4cab-9da6-5d5ea004344e-kube-api-access-ggbf6\") pod \"maas-api-e2e-inf-a-6329ea-864b49f685-nshzt\" (UID: \"4bfddd5e-726c-4cab-9da6-5d5ea004344e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6329ea-864b49f685-nshzt"
Jul 26 18:47:26.888817 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:26.888643    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4bfddd5e-726c-4cab-9da6-5d5ea004344e-maas-api-tls\") pod \"maas-api-e2e-inf-a-6329ea-864b49f685-nshzt\" (UID: \"4bfddd5e-726c-4cab-9da6-5d5ea004344e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6329ea-864b49f685-nshzt"
Jul 26 18:47:26.971804 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:26.971724    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:47:26.989994 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:26.989961    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggbf6\" (UniqueName: \"kubernetes.io/projected/4bfddd5e-726c-4cab-9da6-5d5ea004344e-kube-api-access-ggbf6\") pod \"maas-api-e2e-inf-a-6329ea-864b49f685-nshzt\" (UID: \"4bfddd5e-726c-4cab-9da6-5d5ea004344e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6329ea-864b49f685-nshzt"
Jul 26 18:47:26.990112 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:26.990067    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4bfddd5e-726c-4cab-9da6-5d5ea004344e-maas-api-tls\") pod \"maas-api-e2e-inf-a-6329ea-864b49f685-nshzt\" (UID: \"4bfddd5e-726c-4cab-9da6-5d5ea004344e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6329ea-864b49f685-nshzt"
Jul 26 18:47:26.990112 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:26.990096    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bfddd5e-726c-4cab-9da6-5d5ea004344e-openshift-service-ca\") pod \"maas-api-e2e-inf-a-6329ea-864b49f685-nshzt\" (UID: \"4bfddd5e-726c-4cab-9da6-5d5ea004344e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6329ea-864b49f685-nshzt"
Jul 26 18:47:26.990727 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:26.990704    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bfddd5e-726c-4cab-9da6-5d5ea004344e-openshift-service-ca\") pod \"maas-api-e2e-inf-a-6329ea-864b49f685-nshzt\" (UID: \"4bfddd5e-726c-4cab-9da6-5d5ea004344e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6329ea-864b49f685-nshzt"
Jul 26 18:47:26.992464 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:26.992446    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4bfddd5e-726c-4cab-9da6-5d5ea004344e-maas-api-tls\") pod \"maas-api-e2e-inf-a-6329ea-864b49f685-nshzt\" (UID: \"4bfddd5e-726c-4cab-9da6-5d5ea004344e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6329ea-864b49f685-nshzt"
Jul 26 18:47:26.998997 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:26.998971    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggbf6\" (UniqueName: \"kubernetes.io/projected/4bfddd5e-726c-4cab-9da6-5d5ea004344e-kube-api-access-ggbf6\") pod \"maas-api-e2e-inf-a-6329ea-864b49f685-nshzt\" (UID: \"4bfddd5e-726c-4cab-9da6-5d5ea004344e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6329ea-864b49f685-nshzt"
Jul 26 18:47:27.098728 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:27.098689    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6329ea-864b49f685-nshzt"
Jul 26 18:47:27.235172 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:27.235147    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6329ea-864b49f685-nshzt"]
Jul 26 18:47:27.237636 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:47:27.237594    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bfddd5e_726c_4cab_9da6_5d5ea004344e.slice/crio-75cbbc6d80b15d553a14a112fe65c055a7e33a6045afaee5d302c4208781e588 WatchSource:0}: Error finding container 75cbbc6d80b15d553a14a112fe65c055a7e33a6045afaee5d302c4208781e588: Status 404 returned error can't find the container with id 75cbbc6d80b15d553a14a112fe65c055a7e33a6045afaee5d302c4208781e588
Jul 26 18:47:27.295814 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:27.295783    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:47:27.402179 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:27.402091    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:47:27.648175 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:27.648141    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6329ea-864b49f685-nshzt" event={"ID":"4bfddd5e-726c-4cab-9da6-5d5ea004344e","Type":"ContainerStarted","Data":"75cbbc6d80b15d553a14a112fe65c055a7e33a6045afaee5d302c4208781e588"}
Jul 26 18:47:28.653725 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:28.653688    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6329ea-864b49f685-nshzt" event={"ID":"4bfddd5e-726c-4cab-9da6-5d5ea004344e","Type":"ContainerStarted","Data":"15073cfc6442c6d7d9b08b85e1b2961cfc94d77fbc7d80bb317931898113b727"}
Jul 26 18:47:28.654213 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:28.653797    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6329ea-864b49f685-nshzt"
Jul 26 18:47:28.673326 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:28.673263    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6329ea-864b49f685-nshzt" podStartSLOduration=2.249398206 podStartE2EDuration="2.673246688s" totalImagesPullingTime="423.848482ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:47:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:47:28.670803748 +0000 UTC m=+2134.378210808" watchObservedRunningTime="2026-07-26 18:47:28.673246688 +0000 UTC m=+2134.380653764"
Jul 26 18:47:31.012274 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:31.012243    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:47:31.580680 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:31.580634    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"]
Jul 26 18:47:31.584638 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:31.584614    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:47:31.587288 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:31.587264    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-ca4228-openshift-default-dockercfg-l7gwk\""
Jul 26 18:47:31.594762 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:31.594712    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"]
Jul 26 18:47:31.731838 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:31.731804    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fb0a49c-3580-4b5e-a134-4340ad6287ff-istiod-ca-cert\") pod \"e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw\" (UID: \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\") " pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:47:31.732059 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:31.731863    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fb0a49c-3580-4b5e-a134-4340ad6287ff-istio-podinfo\") pod \"e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw\" (UID: \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\") " pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:47:31.732059 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:31.731939    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fb0a49c-3580-4b5e-a134-4340ad6287ff-istio-envoy\") pod \"e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw\" (UID: \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\") " pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:47:31.732059 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:31.731978    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fb0a49c-3580-4b5e-a134-4340ad6287ff-credential-socket\") pod \"e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw\" (UID: \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\") " pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:47:31.732059 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:31.732017    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fb0a49c-3580-4b5e-a134-4340ad6287ff-istio-data\") pod \"e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw\" (UID: \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\") " pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:47:31.732059 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:31.732047    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fb0a49c-3580-4b5e-a134-4340ad6287ff-workload-socket\") pod \"e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw\" (UID: \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\") " pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:47:31.732059 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:31.732062    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fb0a49c-3580-4b5e-a134-4340ad6287ff-workload-certs\") pod \"e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw\" (UID: \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\") " pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:47:31.732385 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:31.732078    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czb8g\" (UniqueName: \"kubernetes.io/projected/9fb0a49c-3580-4b5e-a134-4340ad6287ff-kube-api-access-czb8g\") pod \"e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw\" (UID: \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\") " pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:47:31.732385 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:31.732095    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fb0a49c-3580-4b5e-a134-4340ad6287ff-istio-token\") pod \"e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw\" (UID: \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\") " pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:47:31.833280 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:31.833173    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fb0a49c-3580-4b5e-a134-4340ad6287ff-istio-data\") pod \"e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw\" (UID: \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\") " pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:47:31.833280 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:31.833225    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fb0a49c-3580-4b5e-a134-4340ad6287ff-workload-socket\") pod \"e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw\" (UID: \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\") " pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:47:31.833280 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:31.833244    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fb0a49c-3580-4b5e-a134-4340ad6287ff-workload-certs\") pod \"e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw\" (UID: \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\") " pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:47:31.833280 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:31.833263    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czb8g\" (UniqueName: \"kubernetes.io/projected/9fb0a49c-3580-4b5e-a134-4340ad6287ff-kube-api-access-czb8g\") pod \"e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw\" (UID: \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\") " pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:47:31.833280 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:31.833281    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fb0a49c-3580-4b5e-a134-4340ad6287ff-istio-token\") pod \"e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw\" (UID: \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\") " pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:47:31.833657 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:31.833309    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fb0a49c-3580-4b5e-a134-4340ad6287ff-istiod-ca-cert\") pod \"e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw\" (UID: \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\") " pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:47:31.833657 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:31.833403    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fb0a49c-3580-4b5e-a134-4340ad6287ff-istio-podinfo\") pod \"e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw\" (UID: \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\") " pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:47:31.833657 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:31.833516    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fb0a49c-3580-4b5e-a134-4340ad6287ff-istio-envoy\") pod \"e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw\" (UID: \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\") " pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:47:31.833657 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:31.833608    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fb0a49c-3580-4b5e-a134-4340ad6287ff-credential-socket\") pod \"e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw\" (UID: \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\") " pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:47:31.833949 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:31.833679    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fb0a49c-3580-4b5e-a134-4340ad6287ff-workload-socket\") pod \"e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw\" (UID: \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\") " pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:47:31.833949 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:31.833858    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fb0a49c-3580-4b5e-a134-4340ad6287ff-istio-data\") pod \"e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw\" (UID: \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\") " pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:47:31.833949 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:31.833905    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fb0a49c-3580-4b5e-a134-4340ad6287ff-workload-certs\") pod \"e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw\" (UID: \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\") " pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:47:31.834100 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:31.833971    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fb0a49c-3580-4b5e-a134-4340ad6287ff-credential-socket\") pod \"e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw\" (UID: \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\") " pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:47:31.834301 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:31.834274    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fb0a49c-3580-4b5e-a134-4340ad6287ff-istiod-ca-cert\") pod \"e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw\" (UID: \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\") " pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:47:31.835858 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:31.835832    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fb0a49c-3580-4b5e-a134-4340ad6287ff-istio-envoy\") pod \"e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw\" (UID: \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\") " pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:47:31.836090 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:31.836071    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fb0a49c-3580-4b5e-a134-4340ad6287ff-istio-podinfo\") pod \"e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw\" (UID: \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\") " pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:47:31.841064 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:31.841037    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fb0a49c-3580-4b5e-a134-4340ad6287ff-istio-token\") pod \"e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw\" (UID: \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\") " pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:47:31.841346 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:31.841322    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czb8g\" (UniqueName: \"kubernetes.io/projected/9fb0a49c-3580-4b5e-a134-4340ad6287ff-kube-api-access-czb8g\") pod \"e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw\" (UID: \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\") " pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:47:31.896480 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:31.896443    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:47:32.048913 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:32.048888    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"]
Jul 26 18:47:32.051684 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:47:32.051653    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fb0a49c_3580_4b5e_a134_4340ad6287ff.slice/crio-0dc09bc4dbedb69cc075e67b5a89120269ee227ee278e62c842746e933a9eb84 WatchSource:0}: Error finding container 0dc09bc4dbedb69cc075e67b5a89120269ee227ee278e62c842746e933a9eb84: Status 404 returned error can't find the container with id 0dc09bc4dbedb69cc075e67b5a89120269ee227ee278e62c842746e933a9eb84
Jul 26 18:47:32.055410 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:32.055372    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:47:32.055534 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:32.055459    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:47:32.055534 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:32.055516    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:47:32.188763 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:32.188705    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:47:32.643457 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:32.643424    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:47:32.674609 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:32.674577    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw" event={"ID":"9fb0a49c-3580-4b5e-a134-4340ad6287ff","Type":"ContainerStarted","Data":"4e33c6e233d007f10e15aa5167b407186e3ad24f576c632d49fd61edfbf75edd"}
Jul 26 18:47:32.674815 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:32.674633    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw" event={"ID":"9fb0a49c-3580-4b5e-a134-4340ad6287ff","Type":"ContainerStarted","Data":"0dc09bc4dbedb69cc075e67b5a89120269ee227ee278e62c842746e933a9eb84"}
Jul 26 18:47:32.695997 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:32.695944    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw" podStartSLOduration=1.695921477 podStartE2EDuration="1.695921477s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:47:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:47:32.693952681 +0000 UTC m=+2138.401359743" watchObservedRunningTime="2026-07-26 18:47:32.695921477 +0000 UTC m=+2138.403328536"
Jul 26 18:47:32.897011 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:32.896905    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:47:33.902077 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:33.902039    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:47:34.557715 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:34.557681    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:47:34.633581 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:34.633541    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb"]
Jul 26 18:47:34.637145 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:34.637120    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb"
Jul 26 18:47:34.639598 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:34.639573    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:47:34.639734 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:34.639609    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-ca4228-serving-cert\""
Jul 26 18:47:34.685459 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:34.684189    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb"]
Jul 26 18:47:34.690085 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:34.689732    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvsgv\" (UniqueName: \"kubernetes.io/projected/32651104-f2fe-46d5-bf4a-2549fb7735a9-kube-api-access-hvsgv\") pod \"maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb\" (UID: \"32651104-f2fe-46d5-bf4a-2549fb7735a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb"
Jul 26 18:47:34.690085 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:34.689884    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32651104-f2fe-46d5-bf4a-2549fb7735a9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb\" (UID: \"32651104-f2fe-46d5-bf4a-2549fb7735a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb"
Jul 26 18:47:34.690085 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:34.689928    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32651104-f2fe-46d5-bf4a-2549fb7735a9-maas-api-tls\") pod \"maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb\" (UID: \"32651104-f2fe-46d5-bf4a-2549fb7735a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb"
Jul 26 18:47:34.692908 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:34.692883    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6329ea-864b49f685-nshzt"
Jul 26 18:47:34.695800 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:34.695775    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:47:34.696639 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:34.696621    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:47:34.702958 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:34.702937    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:47:34.790709 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:34.790664    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32651104-f2fe-46d5-bf4a-2549fb7735a9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb\" (UID: \"32651104-f2fe-46d5-bf4a-2549fb7735a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb"
Jul 26 18:47:34.790709 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:34.790700    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32651104-f2fe-46d5-bf4a-2549fb7735a9-maas-api-tls\") pod \"maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb\" (UID: \"32651104-f2fe-46d5-bf4a-2549fb7735a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb"
Jul 26 18:47:34.790987 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:34.790831    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvsgv\" (UniqueName: \"kubernetes.io/projected/32651104-f2fe-46d5-bf4a-2549fb7735a9-kube-api-access-hvsgv\") pod \"maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb\" (UID: \"32651104-f2fe-46d5-bf4a-2549fb7735a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb"
Jul 26 18:47:34.790987 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:47:34.790853    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-ca4228-serving-cert: secret "maas-api-e2e-inf-b-ca4228-serving-cert" not found
Jul 26 18:47:34.790987 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:47:34.790959    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32651104-f2fe-46d5-bf4a-2549fb7735a9-maas-api-tls podName:32651104-f2fe-46d5-bf4a-2549fb7735a9 nodeName:}" failed. No retries permitted until 2026-07-26 18:47:35.290935949 +0000 UTC m=+2140.998343003 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/32651104-f2fe-46d5-bf4a-2549fb7735a9-maas-api-tls") pod "maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb" (UID: "32651104-f2fe-46d5-bf4a-2549fb7735a9") : secret "maas-api-e2e-inf-b-ca4228-serving-cert" not found
Jul 26 18:47:34.791466 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:34.791440    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32651104-f2fe-46d5-bf4a-2549fb7735a9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb\" (UID: \"32651104-f2fe-46d5-bf4a-2549fb7735a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb"
Jul 26 18:47:34.800389 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:34.800362    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvsgv\" (UniqueName: \"kubernetes.io/projected/32651104-f2fe-46d5-bf4a-2549fb7735a9-kube-api-access-hvsgv\") pod \"maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb\" (UID: \"32651104-f2fe-46d5-bf4a-2549fb7735a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb"
Jul 26 18:47:35.161217 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:35.161184    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:47:35.290518 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:35.290468    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:47:35.294696 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:35.294663    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32651104-f2fe-46d5-bf4a-2549fb7735a9-maas-api-tls\") pod \"maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb\" (UID: \"32651104-f2fe-46d5-bf4a-2549fb7735a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb"
Jul 26 18:47:35.294828 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:47:35.294814    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-ca4228-serving-cert: secret "maas-api-e2e-inf-b-ca4228-serving-cert" not found
Jul 26 18:47:35.294902 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:47:35.294892    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32651104-f2fe-46d5-bf4a-2549fb7735a9-maas-api-tls podName:32651104-f2fe-46d5-bf4a-2549fb7735a9 nodeName:}" failed. No retries permitted until 2026-07-26 18:47:36.294877778 +0000 UTC m=+2142.002284816 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/32651104-f2fe-46d5-bf4a-2549fb7735a9-maas-api-tls") pod "maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb" (UID: "32651104-f2fe-46d5-bf4a-2549fb7735a9") : secret "maas-api-e2e-inf-b-ca4228-serving-cert" not found
Jul 26 18:47:35.427100 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:35.427069    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:47:35.476536 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:35.476490    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ca4228-54b669ff64-8v5p7"]
Jul 26 18:47:35.480039 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:35.480015    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ca4228-54b669ff64-8v5p7"
Jul 26 18:47:35.483156 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:35.483133    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-ca4228\""
Jul 26 18:47:35.483156 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:35.483147    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-ca4228-dockercfg-74qtb\""
Jul 26 18:47:35.490484 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:35.490459    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ca4228-54b669ff64-8v5p7"]
Jul 26 18:47:35.497265 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:35.497038    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9wqh\" (UniqueName: \"kubernetes.io/projected/56b5bda7-edfd-4245-8ed9-e811b47500c1-kube-api-access-d9wqh\") pod \"payload-processing-e2e-inf-b-ca4228-54b669ff64-8v5p7\" (UID: \"56b5bda7-edfd-4245-8ed9-e811b47500c1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ca4228-54b669ff64-8v5p7"
Jul 26 18:47:35.497265 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:35.497124    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56b5bda7-edfd-4245-8ed9-e811b47500c1-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-ca4228-54b669ff64-8v5p7\" (UID: \"56b5bda7-edfd-4245-8ed9-e811b47500c1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ca4228-54b669ff64-8v5p7"
Jul 26 18:47:35.597658 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:35.597621    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9wqh\" (UniqueName: \"kubernetes.io/projected/56b5bda7-edfd-4245-8ed9-e811b47500c1-kube-api-access-d9wqh\") pod \"payload-processing-e2e-inf-b-ca4228-54b669ff64-8v5p7\" (UID: \"56b5bda7-edfd-4245-8ed9-e811b47500c1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ca4228-54b669ff64-8v5p7"
Jul 26 18:47:35.597868 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:35.597677    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56b5bda7-edfd-4245-8ed9-e811b47500c1-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-ca4228-54b669ff64-8v5p7\" (UID: \"56b5bda7-edfd-4245-8ed9-e811b47500c1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ca4228-54b669ff64-8v5p7"
Jul 26 18:47:35.598461 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:35.598440    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56b5bda7-edfd-4245-8ed9-e811b47500c1-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-ca4228-54b669ff64-8v5p7\" (UID: \"56b5bda7-edfd-4245-8ed9-e811b47500c1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ca4228-54b669ff64-8v5p7"
Jul 26 18:47:35.606263 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:35.606232    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9wqh\" (UniqueName: \"kubernetes.io/projected/56b5bda7-edfd-4245-8ed9-e811b47500c1-kube-api-access-d9wqh\") pod \"payload-processing-e2e-inf-b-ca4228-54b669ff64-8v5p7\" (UID: \"56b5bda7-edfd-4245-8ed9-e811b47500c1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ca4228-54b669ff64-8v5p7"
Jul 26 18:47:35.792475 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:35.792381    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ca4228-54b669ff64-8v5p7"
Jul 26 18:47:35.962801 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:35.952846    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ca4228-54b669ff64-8v5p7"]
Jul 26 18:47:35.973172 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:47:35.973141    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56b5bda7_edfd_4245_8ed9_e811b47500c1.slice/crio-16ba9aac5b82f3b37433e0979ce02200a451af16285383c74a90a06c5977300e WatchSource:0}: Error finding container 16ba9aac5b82f3b37433e0979ce02200a451af16285383c74a90a06c5977300e: Status 404 returned error can't find the container with id 16ba9aac5b82f3b37433e0979ce02200a451af16285383c74a90a06c5977300e
Jul 26 18:47:36.305077 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:36.305035    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32651104-f2fe-46d5-bf4a-2549fb7735a9-maas-api-tls\") pod \"maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb\" (UID: \"32651104-f2fe-46d5-bf4a-2549fb7735a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb"
Jul 26 18:47:36.307590 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:36.307564    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32651104-f2fe-46d5-bf4a-2549fb7735a9-maas-api-tls\") pod \"maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb\" (UID: \"32651104-f2fe-46d5-bf4a-2549fb7735a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb"
Jul 26 18:47:36.491278 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:36.491231    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb"
Jul 26 18:47:36.706453 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:36.706417    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ca4228-54b669ff64-8v5p7" event={"ID":"56b5bda7-edfd-4245-8ed9-e811b47500c1","Type":"ContainerStarted","Data":"ef162e2afd51d48d8c6b21190b59f9dcb78b2f381ac2a513cd77b254a7cd1509"}
Jul 26 18:47:36.706640 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:36.706473    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ca4228-54b669ff64-8v5p7" event={"ID":"56b5bda7-edfd-4245-8ed9-e811b47500c1","Type":"ContainerStarted","Data":"16ba9aac5b82f3b37433e0979ce02200a451af16285383c74a90a06c5977300e"}
Jul 26 18:47:36.722229 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:36.722181    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-ca4228-54b669ff64-8v5p7" podStartSLOduration=1.722165009 podStartE2EDuration="1.722165009s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:47:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:47:36.722064671 +0000 UTC m=+2142.429471731" watchObservedRunningTime="2026-07-26 18:47:36.722165009 +0000 UTC m=+2142.429572081"
Jul 26 18:47:36.879952 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:36.879924    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb"]
Jul 26 18:47:36.881930 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:47:36.881900    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32651104_f2fe_46d5_bf4a_2549fb7735a9.slice/crio-1c966895be8a23ef6ddb15a740f336164543af87513f1c91a8335d0cbc34fa5c WatchSource:0}: Error finding container 1c966895be8a23ef6ddb15a740f336164543af87513f1c91a8335d0cbc34fa5c: Status 404 returned error can't find the container with id 1c966895be8a23ef6ddb15a740f336164543af87513f1c91a8335d0cbc34fa5c
Jul 26 18:47:37.712227 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:37.712185    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb" event={"ID":"32651104-f2fe-46d5-bf4a-2549fb7735a9","Type":"ContainerStarted","Data":"50b2ae6fc9f30307cca8a1df2e31a85070e0c27661cd01f6f2efd7af968a7aaf"}
Jul 26 18:47:37.712650 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:37.712243    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb" event={"ID":"32651104-f2fe-46d5-bf4a-2549fb7735a9","Type":"ContainerStarted","Data":"1c966895be8a23ef6ddb15a740f336164543af87513f1c91a8335d0cbc34fa5c"}
Jul 26 18:47:37.712650 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:37.712458    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb"
Jul 26 18:47:37.712650 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:37.712491    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-ca4228-54b669ff64-8v5p7"
Jul 26 18:47:37.729820 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:37.729763    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb" podStartSLOduration=3.423723616 podStartE2EDuration="3.729729151s" totalImagesPullingTime="306.005535ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:47:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:47:37.728252264 +0000 UTC m=+2143.435659325" watchObservedRunningTime="2026-07-26 18:47:37.729729151 +0000 UTC m=+2143.437136210"
Jul 26 18:47:39.030201 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:39.030157    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:47:40.665072 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:40.665032    2576 patch_prober.go:28] interesting pod/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": read tcp 10.133.0.2:32952->10.133.0.41:15021: read: connection reset by peer" start-of-body=
Jul 26 18:47:40.665518 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:40.665108    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg" podUID="7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": read tcp 10.133.0.2:32952->10.133.0.41:15021: read: connection reset by peer"
Jul 26 18:47:40.727862 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:40.727824    2576 generic.go:356] "Generic (PLEG): container finished" podID="7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55" containerID="77f6823a9de16946ed2d5e4eb70e25db38785989f13d564cb68d7cc29cdd31b7" exitCode=0
Jul 26 18:47:40.728035 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:40.727883    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg" event={"ID":"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55","Type":"ContainerDied","Data":"77f6823a9de16946ed2d5e4eb70e25db38785989f13d564cb68d7cc29cdd31b7"}
Jul 26 18:47:40.728264 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:40.728248    2576 scope.go:117] "RemoveContainer" containerID="77f6823a9de16946ed2d5e4eb70e25db38785989f13d564cb68d7cc29cdd31b7"
Jul 26 18:47:40.729303 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:40.729271    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:47:40.729407 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:40.729360    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:47:40.729474 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:40.729406    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:47:41.734079 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:41.734046    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg" event={"ID":"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55","Type":"ContainerStarted","Data":"f477583b570c271c641b43d37fabf17c3b4c16b61b0a6e7a4ff7aaffa1f81167"}
Jul 26 18:47:42.055530 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:42.055432    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:47:42.055708 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:42.055600    2576 patch_prober.go:28] interesting pod/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Jul 26 18:47:42.055708 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:42.055667    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg" podUID="7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Jul 26 18:47:43.056376 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:43.056277    2576 patch_prober.go:28] interesting pod/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Jul 26 18:47:43.056950 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:43.056913    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg" podUID="7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Jul 26 18:47:43.724083 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:43.724050    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb"
Jul 26 18:47:44.056316 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:44.056222    2576 patch_prober.go:28] interesting pod/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Jul 26 18:47:44.056566 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:44.056541    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg" podUID="7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Jul 26 18:47:44.489806 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:44.489767    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:47:45.055807 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:45.055736    2576 patch_prober.go:28] interesting pod/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Jul 26 18:47:45.056027 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:45.055833    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg" podUID="7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Jul 26 18:47:46.059290 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:46.059254    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:47:46.059778 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:46.059598    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:47:46.060432 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:46.060413    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:47:46.764115 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:46.764077    2576 generic.go:356] "Generic (PLEG): container finished" podID="9fb0a49c-3580-4b5e-a134-4340ad6287ff" containerID="4e33c6e233d007f10e15aa5167b407186e3ad24f576c632d49fd61edfbf75edd" exitCode=0
Jul 26 18:47:46.764322 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:46.764153    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw" event={"ID":"9fb0a49c-3580-4b5e-a134-4340ad6287ff","Type":"ContainerDied","Data":"4e33c6e233d007f10e15aa5167b407186e3ad24f576c632d49fd61edfbf75edd"}
Jul 26 18:47:46.764703 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:46.764685    2576 scope.go:117] "RemoveContainer" containerID="4e33c6e233d007f10e15aa5167b407186e3ad24f576c632d49fd61edfbf75edd"
Jul 26 18:47:46.765610 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:46.765586    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:47:46.765676 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:46.765658    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:47:46.765732 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:46.765720    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:47:47.771400 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:47.771365    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw" event={"ID":"9fb0a49c-3580-4b5e-a134-4340ad6287ff","Type":"ContainerStarted","Data":"ba77c1bf68fd8ecbcb560aea5c644ba0670437771adc861c0d4e89a529422c85"}
Jul 26 18:47:47.897165 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:47.897123    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:47:47.898370 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:47.898336    2576 patch_prober.go:28] interesting pod/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Jul 26 18:47:47.898502 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:47.898388    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw" podUID="9fb0a49c-3580-4b5e-a134-4340ad6287ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Jul 26 18:47:48.720725 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:48.720689    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-ca4228-54b669ff64-8v5p7"
Jul 26 18:47:48.897066 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:48.897027    2576 patch_prober.go:28] interesting pod/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Jul 26 18:47:48.897488 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:48.897091    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw" podUID="9fb0a49c-3580-4b5e-a134-4340ad6287ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Jul 26 18:47:49.695895 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:49.695848    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:47:49.897702 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:49.897661    2576 patch_prober.go:28] interesting pod/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Jul 26 18:47:49.898097 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:49.897721    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw" podUID="9fb0a49c-3580-4b5e-a134-4340ad6287ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Jul 26 18:47:50.724951 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:50.724914    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:47:50.897202 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:50.897167    2576 patch_prober.go:28] interesting pod/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Jul 26 18:47:50.897395 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:50.897237    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw" podUID="9fb0a49c-3580-4b5e-a134-4340ad6287ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Jul 26 18:47:51.901327 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:51.901294    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:47:51.902378 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:51.902353    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:47:57.147047 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:47:57.147006    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:48:00.587278 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:48:00.587241    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:48:14.087791 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:48:14.087756    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:48:32.392224 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:48:32.392189    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:48:36.387780 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:48:36.387723    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:48:51.711130 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:48:51.711092    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:48:52.896199 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:48:52.896154    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:49:14.258759 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:14.258716    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:49:15.240101 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:15.240066    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:49:19.498613 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:19.498525    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:49:25.729032 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:25.728998    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6329ea-864b49f685-nshzt"]
Jul 26 18:49:25.729616 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:25.729248    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6329ea-864b49f685-nshzt" podUID="4bfddd5e-726c-4cab-9da6-5d5ea004344e" containerName="maas-api" containerID="cri-o://15073cfc6442c6d7d9b08b85e1b2961cfc94d77fbc7d80bb317931898113b727" gracePeriod=30
Jul 26 18:49:25.757598 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:25.757568    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:49:25.858610 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:25.858568    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:49:25.922933 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:25.922894    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:49:25.990169 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:25.990104    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6329ea-864b49f685-nshzt"
Jul 26 18:49:26.120423 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:26.120376    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ggbf6\" (UniqueName: \"kubernetes.io/projected/4bfddd5e-726c-4cab-9da6-5d5ea004344e-kube-api-access-ggbf6\") pod \"4bfddd5e-726c-4cab-9da6-5d5ea004344e\" (UID: \"4bfddd5e-726c-4cab-9da6-5d5ea004344e\") "
Jul 26 18:49:26.120423 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:26.120430    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bfddd5e-726c-4cab-9da6-5d5ea004344e-openshift-service-ca\") pod \"4bfddd5e-726c-4cab-9da6-5d5ea004344e\" (UID: \"4bfddd5e-726c-4cab-9da6-5d5ea004344e\") "
Jul 26 18:49:26.120644 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:26.120497    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4bfddd5e-726c-4cab-9da6-5d5ea004344e-maas-api-tls\") pod \"4bfddd5e-726c-4cab-9da6-5d5ea004344e\" (UID: \"4bfddd5e-726c-4cab-9da6-5d5ea004344e\") "
Jul 26 18:49:26.120887 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:26.120858    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bfddd5e-726c-4cab-9da6-5d5ea004344e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4bfddd5e-726c-4cab-9da6-5d5ea004344e" (UID: "4bfddd5e-726c-4cab-9da6-5d5ea004344e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:49:26.122607 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:26.122581    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4bfddd5e-726c-4cab-9da6-5d5ea004344e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4bfddd5e-726c-4cab-9da6-5d5ea004344e" (UID: "4bfddd5e-726c-4cab-9da6-5d5ea004344e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:49:26.122684 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:26.122640    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bfddd5e-726c-4cab-9da6-5d5ea004344e-kube-api-access-ggbf6" (OuterVolumeSpecName: "kube-api-access-ggbf6") pod "4bfddd5e-726c-4cab-9da6-5d5ea004344e" (UID: "4bfddd5e-726c-4cab-9da6-5d5ea004344e"). InnerVolumeSpecName "kube-api-access-ggbf6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:49:26.213878 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:26.213839    2576 generic.go:356] "Generic (PLEG): container finished" podID="4bfddd5e-726c-4cab-9da6-5d5ea004344e" containerID="15073cfc6442c6d7d9b08b85e1b2961cfc94d77fbc7d80bb317931898113b727" exitCode=0
Jul 26 18:49:26.214045 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:26.213889    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6329ea-864b49f685-nshzt"
Jul 26 18:49:26.214045 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:26.213927    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6329ea-864b49f685-nshzt" event={"ID":"4bfddd5e-726c-4cab-9da6-5d5ea004344e","Type":"ContainerDied","Data":"15073cfc6442c6d7d9b08b85e1b2961cfc94d77fbc7d80bb317931898113b727"}
Jul 26 18:49:26.214045 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:26.213984    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6329ea-864b49f685-nshzt" event={"ID":"4bfddd5e-726c-4cab-9da6-5d5ea004344e","Type":"ContainerDied","Data":"75cbbc6d80b15d553a14a112fe65c055a7e33a6045afaee5d302c4208781e588"}
Jul 26 18:49:26.214045 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:26.214006    2576 scope.go:117] "RemoveContainer" containerID="15073cfc6442c6d7d9b08b85e1b2961cfc94d77fbc7d80bb317931898113b727"
Jul 26 18:49:26.222038 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:26.222003    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ggbf6\" (UniqueName: \"kubernetes.io/projected/4bfddd5e-726c-4cab-9da6-5d5ea004344e-kube-api-access-ggbf6\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:49:26.222038 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:26.222031    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bfddd5e-726c-4cab-9da6-5d5ea004344e-openshift-service-ca\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:49:26.222261 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:26.222047    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4bfddd5e-726c-4cab-9da6-5d5ea004344e-maas-api-tls\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:49:26.224866 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:26.224814    2576 scope.go:117] "RemoveContainer" containerID="15073cfc6442c6d7d9b08b85e1b2961cfc94d77fbc7d80bb317931898113b727"
Jul 26 18:49:26.225121 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:49:26.225099    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15073cfc6442c6d7d9b08b85e1b2961cfc94d77fbc7d80bb317931898113b727\": container with ID starting with 15073cfc6442c6d7d9b08b85e1b2961cfc94d77fbc7d80bb317931898113b727 not found: ID does not exist" containerID="15073cfc6442c6d7d9b08b85e1b2961cfc94d77fbc7d80bb317931898113b727"
Jul 26 18:49:26.225168 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:26.225132    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15073cfc6442c6d7d9b08b85e1b2961cfc94d77fbc7d80bb317931898113b727"} err="failed to get container status \"15073cfc6442c6d7d9b08b85e1b2961cfc94d77fbc7d80bb317931898113b727\": rpc error: code = NotFound desc = could not find container \"15073cfc6442c6d7d9b08b85e1b2961cfc94d77fbc7d80bb317931898113b727\": container with ID starting with 15073cfc6442c6d7d9b08b85e1b2961cfc94d77fbc7d80bb317931898113b727 not found: ID does not exist"
Jul 26 18:49:26.237576 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:26.237553    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6329ea-864b49f685-nshzt"]
Jul 26 18:49:26.243355 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:26.243290    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6329ea-864b49f685-nshzt"]
Jul 26 18:49:26.849813 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:26.849780    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4bfddd5e-726c-4cab-9da6-5d5ea004344e" path="/var/lib/kubelet/pods/4bfddd5e-726c-4cab-9da6-5d5ea004344e/volumes"
Jul 26 18:49:30.423720 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:30.423684    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:49:30.802029 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:30.801949    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:49:34.522806 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:34.522761    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:49:36.498606 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:36.498570    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:49:36.598800 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:36.598765    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:49:36.734562 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:36.734522    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"]
Jul 26 18:49:36.734867 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:36.734833    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg" podUID="7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55" containerName="istio-proxy" containerID="cri-o://f477583b570c271c641b43d37fabf17c3b4c16b61b0a6e7a4ff7aaffa1f81167" gracePeriod=30
Jul 26 18:49:36.754804 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:36.754691    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:49:36.982392 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:36.982361    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:49:42.000308 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.000240    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:49:42.055721 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.055684    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-istiod-ca-cert\") pod \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\" (UID: \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\") "
Jul 26 18:49:42.055895 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.055762    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-istio-envoy\") pod \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\" (UID: \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\") "
Jul 26 18:49:42.055895 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.055788    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-credential-socket\") pod \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\" (UID: \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\") "
Jul 26 18:49:42.055895 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.055819    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-istio-token\") pod \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\" (UID: \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\") "
Jul 26 18:49:42.055895 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.055847    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-workload-socket\") pod \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\" (UID: \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\") "
Jul 26 18:49:42.055895 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.055884    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-workload-certs\") pod \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\" (UID: \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\") "
Jul 26 18:49:42.056150 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.055919    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prldd\" (UniqueName: \"kubernetes.io/projected/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-kube-api-access-prldd\") pod \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\" (UID: \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\") "
Jul 26 18:49:42.056150 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.055972    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-istio-podinfo\") pod \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\" (UID: \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\") "
Jul 26 18:49:42.056150 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.055989    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-istio-data\") pod \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\" (UID: \"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55\") "
Jul 26 18:49:42.056150 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.056096    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55" (UID: "7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:49:42.056364 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.056245    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55" (UID: "7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:49:42.056432 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.056406    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-istiod-ca-cert\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:49:42.056432 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.056427    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-workload-socket\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:49:42.056536 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.056417    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55" (UID: "7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:49:42.056536 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.056463    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55" (UID: "7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:49:42.056933 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.056907    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-istio-data" (OuterVolumeSpecName: "istio-data") pod "7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55" (UID: "7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:49:42.058182 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.058153    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-istio-token" (OuterVolumeSpecName: "istio-token") pod "7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55" (UID: "7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:49:42.058473 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.058455    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-kube-api-access-prldd" (OuterVolumeSpecName: "kube-api-access-prldd") pod "7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55" (UID: "7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55"). InnerVolumeSpecName "kube-api-access-prldd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:49:42.058650 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.058627    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55" (UID: "7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:49:42.058650 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.058635    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55" (UID: "7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 26 18:49:42.157771 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.157712    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-workload-certs\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:49:42.157771 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.157767    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prldd\" (UniqueName: \"kubernetes.io/projected/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-kube-api-access-prldd\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:49:42.157771 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.157778    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-istio-podinfo\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:49:42.158012 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.157787    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-istio-data\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:49:42.158012 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.157795    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-istio-envoy\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:49:42.158012 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.157803    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-credential-socket\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:49:42.158012 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.157810    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55-istio-token\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:49:42.288567 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.288471    2576 generic.go:356] "Generic (PLEG): container finished" podID="7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55" containerID="f477583b570c271c641b43d37fabf17c3b4c16b61b0a6e7a4ff7aaffa1f81167" exitCode=0
Jul 26 18:49:42.288567 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.288527    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"
Jul 26 18:49:42.288567 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.288542    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg" event={"ID":"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55","Type":"ContainerDied","Data":"f477583b570c271c641b43d37fabf17c3b4c16b61b0a6e7a4ff7aaffa1f81167"}
Jul 26 18:49:42.288893 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.288636    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg" event={"ID":"7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55","Type":"ContainerDied","Data":"ff64f454c2e8081b30e8b5aea7a0f0bb287f1595c17deaa3532c18373b47084d"}
Jul 26 18:49:42.288893 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.288667    2576 scope.go:117] "RemoveContainer" containerID="f477583b570c271c641b43d37fabf17c3b4c16b61b0a6e7a4ff7aaffa1f81167"
Jul 26 18:49:42.297568 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.297553    2576 scope.go:117] "RemoveContainer" containerID="77f6823a9de16946ed2d5e4eb70e25db38785989f13d564cb68d7cc29cdd31b7"
Jul 26 18:49:42.305517 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.305499    2576 scope.go:117] "RemoveContainer" containerID="f477583b570c271c641b43d37fabf17c3b4c16b61b0a6e7a4ff7aaffa1f81167"
Jul 26 18:49:42.305847 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:49:42.305822    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f477583b570c271c641b43d37fabf17c3b4c16b61b0a6e7a4ff7aaffa1f81167\": container with ID starting with f477583b570c271c641b43d37fabf17c3b4c16b61b0a6e7a4ff7aaffa1f81167 not found: ID does not exist" containerID="f477583b570c271c641b43d37fabf17c3b4c16b61b0a6e7a4ff7aaffa1f81167"
Jul 26 18:49:42.305914 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.305860    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f477583b570c271c641b43d37fabf17c3b4c16b61b0a6e7a4ff7aaffa1f81167"} err="failed to get container status \"f477583b570c271c641b43d37fabf17c3b4c16b61b0a6e7a4ff7aaffa1f81167\": rpc error: code = NotFound desc = could not find container \"f477583b570c271c641b43d37fabf17c3b4c16b61b0a6e7a4ff7aaffa1f81167\": container with ID starting with f477583b570c271c641b43d37fabf17c3b4c16b61b0a6e7a4ff7aaffa1f81167 not found: ID does not exist"
Jul 26 18:49:42.305914 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.305880    2576 scope.go:117] "RemoveContainer" containerID="77f6823a9de16946ed2d5e4eb70e25db38785989f13d564cb68d7cc29cdd31b7"
Jul 26 18:49:42.306120 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:49:42.306106    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77f6823a9de16946ed2d5e4eb70e25db38785989f13d564cb68d7cc29cdd31b7\": container with ID starting with 77f6823a9de16946ed2d5e4eb70e25db38785989f13d564cb68d7cc29cdd31b7 not found: ID does not exist" containerID="77f6823a9de16946ed2d5e4eb70e25db38785989f13d564cb68d7cc29cdd31b7"
Jul 26 18:49:42.306157 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.306124    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77f6823a9de16946ed2d5e4eb70e25db38785989f13d564cb68d7cc29cdd31b7"} err="failed to get container status \"77f6823a9de16946ed2d5e4eb70e25db38785989f13d564cb68d7cc29cdd31b7\": rpc error: code = NotFound desc = could not find container \"77f6823a9de16946ed2d5e4eb70e25db38785989f13d564cb68d7cc29cdd31b7\": container with ID starting with 77f6823a9de16946ed2d5e4eb70e25db38785989f13d564cb68d7cc29cdd31b7 not found: ID does not exist"
Jul 26 18:49:42.310814 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.310791    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"]
Jul 26 18:49:42.313660 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.313636    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-6329ea-openshift-default-6f6d5c5b5d-xgvkg"]
Jul 26 18:49:42.853893 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:42.853854    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55" path="/var/lib/kubelet/pods/7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55/volumes"
Jul 26 18:49:47.641201 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:47.641165    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb"]
Jul 26 18:49:47.641715 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:47.641447    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb" podUID="32651104-f2fe-46d5-bf4a-2549fb7735a9" containerName="maas-api" containerID="cri-o://50b2ae6fc9f30307cca8a1df2e31a85070e0c27661cd01f6f2efd7af968a7aaf" gracePeriod=30
Jul 26 18:49:47.668347 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:47.668313    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:49:47.694676 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:47.694642    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ca4228-54b669ff64-8v5p7"]
Jul 26 18:49:47.694938 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:47.694910    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-ca4228-54b669ff64-8v5p7" podUID="56b5bda7-edfd-4245-8ed9-e811b47500c1" containerName="payload-processing" containerID="cri-o://ef162e2afd51d48d8c6b21190b59f9dcb78b2f381ac2a513cd77b254a7cd1509" gracePeriod=30
Jul 26 18:49:47.770104 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:47.770069    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:49:47.869794 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:47.869713    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:49:47.997701 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:47.997677    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ca4228-54b669ff64-8v5p7"
Jul 26 18:49:48.100140 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:48.100114    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb"
Jul 26 18:49:48.109539 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:48.109516    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9wqh\" (UniqueName: \"kubernetes.io/projected/56b5bda7-edfd-4245-8ed9-e811b47500c1-kube-api-access-d9wqh\") pod \"56b5bda7-edfd-4245-8ed9-e811b47500c1\" (UID: \"56b5bda7-edfd-4245-8ed9-e811b47500c1\") "
Jul 26 18:49:48.109674 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:48.109575    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56b5bda7-edfd-4245-8ed9-e811b47500c1-plugins-config-volume\") pod \"56b5bda7-edfd-4245-8ed9-e811b47500c1\" (UID: \"56b5bda7-edfd-4245-8ed9-e811b47500c1\") "
Jul 26 18:49:48.109899 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:48.109878    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56b5bda7-edfd-4245-8ed9-e811b47500c1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "56b5bda7-edfd-4245-8ed9-e811b47500c1" (UID: "56b5bda7-edfd-4245-8ed9-e811b47500c1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:49:48.111548 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:48.111528    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56b5bda7-edfd-4245-8ed9-e811b47500c1-kube-api-access-d9wqh" (OuterVolumeSpecName: "kube-api-access-d9wqh") pod "56b5bda7-edfd-4245-8ed9-e811b47500c1" (UID: "56b5bda7-edfd-4245-8ed9-e811b47500c1"). InnerVolumeSpecName "kube-api-access-d9wqh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:49:48.210374 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:48.210272    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32651104-f2fe-46d5-bf4a-2549fb7735a9-openshift-service-ca\") pod \"32651104-f2fe-46d5-bf4a-2549fb7735a9\" (UID: \"32651104-f2fe-46d5-bf4a-2549fb7735a9\") "
Jul 26 18:49:48.210374 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:48.210341    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32651104-f2fe-46d5-bf4a-2549fb7735a9-maas-api-tls\") pod \"32651104-f2fe-46d5-bf4a-2549fb7735a9\" (UID: \"32651104-f2fe-46d5-bf4a-2549fb7735a9\") "
Jul 26 18:49:48.210597 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:48.210395    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hvsgv\" (UniqueName: \"kubernetes.io/projected/32651104-f2fe-46d5-bf4a-2549fb7735a9-kube-api-access-hvsgv\") pod \"32651104-f2fe-46d5-bf4a-2549fb7735a9\" (UID: \"32651104-f2fe-46d5-bf4a-2549fb7735a9\") "
Jul 26 18:49:48.210597 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:48.210529    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9wqh\" (UniqueName: \"kubernetes.io/projected/56b5bda7-edfd-4245-8ed9-e811b47500c1-kube-api-access-d9wqh\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:49:48.210597 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:48.210540    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56b5bda7-edfd-4245-8ed9-e811b47500c1-plugins-config-volume\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:49:48.210780 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:48.210730    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32651104-f2fe-46d5-bf4a-2549fb7735a9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "32651104-f2fe-46d5-bf4a-2549fb7735a9" (UID: "32651104-f2fe-46d5-bf4a-2549fb7735a9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:49:48.212488 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:48.212458    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32651104-f2fe-46d5-bf4a-2549fb7735a9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "32651104-f2fe-46d5-bf4a-2549fb7735a9" (UID: "32651104-f2fe-46d5-bf4a-2549fb7735a9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:49:48.212617 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:48.212499    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32651104-f2fe-46d5-bf4a-2549fb7735a9-kube-api-access-hvsgv" (OuterVolumeSpecName: "kube-api-access-hvsgv") pod "32651104-f2fe-46d5-bf4a-2549fb7735a9" (UID: "32651104-f2fe-46d5-bf4a-2549fb7735a9"). InnerVolumeSpecName "kube-api-access-hvsgv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:49:48.310980 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:48.310937    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvsgv\" (UniqueName: \"kubernetes.io/projected/32651104-f2fe-46d5-bf4a-2549fb7735a9-kube-api-access-hvsgv\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:49:48.310980 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:48.310972    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32651104-f2fe-46d5-bf4a-2549fb7735a9-openshift-service-ca\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:49:48.310980 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:48.310985    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32651104-f2fe-46d5-bf4a-2549fb7735a9-maas-api-tls\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:49:48.318109 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:48.318075    2576 generic.go:356] "Generic (PLEG): container finished" podID="32651104-f2fe-46d5-bf4a-2549fb7735a9" containerID="50b2ae6fc9f30307cca8a1df2e31a85070e0c27661cd01f6f2efd7af968a7aaf" exitCode=0
Jul 26 18:49:48.318259 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:48.318130    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb"
Jul 26 18:49:48.318259 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:48.318158    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb" event={"ID":"32651104-f2fe-46d5-bf4a-2549fb7735a9","Type":"ContainerDied","Data":"50b2ae6fc9f30307cca8a1df2e31a85070e0c27661cd01f6f2efd7af968a7aaf"}
Jul 26 18:49:48.318259 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:48.318244    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb" event={"ID":"32651104-f2fe-46d5-bf4a-2549fb7735a9","Type":"ContainerDied","Data":"1c966895be8a23ef6ddb15a740f336164543af87513f1c91a8335d0cbc34fa5c"}
Jul 26 18:49:48.318428 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:48.318276    2576 scope.go:117] "RemoveContainer" containerID="50b2ae6fc9f30307cca8a1df2e31a85070e0c27661cd01f6f2efd7af968a7aaf"
Jul 26 18:49:48.319566 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:48.319534    2576 generic.go:356] "Generic (PLEG): container finished" podID="56b5bda7-edfd-4245-8ed9-e811b47500c1" containerID="ef162e2afd51d48d8c6b21190b59f9dcb78b2f381ac2a513cd77b254a7cd1509" exitCode=0
Jul 26 18:49:48.319687 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:48.319591    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ca4228-54b669ff64-8v5p7" event={"ID":"56b5bda7-edfd-4245-8ed9-e811b47500c1","Type":"ContainerDied","Data":"ef162e2afd51d48d8c6b21190b59f9dcb78b2f381ac2a513cd77b254a7cd1509"}
Jul 26 18:49:48.319687 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:48.319597    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ca4228-54b669ff64-8v5p7"
Jul 26 18:49:48.319687 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:48.319621    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ca4228-54b669ff64-8v5p7" event={"ID":"56b5bda7-edfd-4245-8ed9-e811b47500c1","Type":"ContainerDied","Data":"16ba9aac5b82f3b37433e0979ce02200a451af16285383c74a90a06c5977300e"}
Jul 26 18:49:48.327350 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:48.327180    2576 scope.go:117] "RemoveContainer" containerID="50b2ae6fc9f30307cca8a1df2e31a85070e0c27661cd01f6f2efd7af968a7aaf"
Jul 26 18:49:48.327516 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:49:48.327485    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50b2ae6fc9f30307cca8a1df2e31a85070e0c27661cd01f6f2efd7af968a7aaf\": container with ID starting with 50b2ae6fc9f30307cca8a1df2e31a85070e0c27661cd01f6f2efd7af968a7aaf not found: ID does not exist" containerID="50b2ae6fc9f30307cca8a1df2e31a85070e0c27661cd01f6f2efd7af968a7aaf"
Jul 26 18:49:48.327625 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:48.327526    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50b2ae6fc9f30307cca8a1df2e31a85070e0c27661cd01f6f2efd7af968a7aaf"} err="failed to get container status \"50b2ae6fc9f30307cca8a1df2e31a85070e0c27661cd01f6f2efd7af968a7aaf\": rpc error: code = NotFound desc = could not find container \"50b2ae6fc9f30307cca8a1df2e31a85070e0c27661cd01f6f2efd7af968a7aaf\": container with ID starting with 50b2ae6fc9f30307cca8a1df2e31a85070e0c27661cd01f6f2efd7af968a7aaf not found: ID does not exist"
Jul 26 18:49:48.327625 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:48.327568    2576 scope.go:117] "RemoveContainer" containerID="ef162e2afd51d48d8c6b21190b59f9dcb78b2f381ac2a513cd77b254a7cd1509"
Jul 26 18:49:48.335443 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:48.335425    2576 scope.go:117] "RemoveContainer" containerID="ef162e2afd51d48d8c6b21190b59f9dcb78b2f381ac2a513cd77b254a7cd1509"
Jul 26 18:49:48.335693 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:49:48.335674    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef162e2afd51d48d8c6b21190b59f9dcb78b2f381ac2a513cd77b254a7cd1509\": container with ID starting with ef162e2afd51d48d8c6b21190b59f9dcb78b2f381ac2a513cd77b254a7cd1509 not found: ID does not exist" containerID="ef162e2afd51d48d8c6b21190b59f9dcb78b2f381ac2a513cd77b254a7cd1509"
Jul 26 18:49:48.335773 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:48.335700    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef162e2afd51d48d8c6b21190b59f9dcb78b2f381ac2a513cd77b254a7cd1509"} err="failed to get container status \"ef162e2afd51d48d8c6b21190b59f9dcb78b2f381ac2a513cd77b254a7cd1509\": rpc error: code = NotFound desc = could not find container \"ef162e2afd51d48d8c6b21190b59f9dcb78b2f381ac2a513cd77b254a7cd1509\": container with ID starting with ef162e2afd51d48d8c6b21190b59f9dcb78b2f381ac2a513cd77b254a7cd1509 not found: ID does not exist"
Jul 26 18:49:48.341123 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:48.341094    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb"]
Jul 26 18:49:48.343693 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:48.343670    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ca4228-6f8f68b48b-wj7rb"]
Jul 26 18:49:48.354121 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:48.354091    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ca4228-54b669ff64-8v5p7"]
Jul 26 18:49:48.359553 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:48.359527    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ca4228-54b669ff64-8v5p7"]
Jul 26 18:49:48.849549 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:48.849516    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32651104-f2fe-46d5-bf4a-2549fb7735a9" path="/var/lib/kubelet/pods/32651104-f2fe-46d5-bf4a-2549fb7735a9/volumes"
Jul 26 18:49:48.849928 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:48.849869    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56b5bda7-edfd-4245-8ed9-e811b47500c1" path="/var/lib/kubelet/pods/56b5bda7-edfd-4245-8ed9-e811b47500c1/volumes"
Jul 26 18:49:50.481503 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:50.481467    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:49:52.182090 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:52.182054    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:49:52.553789 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:52.553688    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:49:58.150037 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:58.149985    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:49:58.251264 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:58.251230    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:49:58.387410 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:58.387381    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"]
Jul 26 18:49:58.387616 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:58.387589    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw" podUID="9fb0a49c-3580-4b5e-a134-4340ad6287ff" containerName="istio-proxy" containerID="cri-o://ba77c1bf68fd8ecbcb560aea5c644ba0670437771adc861c0d4e89a529422c85" gracePeriod=30
Jul 26 18:49:58.416942 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:58.416851    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:49:58.635087 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:58.635042    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:49:58.974645 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:58.974609    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:49:59.253962 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.253847    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"]
Jul 26 18:49:59.254410 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.254314    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bfddd5e-726c-4cab-9da6-5d5ea004344e" containerName="maas-api"
Jul 26 18:49:59.254410 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.254334    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bfddd5e-726c-4cab-9da6-5d5ea004344e" containerName="maas-api"
Jul 26 18:49:59.254410 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.254348    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56b5bda7-edfd-4245-8ed9-e811b47500c1" containerName="payload-processing"
Jul 26 18:49:59.254410 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.254357    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="56b5bda7-edfd-4245-8ed9-e811b47500c1" containerName="payload-processing"
Jul 26 18:49:59.254410 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.254377    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32651104-f2fe-46d5-bf4a-2549fb7735a9" containerName="maas-api"
Jul 26 18:49:59.254410 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.254387    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="32651104-f2fe-46d5-bf4a-2549fb7735a9" containerName="maas-api"
Jul 26 18:49:59.254410 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.254396    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55" containerName="istio-proxy"
Jul 26 18:49:59.254410 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.254404    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55" containerName="istio-proxy"
Jul 26 18:49:59.254853 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.254420    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55" containerName="istio-proxy"
Jul 26 18:49:59.254853 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.254426    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55" containerName="istio-proxy"
Jul 26 18:49:59.254853 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.254496    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bfddd5e-726c-4cab-9da6-5d5ea004344e" containerName="maas-api"
Jul 26 18:49:59.254853 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.254507    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55" containerName="istio-proxy"
Jul 26 18:49:59.254853 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.254513    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="32651104-f2fe-46d5-bf4a-2549fb7735a9" containerName="maas-api"
Jul 26 18:49:59.254853 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.254519    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ad356e4-5b9a-4ec8-a803-1a2bf20d1e55" containerName="istio-proxy"
Jul 26 18:49:59.254853 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.254524    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="56b5bda7-edfd-4245-8ed9-e811b47500c1" containerName="payload-processing"
Jul 26 18:49:59.259408 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.259388    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:49:59.262428 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.262408    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-991a15-openshift-default-dockercfg-rlqwb\""
Jul 26 18:49:59.270535 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.270512    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"]
Jul 26 18:49:59.287248 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.287206    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:49:59.305801 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.305764    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fe5845c-8eb1-4971-99be-1f66ef1654c0-istiod-ca-cert\") pod \"e2e-api-a-991a15-openshift-default-6c56684744-5wpp8\" (UID: \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\") " pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:49:59.305964 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.305811    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fe5845c-8eb1-4971-99be-1f66ef1654c0-istio-envoy\") pod \"e2e-api-a-991a15-openshift-default-6c56684744-5wpp8\" (UID: \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\") " pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:49:59.305964 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.305840    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9zsr\" (UniqueName: \"kubernetes.io/projected/5fe5845c-8eb1-4971-99be-1f66ef1654c0-kube-api-access-j9zsr\") pod \"e2e-api-a-991a15-openshift-default-6c56684744-5wpp8\" (UID: \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\") " pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:49:59.305964 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.305889    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fe5845c-8eb1-4971-99be-1f66ef1654c0-workload-certs\") pod \"e2e-api-a-991a15-openshift-default-6c56684744-5wpp8\" (UID: \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\") " pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:49:59.305964 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.305958    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fe5845c-8eb1-4971-99be-1f66ef1654c0-istio-podinfo\") pod \"e2e-api-a-991a15-openshift-default-6c56684744-5wpp8\" (UID: \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\") " pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:49:59.306123 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.305992    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fe5845c-8eb1-4971-99be-1f66ef1654c0-istio-data\") pod \"e2e-api-a-991a15-openshift-default-6c56684744-5wpp8\" (UID: \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\") " pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:49:59.306123 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.306014    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fe5845c-8eb1-4971-99be-1f66ef1654c0-istio-token\") pod \"e2e-api-a-991a15-openshift-default-6c56684744-5wpp8\" (UID: \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\") " pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:49:59.306123 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.306041    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fe5845c-8eb1-4971-99be-1f66ef1654c0-workload-socket\") pod \"e2e-api-a-991a15-openshift-default-6c56684744-5wpp8\" (UID: \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\") " pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:49:59.306123 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.306095    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fe5845c-8eb1-4971-99be-1f66ef1654c0-credential-socket\") pod \"e2e-api-a-991a15-openshift-default-6c56684744-5wpp8\" (UID: \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\") " pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:49:59.407134 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.407092    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fe5845c-8eb1-4971-99be-1f66ef1654c0-istiod-ca-cert\") pod \"e2e-api-a-991a15-openshift-default-6c56684744-5wpp8\" (UID: \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\") " pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:49:59.407134 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.407138    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fe5845c-8eb1-4971-99be-1f66ef1654c0-istio-envoy\") pod \"e2e-api-a-991a15-openshift-default-6c56684744-5wpp8\" (UID: \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\") " pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:49:59.407372 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.407177    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9zsr\" (UniqueName: \"kubernetes.io/projected/5fe5845c-8eb1-4971-99be-1f66ef1654c0-kube-api-access-j9zsr\") pod \"e2e-api-a-991a15-openshift-default-6c56684744-5wpp8\" (UID: \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\") " pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:49:59.407372 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.407206    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fe5845c-8eb1-4971-99be-1f66ef1654c0-workload-certs\") pod \"e2e-api-a-991a15-openshift-default-6c56684744-5wpp8\" (UID: \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\") " pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:49:59.407372 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.407237    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fe5845c-8eb1-4971-99be-1f66ef1654c0-istio-podinfo\") pod \"e2e-api-a-991a15-openshift-default-6c56684744-5wpp8\" (UID: \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\") " pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:49:59.407372 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.407276    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fe5845c-8eb1-4971-99be-1f66ef1654c0-istio-data\") pod \"e2e-api-a-991a15-openshift-default-6c56684744-5wpp8\" (UID: \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\") " pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:49:59.407372 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.407302    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fe5845c-8eb1-4971-99be-1f66ef1654c0-istio-token\") pod \"e2e-api-a-991a15-openshift-default-6c56684744-5wpp8\" (UID: \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\") " pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:49:59.407633 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.407440    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fe5845c-8eb1-4971-99be-1f66ef1654c0-workload-socket\") pod \"e2e-api-a-991a15-openshift-default-6c56684744-5wpp8\" (UID: \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\") " pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:49:59.407633 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.407514    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fe5845c-8eb1-4971-99be-1f66ef1654c0-credential-socket\") pod \"e2e-api-a-991a15-openshift-default-6c56684744-5wpp8\" (UID: \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\") " pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:49:59.407892 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.407869    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fe5845c-8eb1-4971-99be-1f66ef1654c0-istio-data\") pod \"e2e-api-a-991a15-openshift-default-6c56684744-5wpp8\" (UID: \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\") " pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:49:59.408068 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.408044    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fe5845c-8eb1-4971-99be-1f66ef1654c0-workload-socket\") pod \"e2e-api-a-991a15-openshift-default-6c56684744-5wpp8\" (UID: \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\") " pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:49:59.408068 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.407890    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fe5845c-8eb1-4971-99be-1f66ef1654c0-credential-socket\") pod \"e2e-api-a-991a15-openshift-default-6c56684744-5wpp8\" (UID: \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\") " pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:49:59.408220 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.407972    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fe5845c-8eb1-4971-99be-1f66ef1654c0-istiod-ca-cert\") pod \"e2e-api-a-991a15-openshift-default-6c56684744-5wpp8\" (UID: \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\") " pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:49:59.408220 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.407963    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fe5845c-8eb1-4971-99be-1f66ef1654c0-workload-certs\") pod \"e2e-api-a-991a15-openshift-default-6c56684744-5wpp8\" (UID: \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\") " pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:49:59.409739 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.409718    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fe5845c-8eb1-4971-99be-1f66ef1654c0-istio-envoy\") pod \"e2e-api-a-991a15-openshift-default-6c56684744-5wpp8\" (UID: \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\") " pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:49:59.410067 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.410044    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fe5845c-8eb1-4971-99be-1f66ef1654c0-istio-podinfo\") pod \"e2e-api-a-991a15-openshift-default-6c56684744-5wpp8\" (UID: \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\") " pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:49:59.415902 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.415878    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fe5845c-8eb1-4971-99be-1f66ef1654c0-istio-token\") pod \"e2e-api-a-991a15-openshift-default-6c56684744-5wpp8\" (UID: \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\") " pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:49:59.416046 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.416025    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9zsr\" (UniqueName: \"kubernetes.io/projected/5fe5845c-8eb1-4971-99be-1f66ef1654c0-kube-api-access-j9zsr\") pod \"e2e-api-a-991a15-openshift-default-6c56684744-5wpp8\" (UID: \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\") " pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:49:59.570219 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.570116    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:49:59.710149 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.710122    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"]
Jul 26 18:49:59.712405 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:49:59.712375    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fe5845c_8eb1_4971_99be_1f66ef1654c0.slice/crio-60b59dcdbc12663e9c26fe636caa8c5bca191f1f6392cc35d12441b2927e4c28 WatchSource:0}: Error finding container 60b59dcdbc12663e9c26fe636caa8c5bca191f1f6392cc35d12441b2927e4c28: Status 404 returned error can't find the container with id 60b59dcdbc12663e9c26fe636caa8c5bca191f1f6392cc35d12441b2927e4c28
Jul 26 18:49:59.714517 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.714489    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:49:59.714593 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.714581    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:49:59.714663 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:49:59.714611    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:50:00.033564 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:00.033532    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:50:00.375621 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:00.375532    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8" event={"ID":"5fe5845c-8eb1-4971-99be-1f66ef1654c0","Type":"ContainerStarted","Data":"64bbceaef48ec525e586d52d1c6522da786612eacfa3e6c13468b807ba5e03f6"}
Jul 26 18:50:00.375621 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:00.375593    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8" event={"ID":"5fe5845c-8eb1-4971-99be-1f66ef1654c0","Type":"ContainerStarted","Data":"60b59dcdbc12663e9c26fe636caa8c5bca191f1f6392cc35d12441b2927e4c28"}
Jul 26 18:50:00.395616 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:00.395561    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8" podStartSLOduration=1.39554195 podStartE2EDuration="1.39554195s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:49:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:50:00.394805046 +0000 UTC m=+2286.102212106" watchObservedRunningTime="2026-07-26 18:50:00.39554195 +0000 UTC m=+2286.102949008"
Jul 26 18:50:00.570682 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:00.570646    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:50:00.575329 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:00.575303    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:50:01.380325 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:01.380297    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:50:01.381469 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:01.381452    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:50:02.313326 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:02.313293    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:50:02.383339 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:02.383294    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w"]
Jul 26 18:50:02.386831 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:02.386808    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w"
Jul 26 18:50:02.389425 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:02.389391    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:50:02.389774 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:02.389722    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9n9b7\""
Jul 26 18:50:02.389897 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:02.389786    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-991a15-serving-cert\""
Jul 26 18:50:02.389897 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:02.389802    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 26 18:50:02.389897 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:02.389722    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 18:50:02.396246 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:02.396223    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w"]
Jul 26 18:50:02.433980 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:02.433937    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/596fb369-0dcb-46d3-91f3-51fe45340a81-maas-api-tls\") pod \"maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w\" (UID: \"596fb369-0dcb-46d3-91f3-51fe45340a81\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w"
Jul 26 18:50:02.434198 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:02.434020    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/596fb369-0dcb-46d3-91f3-51fe45340a81-openshift-service-ca\") pod \"maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w\" (UID: \"596fb369-0dcb-46d3-91f3-51fe45340a81\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w"
Jul 26 18:50:02.434198 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:02.434064    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkdhl\" (UniqueName: \"kubernetes.io/projected/596fb369-0dcb-46d3-91f3-51fe45340a81-kube-api-access-dkdhl\") pod \"maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w\" (UID: \"596fb369-0dcb-46d3-91f3-51fe45340a81\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w"
Jul 26 18:50:02.453684 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:02.453651    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:50:02.535000 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:02.534957    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/596fb369-0dcb-46d3-91f3-51fe45340a81-maas-api-tls\") pod \"maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w\" (UID: \"596fb369-0dcb-46d3-91f3-51fe45340a81\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w"
Jul 26 18:50:02.535259 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:02.535008    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/596fb369-0dcb-46d3-91f3-51fe45340a81-openshift-service-ca\") pod \"maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w\" (UID: \"596fb369-0dcb-46d3-91f3-51fe45340a81\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w"
Jul 26 18:50:02.535259 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:02.535131    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkdhl\" (UniqueName: \"kubernetes.io/projected/596fb369-0dcb-46d3-91f3-51fe45340a81-kube-api-access-dkdhl\") pod \"maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w\" (UID: \"596fb369-0dcb-46d3-91f3-51fe45340a81\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w"
Jul 26 18:50:02.535259 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:50:02.535136    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-991a15-serving-cert: secret "maas-api-e2e-api-a-991a15-serving-cert" not found
Jul 26 18:50:02.535259 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:50:02.535241    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/596fb369-0dcb-46d3-91f3-51fe45340a81-maas-api-tls podName:596fb369-0dcb-46d3-91f3-51fe45340a81 nodeName:}" failed. No retries permitted until 2026-07-26 18:50:03.035221558 +0000 UTC m=+2288.742628595 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/596fb369-0dcb-46d3-91f3-51fe45340a81-maas-api-tls") pod "maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w" (UID: "596fb369-0dcb-46d3-91f3-51fe45340a81") : secret "maas-api-e2e-api-a-991a15-serving-cert" not found
Jul 26 18:50:02.535870 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:02.535845    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/596fb369-0dcb-46d3-91f3-51fe45340a81-openshift-service-ca\") pod \"maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w\" (UID: \"596fb369-0dcb-46d3-91f3-51fe45340a81\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w"
Jul 26 18:50:02.546143 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:02.546119    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkdhl\" (UniqueName: \"kubernetes.io/projected/596fb369-0dcb-46d3-91f3-51fe45340a81-kube-api-access-dkdhl\") pod \"maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w\" (UID: \"596fb369-0dcb-46d3-91f3-51fe45340a81\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w"
Jul 26 18:50:02.759127 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:02.759071    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:50:02.957689 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:02.957649    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:50:03.040892 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:03.040786    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/596fb369-0dcb-46d3-91f3-51fe45340a81-maas-api-tls\") pod \"maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w\" (UID: \"596fb369-0dcb-46d3-91f3-51fe45340a81\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w"
Jul 26 18:50:03.043373 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:03.043344    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/596fb369-0dcb-46d3-91f3-51fe45340a81-maas-api-tls\") pod \"maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w\" (UID: \"596fb369-0dcb-46d3-91f3-51fe45340a81\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w"
Jul 26 18:50:03.055451 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:03.055422    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:50:03.301723 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:03.301613    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w"
Jul 26 18:50:03.439558 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:03.439533    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w"]
Jul 26 18:50:03.449663 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:50:03.449629    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod596fb369_0dcb_46d3_91f3_51fe45340a81.slice/crio-f5c01597da59bb10fcf7cd9f15bced7f26e922db3530df43130b6ce316128f9e WatchSource:0}: Error finding container f5c01597da59bb10fcf7cd9f15bced7f26e922db3530df43130b6ce316128f9e: Status 404 returned error can't find the container with id f5c01597da59bb10fcf7cd9f15bced7f26e922db3530df43130b6ce316128f9e
Jul 26 18:50:03.492020 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:03.491997    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 18:50:03.647340 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:03.647311    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:50:03.746852 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:03.746815    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fb0a49c-3580-4b5e-a134-4340ad6287ff-istio-data\") pod \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\" (UID: \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\") "
Jul 26 18:50:03.746852 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:03.746858    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fb0a49c-3580-4b5e-a134-4340ad6287ff-workload-socket\") pod \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\" (UID: \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\") "
Jul 26 18:50:03.747103 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:03.746879    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fb0a49c-3580-4b5e-a134-4340ad6287ff-istio-token\") pod \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\" (UID: \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\") "
Jul 26 18:50:03.747103 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:03.746917    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fb0a49c-3580-4b5e-a134-4340ad6287ff-istiod-ca-cert\") pod \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\" (UID: \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\") "
Jul 26 18:50:03.747103 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:03.746959    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fb0a49c-3580-4b5e-a134-4340ad6287ff-workload-certs\") pod \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\" (UID: \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\") "
Jul 26 18:50:03.747103 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:03.747011    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fb0a49c-3580-4b5e-a134-4340ad6287ff-istio-podinfo\") pod \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\" (UID: \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\") "
Jul 26 18:50:03.747103 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:03.747034    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fb0a49c-3580-4b5e-a134-4340ad6287ff-istio-envoy\") pod \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\" (UID: \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\") "
Jul 26 18:50:03.747103 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:03.747079    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fb0a49c-3580-4b5e-a134-4340ad6287ff-credential-socket\") pod \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\" (UID: \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\") "
Jul 26 18:50:03.747401 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:03.747119    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-czb8g\" (UniqueName: \"kubernetes.io/projected/9fb0a49c-3580-4b5e-a134-4340ad6287ff-kube-api-access-czb8g\") pod \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\" (UID: \"9fb0a49c-3580-4b5e-a134-4340ad6287ff\") "
Jul 26 18:50:03.747401 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:03.747184    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fb0a49c-3580-4b5e-a134-4340ad6287ff-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9fb0a49c-3580-4b5e-a134-4340ad6287ff" (UID: "9fb0a49c-3580-4b5e-a134-4340ad6287ff"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:50:03.747401 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:03.747384    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fb0a49c-3580-4b5e-a134-4340ad6287ff-workload-socket\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:50:03.747574 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:03.747460    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fb0a49c-3580-4b5e-a134-4340ad6287ff-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9fb0a49c-3580-4b5e-a134-4340ad6287ff" (UID: "9fb0a49c-3580-4b5e-a134-4340ad6287ff"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:50:03.747574 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:03.747532    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fb0a49c-3580-4b5e-a134-4340ad6287ff-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9fb0a49c-3580-4b5e-a134-4340ad6287ff" (UID: "9fb0a49c-3580-4b5e-a134-4340ad6287ff"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:50:03.747779 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:03.747725    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fb0a49c-3580-4b5e-a134-4340ad6287ff-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9fb0a49c-3580-4b5e-a134-4340ad6287ff" (UID: "9fb0a49c-3580-4b5e-a134-4340ad6287ff"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:50:03.747882 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:03.747783    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fb0a49c-3580-4b5e-a134-4340ad6287ff-istio-data" (OuterVolumeSpecName: "istio-data") pod "9fb0a49c-3580-4b5e-a134-4340ad6287ff" (UID: "9fb0a49c-3580-4b5e-a134-4340ad6287ff"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:50:03.749364 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:03.749337    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fb0a49c-3580-4b5e-a134-4340ad6287ff-istio-token" (OuterVolumeSpecName: "istio-token") pod "9fb0a49c-3580-4b5e-a134-4340ad6287ff" (UID: "9fb0a49c-3580-4b5e-a134-4340ad6287ff"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:50:03.749490 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:03.749456    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fb0a49c-3580-4b5e-a134-4340ad6287ff-kube-api-access-czb8g" (OuterVolumeSpecName: "kube-api-access-czb8g") pod "9fb0a49c-3580-4b5e-a134-4340ad6287ff" (UID: "9fb0a49c-3580-4b5e-a134-4340ad6287ff"). InnerVolumeSpecName "kube-api-access-czb8g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:50:03.749548 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:03.749463    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fb0a49c-3580-4b5e-a134-4340ad6287ff-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9fb0a49c-3580-4b5e-a134-4340ad6287ff" (UID: "9fb0a49c-3580-4b5e-a134-4340ad6287ff"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:50:03.749605 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:03.749583    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9fb0a49c-3580-4b5e-a134-4340ad6287ff-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9fb0a49c-3580-4b5e-a134-4340ad6287ff" (UID: "9fb0a49c-3580-4b5e-a134-4340ad6287ff"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 26 18:50:03.848708 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:03.848629    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fb0a49c-3580-4b5e-a134-4340ad6287ff-istio-podinfo\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:50:03.848708 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:03.848658    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fb0a49c-3580-4b5e-a134-4340ad6287ff-istio-envoy\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:50:03.848708 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:03.848668    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fb0a49c-3580-4b5e-a134-4340ad6287ff-credential-socket\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:50:03.848708 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:03.848679    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czb8g\" (UniqueName: \"kubernetes.io/projected/9fb0a49c-3580-4b5e-a134-4340ad6287ff-kube-api-access-czb8g\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:50:03.848708 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:03.848688    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fb0a49c-3580-4b5e-a134-4340ad6287ff-istio-data\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:50:03.848708 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:03.848697    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fb0a49c-3580-4b5e-a134-4340ad6287ff-istio-token\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:50:03.848708 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:03.848704    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fb0a49c-3580-4b5e-a134-4340ad6287ff-istiod-ca-cert\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:50:03.848708 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:03.848712    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fb0a49c-3580-4b5e-a134-4340ad6287ff-workload-certs\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:50:04.397552 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:04.397520    2576 generic.go:356] "Generic (PLEG): container finished" podID="9fb0a49c-3580-4b5e-a134-4340ad6287ff" containerID="ba77c1bf68fd8ecbcb560aea5c644ba0670437771adc861c0d4e89a529422c85" exitCode=0
Jul 26 18:50:04.397763 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:04.397571    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"
Jul 26 18:50:04.397763 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:04.397606    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw" event={"ID":"9fb0a49c-3580-4b5e-a134-4340ad6287ff","Type":"ContainerDied","Data":"ba77c1bf68fd8ecbcb560aea5c644ba0670437771adc861c0d4e89a529422c85"}
Jul 26 18:50:04.397763 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:04.397662    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw" event={"ID":"9fb0a49c-3580-4b5e-a134-4340ad6287ff","Type":"ContainerDied","Data":"0dc09bc4dbedb69cc075e67b5a89120269ee227ee278e62c842746e933a9eb84"}
Jul 26 18:50:04.397763 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:04.397693    2576 scope.go:117] "RemoveContainer" containerID="ba77c1bf68fd8ecbcb560aea5c644ba0670437771adc861c0d4e89a529422c85"
Jul 26 18:50:04.399232 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:04.399202    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w" event={"ID":"596fb369-0dcb-46d3-91f3-51fe45340a81","Type":"ContainerStarted","Data":"a71307db5626d73fac4f9f1e569e18139a7bc6d052c9c21c9d2a7602ad108c21"}
Jul 26 18:50:04.399307 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:04.399247    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w" event={"ID":"596fb369-0dcb-46d3-91f3-51fe45340a81","Type":"ContainerStarted","Data":"f5c01597da59bb10fcf7cd9f15bced7f26e922db3530df43130b6ce316128f9e"}
Jul 26 18:50:04.399307 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:04.399278    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w"
Jul 26 18:50:04.406913 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:04.406883    2576 scope.go:117] "RemoveContainer" containerID="4e33c6e233d007f10e15aa5167b407186e3ad24f576c632d49fd61edfbf75edd"
Jul 26 18:50:04.414861 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:04.414840    2576 scope.go:117] "RemoveContainer" containerID="ba77c1bf68fd8ecbcb560aea5c644ba0670437771adc861c0d4e89a529422c85"
Jul 26 18:50:04.415144 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:50:04.415124    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba77c1bf68fd8ecbcb560aea5c644ba0670437771adc861c0d4e89a529422c85\": container with ID starting with ba77c1bf68fd8ecbcb560aea5c644ba0670437771adc861c0d4e89a529422c85 not found: ID does not exist" containerID="ba77c1bf68fd8ecbcb560aea5c644ba0670437771adc861c0d4e89a529422c85"
Jul 26 18:50:04.415188 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:04.415151    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba77c1bf68fd8ecbcb560aea5c644ba0670437771adc861c0d4e89a529422c85"} err="failed to get container status \"ba77c1bf68fd8ecbcb560aea5c644ba0670437771adc861c0d4e89a529422c85\": rpc error: code = NotFound desc = could not find container \"ba77c1bf68fd8ecbcb560aea5c644ba0670437771adc861c0d4e89a529422c85\": container with ID starting with ba77c1bf68fd8ecbcb560aea5c644ba0670437771adc861c0d4e89a529422c85 not found: ID does not exist"
Jul 26 18:50:04.415188 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:04.415170    2576 scope.go:117] "RemoveContainer" containerID="4e33c6e233d007f10e15aa5167b407186e3ad24f576c632d49fd61edfbf75edd"
Jul 26 18:50:04.415439 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:50:04.415415    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e33c6e233d007f10e15aa5167b407186e3ad24f576c632d49fd61edfbf75edd\": container with ID starting with 4e33c6e233d007f10e15aa5167b407186e3ad24f576c632d49fd61edfbf75edd not found: ID does not exist" containerID="4e33c6e233d007f10e15aa5167b407186e3ad24f576c632d49fd61edfbf75edd"
Jul 26 18:50:04.415505 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:04.415448    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e33c6e233d007f10e15aa5167b407186e3ad24f576c632d49fd61edfbf75edd"} err="failed to get container status \"4e33c6e233d007f10e15aa5167b407186e3ad24f576c632d49fd61edfbf75edd\": rpc error: code = NotFound desc = could not find container \"4e33c6e233d007f10e15aa5167b407186e3ad24f576c632d49fd61edfbf75edd\": container with ID starting with 4e33c6e233d007f10e15aa5167b407186e3ad24f576c632d49fd61edfbf75edd not found: ID does not exist"
Jul 26 18:50:04.418589 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:04.418523    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w" podStartSLOduration=1.991919351 podStartE2EDuration="2.418509819s" totalImagesPullingTime="426.590468ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:50:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:50:04.417619137 +0000 UTC m=+2290.125026196" watchObservedRunningTime="2026-07-26 18:50:04.418509819 +0000 UTC m=+2290.125916877"
Jul 26 18:50:04.430652 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:04.430622    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"]
Jul 26 18:50:04.437401 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:04.437365    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-ca4228-openshift-default-5f4c56c976-6g2jw"]
Jul 26 18:50:04.849790 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:04.849739    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fb0a49c-3580-4b5e-a134-4340ad6287ff" path="/var/lib/kubelet/pods/9fb0a49c-3580-4b5e-a134-4340ad6287ff/volumes"
Jul 26 18:50:06.763246 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:06.763213    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:50:07.193847 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.193815    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:50:07.453076 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.452983    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"]
Jul 26 18:50:07.453481 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.453456    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fb0a49c-3580-4b5e-a134-4340ad6287ff" containerName="istio-proxy"
Jul 26 18:50:07.453661 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.453639    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fb0a49c-3580-4b5e-a134-4340ad6287ff" containerName="istio-proxy"
Jul 26 18:50:07.453790 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.453690    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fb0a49c-3580-4b5e-a134-4340ad6287ff" containerName="istio-proxy"
Jul 26 18:50:07.453790 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.453701    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fb0a49c-3580-4b5e-a134-4340ad6287ff" containerName="istio-proxy"
Jul 26 18:50:07.453929 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.453826    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fb0a49c-3580-4b5e-a134-4340ad6287ff" containerName="istio-proxy"
Jul 26 18:50:07.458323 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.458304    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:50:07.461186 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.461167    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-b9a04e-openshift-default-dockercfg-jv58c\""
Jul 26 18:50:07.468316 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.468298    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"]
Jul 26 18:50:07.484074 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.484045    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:50:07.585469 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.585427    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6f9k4\" (UniqueName: \"kubernetes.io/projected/a6eda582-d5f9-453a-ae0e-77d101298268-kube-api-access-6f9k4\") pod \"e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns\" (UID: \"a6eda582-d5f9-453a-ae0e-77d101298268\") " pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:50:07.585469 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.585467    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6eda582-d5f9-453a-ae0e-77d101298268-istio-data\") pod \"e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns\" (UID: \"a6eda582-d5f9-453a-ae0e-77d101298268\") " pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:50:07.585693 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.585487    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6eda582-d5f9-453a-ae0e-77d101298268-credential-socket\") pod \"e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns\" (UID: \"a6eda582-d5f9-453a-ae0e-77d101298268\") " pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:50:07.585693 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.585576    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6eda582-d5f9-453a-ae0e-77d101298268-istio-envoy\") pod \"e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns\" (UID: \"a6eda582-d5f9-453a-ae0e-77d101298268\") " pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:50:07.585693 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.585616    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6eda582-d5f9-453a-ae0e-77d101298268-istio-podinfo\") pod \"e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns\" (UID: \"a6eda582-d5f9-453a-ae0e-77d101298268\") " pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:50:07.585693 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.585645    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6eda582-d5f9-453a-ae0e-77d101298268-istio-token\") pod \"e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns\" (UID: \"a6eda582-d5f9-453a-ae0e-77d101298268\") " pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:50:07.585693 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.585673    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6eda582-d5f9-453a-ae0e-77d101298268-istiod-ca-cert\") pod \"e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns\" (UID: \"a6eda582-d5f9-453a-ae0e-77d101298268\") " pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:50:07.585882 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.585710    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6eda582-d5f9-453a-ae0e-77d101298268-workload-socket\") pod \"e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns\" (UID: \"a6eda582-d5f9-453a-ae0e-77d101298268\") " pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:50:07.585882 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.585785    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6eda582-d5f9-453a-ae0e-77d101298268-workload-certs\") pod \"e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns\" (UID: \"a6eda582-d5f9-453a-ae0e-77d101298268\") " pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:50:07.686203 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.686162    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6eda582-d5f9-453a-ae0e-77d101298268-istio-podinfo\") pod \"e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns\" (UID: \"a6eda582-d5f9-453a-ae0e-77d101298268\") " pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:50:07.686203 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.686203    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6eda582-d5f9-453a-ae0e-77d101298268-istio-token\") pod \"e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns\" (UID: \"a6eda582-d5f9-453a-ae0e-77d101298268\") " pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:50:07.686454 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.686219    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6eda582-d5f9-453a-ae0e-77d101298268-istiod-ca-cert\") pod \"e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns\" (UID: \"a6eda582-d5f9-453a-ae0e-77d101298268\") " pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:50:07.686454 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.686237    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6eda582-d5f9-453a-ae0e-77d101298268-workload-socket\") pod \"e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns\" (UID: \"a6eda582-d5f9-453a-ae0e-77d101298268\") " pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:50:07.686454 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.686276    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6eda582-d5f9-453a-ae0e-77d101298268-workload-certs\") pod \"e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns\" (UID: \"a6eda582-d5f9-453a-ae0e-77d101298268\") " pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:50:07.686454 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.686322    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6f9k4\" (UniqueName: \"kubernetes.io/projected/a6eda582-d5f9-453a-ae0e-77d101298268-kube-api-access-6f9k4\") pod \"e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns\" (UID: \"a6eda582-d5f9-453a-ae0e-77d101298268\") " pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:50:07.686454 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.686350    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6eda582-d5f9-453a-ae0e-77d101298268-istio-data\") pod \"e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns\" (UID: \"a6eda582-d5f9-453a-ae0e-77d101298268\") " pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:50:07.686454 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.686382    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6eda582-d5f9-453a-ae0e-77d101298268-credential-socket\") pod \"e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns\" (UID: \"a6eda582-d5f9-453a-ae0e-77d101298268\") " pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:50:07.686454 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.686422    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6eda582-d5f9-453a-ae0e-77d101298268-istio-envoy\") pod \"e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns\" (UID: \"a6eda582-d5f9-453a-ae0e-77d101298268\") " pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:50:07.686801 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.686722    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6eda582-d5f9-453a-ae0e-77d101298268-workload-socket\") pod \"e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns\" (UID: \"a6eda582-d5f9-453a-ae0e-77d101298268\") " pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:50:07.686856 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.686831    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6eda582-d5f9-453a-ae0e-77d101298268-workload-certs\") pod \"e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns\" (UID: \"a6eda582-d5f9-453a-ae0e-77d101298268\") " pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:50:07.686935 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.686915    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6eda582-d5f9-453a-ae0e-77d101298268-credential-socket\") pod \"e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns\" (UID: \"a6eda582-d5f9-453a-ae0e-77d101298268\") " pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:50:07.687054 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.687029    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6eda582-d5f9-453a-ae0e-77d101298268-istiod-ca-cert\") pod \"e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns\" (UID: \"a6eda582-d5f9-453a-ae0e-77d101298268\") " pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:50:07.687131 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.687078    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6eda582-d5f9-453a-ae0e-77d101298268-istio-data\") pod \"e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns\" (UID: \"a6eda582-d5f9-453a-ae0e-77d101298268\") " pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:50:07.688887 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.688862    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6eda582-d5f9-453a-ae0e-77d101298268-istio-envoy\") pod \"e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns\" (UID: \"a6eda582-d5f9-453a-ae0e-77d101298268\") " pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:50:07.688994 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.688982    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6eda582-d5f9-453a-ae0e-77d101298268-istio-podinfo\") pod \"e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns\" (UID: \"a6eda582-d5f9-453a-ae0e-77d101298268\") " pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:50:07.696338 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.696305    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6f9k4\" (UniqueName: \"kubernetes.io/projected/a6eda582-d5f9-453a-ae0e-77d101298268-kube-api-access-6f9k4\") pod \"e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns\" (UID: \"a6eda582-d5f9-453a-ae0e-77d101298268\") " pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:50:07.699810 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.699783    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6eda582-d5f9-453a-ae0e-77d101298268-istio-token\") pod \"e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns\" (UID: \"a6eda582-d5f9-453a-ae0e-77d101298268\") " pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:50:07.771812 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.771700    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:50:07.913302 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.913261    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"]
Jul 26 18:50:07.916171 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:50:07.916135    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6eda582_d5f9_453a_ae0e_77d101298268.slice/crio-2a423bd07312a0579478129d50c7446ec275c75e534816bd52d9677d073813a7 WatchSource:0}: Error finding container 2a423bd07312a0579478129d50c7446ec275c75e534816bd52d9677d073813a7: Status 404 returned error can't find the container with id 2a423bd07312a0579478129d50c7446ec275c75e534816bd52d9677d073813a7
Jul 26 18:50:07.918412 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.918384    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:50:07.918489 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.918449    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:50:07.918489 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.918481    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:50:07.975958 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:07.975908    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:50:08.420039 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:08.420002    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns" event={"ID":"a6eda582-d5f9-453a-ae0e-77d101298268","Type":"ContainerStarted","Data":"877f35d23c0bc9563bc63b02631fe42bb1f02098e98d48ff79ce67fdd1f7fd59"}
Jul 26 18:50:08.420223 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:08.420084    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns" event={"ID":"a6eda582-d5f9-453a-ae0e-77d101298268","Type":"ContainerStarted","Data":"2a423bd07312a0579478129d50c7446ec275c75e534816bd52d9677d073813a7"}
Jul 26 18:50:08.443012 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:08.442947    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns" podStartSLOduration=1.442925725 podStartE2EDuration="1.442925725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:50:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:50:08.441378525 +0000 UTC m=+2294.148785577" watchObservedRunningTime="2026-07-26 18:50:08.442925725 +0000 UTC m=+2294.150332784"
Jul 26 18:50:08.772297 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:08.772253    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:50:08.777094 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:08.777065    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:50:09.424660 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:09.424621    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:50:09.425683 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:09.425662    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:50:09.593394 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:09.593359    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:50:10.339468 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:10.339435    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:50:10.409452 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:10.409425    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w"
Jul 26 18:50:10.434376 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:10.434345    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:50:10.510241 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:10.510207    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz"]
Jul 26 18:50:10.511303 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:10.511276    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fb0a49c-3580-4b5e-a134-4340ad6287ff" containerName="istio-proxy"
Jul 26 18:50:10.514781 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:10.514758    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz"
Jul 26 18:50:10.517859 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:10.517839    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-b9a04e-serving-cert\""
Jul 26 18:50:10.529989 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:10.529963    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz"]
Jul 26 18:50:10.551921 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:10.551877    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:50:10.614506 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:10.613687    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lz7cn\" (UniqueName: \"kubernetes.io/projected/ae9fa5eb-09a9-49b9-b957-e3041741c11c-kube-api-access-lz7cn\") pod \"maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz\" (UID: \"ae9fa5eb-09a9-49b9-b957-e3041741c11c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz"
Jul 26 18:50:10.614506 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:10.613777    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae9fa5eb-09a9-49b9-b957-e3041741c11c-maas-api-tls\") pod \"maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz\" (UID: \"ae9fa5eb-09a9-49b9-b957-e3041741c11c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz"
Jul 26 18:50:10.614506 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:10.613827    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae9fa5eb-09a9-49b9-b957-e3041741c11c-openshift-service-ca\") pod \"maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz\" (UID: \"ae9fa5eb-09a9-49b9-b957-e3041741c11c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz"
Jul 26 18:50:10.714936 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:10.714901    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz7cn\" (UniqueName: \"kubernetes.io/projected/ae9fa5eb-09a9-49b9-b957-e3041741c11c-kube-api-access-lz7cn\") pod \"maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz\" (UID: \"ae9fa5eb-09a9-49b9-b957-e3041741c11c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz"
Jul 26 18:50:10.715133 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:10.714947    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae9fa5eb-09a9-49b9-b957-e3041741c11c-maas-api-tls\") pod \"maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz\" (UID: \"ae9fa5eb-09a9-49b9-b957-e3041741c11c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz"
Jul 26 18:50:10.715133 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:50:10.715043    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-b9a04e-serving-cert: secret "maas-api-e2e-api-b-b9a04e-serving-cert" not found
Jul 26 18:50:10.715133 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:10.715096    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae9fa5eb-09a9-49b9-b957-e3041741c11c-openshift-service-ca\") pod \"maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz\" (UID: \"ae9fa5eb-09a9-49b9-b957-e3041741c11c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz"
Jul 26 18:50:10.715279 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:50:10.715194    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae9fa5eb-09a9-49b9-b957-e3041741c11c-maas-api-tls podName:ae9fa5eb-09a9-49b9-b957-e3041741c11c nodeName:}" failed. No retries permitted until 2026-07-26 18:50:11.215167887 +0000 UTC m=+2296.922574925 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ae9fa5eb-09a9-49b9-b957-e3041741c11c-maas-api-tls") pod "maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz" (UID: "ae9fa5eb-09a9-49b9-b957-e3041741c11c") : secret "maas-api-e2e-api-b-b9a04e-serving-cert" not found
Jul 26 18:50:10.715690 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:10.715670    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae9fa5eb-09a9-49b9-b957-e3041741c11c-openshift-service-ca\") pod \"maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz\" (UID: \"ae9fa5eb-09a9-49b9-b957-e3041741c11c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz"
Jul 26 18:50:10.724114 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:10.724086    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lz7cn\" (UniqueName: \"kubernetes.io/projected/ae9fa5eb-09a9-49b9-b957-e3041741c11c-kube-api-access-lz7cn\") pod \"maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz\" (UID: \"ae9fa5eb-09a9-49b9-b957-e3041741c11c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz"
Jul 26 18:50:10.912340 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:10.912246    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:50:11.138351 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:11.138318    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:50:11.218675 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:11.218636    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae9fa5eb-09a9-49b9-b957-e3041741c11c-maas-api-tls\") pod \"maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz\" (UID: \"ae9fa5eb-09a9-49b9-b957-e3041741c11c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz"
Jul 26 18:50:11.221231 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:11.221204    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae9fa5eb-09a9-49b9-b957-e3041741c11c-maas-api-tls\") pod \"maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz\" (UID: \"ae9fa5eb-09a9-49b9-b957-e3041741c11c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz"
Jul 26 18:50:11.241619 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:11.241576    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:50:11.284965 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:11.284930    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b9a04e-5fc8cc88-2zh6g"]
Jul 26 18:50:11.288379 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:11.288362    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-b9a04e-5fc8cc88-2zh6g"
Jul 26 18:50:11.291197 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:11.291175    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-b9a04e-dockercfg-h2bjp\""
Jul 26 18:50:11.291328 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:11.291175    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-b9a04e\""
Jul 26 18:50:11.297186 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:11.297063    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b9a04e-5fc8cc88-2zh6g"]
Jul 26 18:50:11.420928 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:11.420865    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/855ecb42-48c1-49e9-9360-cd34d00c05d9-plugins-config-volume\") pod \"payload-processing-e2e-api-b-b9a04e-5fc8cc88-2zh6g\" (UID: \"855ecb42-48c1-49e9-9360-cd34d00c05d9\") " pod="openshift-ingress/payload-processing-e2e-api-b-b9a04e-5fc8cc88-2zh6g"
Jul 26 18:50:11.421316 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:11.420976    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tl87m\" (UniqueName: \"kubernetes.io/projected/855ecb42-48c1-49e9-9360-cd34d00c05d9-kube-api-access-tl87m\") pod \"payload-processing-e2e-api-b-b9a04e-5fc8cc88-2zh6g\" (UID: \"855ecb42-48c1-49e9-9360-cd34d00c05d9\") " pod="openshift-ingress/payload-processing-e2e-api-b-b9a04e-5fc8cc88-2zh6g"
Jul 26 18:50:11.429786 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:11.429741    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz"
Jul 26 18:50:11.522519 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:11.522478    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/855ecb42-48c1-49e9-9360-cd34d00c05d9-plugins-config-volume\") pod \"payload-processing-e2e-api-b-b9a04e-5fc8cc88-2zh6g\" (UID: \"855ecb42-48c1-49e9-9360-cd34d00c05d9\") " pod="openshift-ingress/payload-processing-e2e-api-b-b9a04e-5fc8cc88-2zh6g"
Jul 26 18:50:11.522726 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:11.522553    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tl87m\" (UniqueName: \"kubernetes.io/projected/855ecb42-48c1-49e9-9360-cd34d00c05d9-kube-api-access-tl87m\") pod \"payload-processing-e2e-api-b-b9a04e-5fc8cc88-2zh6g\" (UID: \"855ecb42-48c1-49e9-9360-cd34d00c05d9\") " pod="openshift-ingress/payload-processing-e2e-api-b-b9a04e-5fc8cc88-2zh6g"
Jul 26 18:50:11.523464 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:11.523436    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/855ecb42-48c1-49e9-9360-cd34d00c05d9-plugins-config-volume\") pod \"payload-processing-e2e-api-b-b9a04e-5fc8cc88-2zh6g\" (UID: \"855ecb42-48c1-49e9-9360-cd34d00c05d9\") " pod="openshift-ingress/payload-processing-e2e-api-b-b9a04e-5fc8cc88-2zh6g"
Jul 26 18:50:11.530890 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:11.530867    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tl87m\" (UniqueName: \"kubernetes.io/projected/855ecb42-48c1-49e9-9360-cd34d00c05d9-kube-api-access-tl87m\") pod \"payload-processing-e2e-api-b-b9a04e-5fc8cc88-2zh6g\" (UID: \"855ecb42-48c1-49e9-9360-cd34d00c05d9\") " pod="openshift-ingress/payload-processing-e2e-api-b-b9a04e-5fc8cc88-2zh6g"
Jul 26 18:50:11.600398 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:11.600363    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-b9a04e-5fc8cc88-2zh6g"
Jul 26 18:50:11.627687 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:11.623840    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz"]
Jul 26 18:50:11.633849 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:50:11.633818    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae9fa5eb_09a9_49b9_b957_e3041741c11c.slice/crio-bf4d80f6dd4eb00952b1d2b5b34df2bc517ccb650af9f95208c9cb8feb68980f WatchSource:0}: Error finding container bf4d80f6dd4eb00952b1d2b5b34df2bc517ccb650af9f95208c9cb8feb68980f: Status 404 returned error can't find the container with id bf4d80f6dd4eb00952b1d2b5b34df2bc517ccb650af9f95208c9cb8feb68980f
Jul 26 18:50:11.787807 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:11.787779    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b9a04e-5fc8cc88-2zh6g"]
Jul 26 18:50:11.790458 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:50:11.790431    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod855ecb42_48c1_49e9_9360_cd34d00c05d9.slice/crio-459e7827dbef96ec333d081e0f2e70bc8a88a940b97b20cdda864985eb504fb2 WatchSource:0}: Error finding container 459e7827dbef96ec333d081e0f2e70bc8a88a940b97b20cdda864985eb504fb2: Status 404 returned error can't find the container with id 459e7827dbef96ec333d081e0f2e70bc8a88a940b97b20cdda864985eb504fb2
Jul 26 18:50:12.440166 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:12.440126    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz" event={"ID":"ae9fa5eb-09a9-49b9-b957-e3041741c11c","Type":"ContainerStarted","Data":"e35516bd54be96a5b97d5278b52441035f32cc53d88713c954a32f4dc0919de5"}
Jul 26 18:50:12.440567 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:12.440187    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz" event={"ID":"ae9fa5eb-09a9-49b9-b957-e3041741c11c","Type":"ContainerStarted","Data":"bf4d80f6dd4eb00952b1d2b5b34df2bc517ccb650af9f95208c9cb8feb68980f"}
Jul 26 18:50:12.440567 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:12.440231    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz"
Jul 26 18:50:12.442173 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:12.442134    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-b9a04e-5fc8cc88-2zh6g" event={"ID":"855ecb42-48c1-49e9-9360-cd34d00c05d9","Type":"ContainerStarted","Data":"52e6be9adad57fc8690f9c5419157854f2705eaf0e044e23fa1c5bc0b8b5e922"}
Jul 26 18:50:12.442334 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:12.442200    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-b9a04e-5fc8cc88-2zh6g" event={"ID":"855ecb42-48c1-49e9-9360-cd34d00c05d9","Type":"ContainerStarted","Data":"459e7827dbef96ec333d081e0f2e70bc8a88a940b97b20cdda864985eb504fb2"}
Jul 26 18:50:12.442779 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:12.442729    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-b9a04e-5fc8cc88-2zh6g"
Jul 26 18:50:12.462102 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:12.462041    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz" podStartSLOduration=2.108700876 podStartE2EDuration="2.462015387s" totalImagesPullingTime="353.314511ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:50:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:50:12.45989504 +0000 UTC m=+2298.167302102" watchObservedRunningTime="2026-07-26 18:50:12.462015387 +0000 UTC m=+2298.169422449"
Jul 26 18:50:12.476100 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:12.476033    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-b9a04e-5fc8cc88-2zh6g" podStartSLOduration=1.476013779 podStartE2EDuration="1.476013779s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:50:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:50:12.474300508 +0000 UTC m=+2298.181707556" watchObservedRunningTime="2026-07-26 18:50:12.476013779 +0000 UTC m=+2298.183420838"
Jul 26 18:50:14.789905 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:14.789875    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:50:17.421937 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:17.421897    2576 patch_prober.go:28] interesting pod/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 26 18:50:17.422343 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:17.421953    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8" podUID="5fe5845c-8eb1-4971-99be-1f66ef1654c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 26 18:50:18.455155 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:18.455128    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz"
Jul 26 18:50:18.476508 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:18.476472    2576 generic.go:356] "Generic (PLEG): container finished" podID="5fe5845c-8eb1-4971-99be-1f66ef1654c0" containerID="64bbceaef48ec525e586d52d1c6522da786612eacfa3e6c13468b807ba5e03f6" exitCode=0
Jul 26 18:50:18.476678 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:18.476543    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8" event={"ID":"5fe5845c-8eb1-4971-99be-1f66ef1654c0","Type":"ContainerDied","Data":"64bbceaef48ec525e586d52d1c6522da786612eacfa3e6c13468b807ba5e03f6"}
Jul 26 18:50:18.476972 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:18.476958    2576 scope.go:117] "RemoveContainer" containerID="64bbceaef48ec525e586d52d1c6522da786612eacfa3e6c13468b807ba5e03f6"
Jul 26 18:50:18.477973 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:18.477951    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:50:18.478035 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:18.478009    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:50:18.478071 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:18.478039    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:50:19.485909 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:19.485861    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8" event={"ID":"5fe5845c-8eb1-4971-99be-1f66ef1654c0","Type":"ContainerStarted","Data":"0ef0b58769505f59239458f2c3186932882bdf1b430e2b90eba975a0c10fd3da"}
Jul 26 18:50:19.570511 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:19.570464    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:50:19.570922 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:19.570886    2576 patch_prober.go:28] interesting pod/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Jul 26 18:50:19.571033 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:19.571009    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8" podUID="5fe5845c-8eb1-4971-99be-1f66ef1654c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Jul 26 18:50:20.571626 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:20.571579    2576 patch_prober.go:28] interesting pod/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Jul 26 18:50:20.572126 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:20.571663    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8" podUID="5fe5845c-8eb1-4971-99be-1f66ef1654c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Jul 26 18:50:21.571255 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:21.571205    2576 patch_prober.go:28] interesting pod/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Jul 26 18:50:21.571477 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:21.571292    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8" podUID="5fe5845c-8eb1-4971-99be-1f66ef1654c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Jul 26 18:50:22.571043 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:22.571002    2576 patch_prober.go:28] interesting pod/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Jul 26 18:50:22.572194 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:22.572162    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8" podUID="5fe5845c-8eb1-4971-99be-1f66ef1654c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Jul 26 18:50:23.570824 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:23.570702    2576 patch_prober.go:28] interesting pod/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Jul 26 18:50:23.570824 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:23.570789    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8" podUID="5fe5845c-8eb1-4971-99be-1f66ef1654c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Jul 26 18:50:24.458681 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:24.458642    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-b9a04e-5fc8cc88-2zh6g"
Jul 26 18:50:24.576366 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:24.576332    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:50:24.576763 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:24.576716    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:50:24.577425 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:24.577409    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:50:25.467925 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:25.467882    2576 patch_prober.go:28] interesting pod/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 26 18:50:25.468332 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:25.467956    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns" podUID="a6eda582-d5f9-453a-ae0e-77d101298268" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 26 18:50:25.493156 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:25.493114    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:50:26.521883 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:26.521852    2576 generic.go:356] "Generic (PLEG): container finished" podID="a6eda582-d5f9-453a-ae0e-77d101298268" containerID="877f35d23c0bc9563bc63b02631fe42bb1f02098e98d48ff79ce67fdd1f7fd59" exitCode=0
Jul 26 18:50:26.522300 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:26.521927    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns" event={"ID":"a6eda582-d5f9-453a-ae0e-77d101298268","Type":"ContainerDied","Data":"877f35d23c0bc9563bc63b02631fe42bb1f02098e98d48ff79ce67fdd1f7fd59"}
Jul 26 18:50:26.522456 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:26.522438    2576 scope.go:117] "RemoveContainer" containerID="877f35d23c0bc9563bc63b02631fe42bb1f02098e98d48ff79ce67fdd1f7fd59"
Jul 26 18:50:26.523402 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:26.523377    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:50:26.523513 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:26.523440    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:50:26.523513 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:26.523469    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:50:27.527597 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:27.527557    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns" event={"ID":"a6eda582-d5f9-453a-ae0e-77d101298268","Type":"ContainerStarted","Data":"8d64babb556b6a31b0da938c3f2faa603ab3dafb769303a8ae6f46535e59c822"}
Jul 26 18:50:27.772572 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:27.772533    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:50:27.772846 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:27.772821    2576 patch_prober.go:28] interesting pod/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 26 18:50:27.772936 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:27.772883    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns" podUID="a6eda582-d5f9-453a-ae0e-77d101298268" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 26 18:50:28.772389 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:28.772346    2576 patch_prober.go:28] interesting pod/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 26 18:50:28.772937 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:28.772415    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns" podUID="a6eda582-d5f9-453a-ae0e-77d101298268" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 26 18:50:29.772446 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:29.772406    2576 patch_prober.go:28] interesting pod/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 26 18:50:29.772890 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:29.772470    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns" podUID="a6eda582-d5f9-453a-ae0e-77d101298268" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 26 18:50:30.773121 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:30.773077    2576 patch_prober.go:28] interesting pod/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 26 18:50:30.773609 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:30.773143    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns" podUID="a6eda582-d5f9-453a-ae0e-77d101298268" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 26 18:50:31.776057 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:31.776026    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:50:31.776453 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:31.776278    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:50:31.776936 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:31.776916    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:50:37.919271 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:37.919237    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w"]
Jul 26 18:50:37.919871 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:37.919464    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w" podUID="596fb369-0dcb-46d3-91f3-51fe45340a81" containerName="maas-api" containerID="cri-o://a71307db5626d73fac4f9f1e569e18139a7bc6d052c9c21c9d2a7602ad108c21" gracePeriod=30
Jul 26 18:50:37.952220 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:37.952190    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:50:38.052528 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:38.052498    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:50:38.154049 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:38.154016    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:50:38.176710 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:38.176684    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w"
Jul 26 18:50:38.274991 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:38.274959    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/596fb369-0dcb-46d3-91f3-51fe45340a81-maas-api-tls\") pod \"596fb369-0dcb-46d3-91f3-51fe45340a81\" (UID: \"596fb369-0dcb-46d3-91f3-51fe45340a81\") "
Jul 26 18:50:38.275188 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:38.275046    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dkdhl\" (UniqueName: \"kubernetes.io/projected/596fb369-0dcb-46d3-91f3-51fe45340a81-kube-api-access-dkdhl\") pod \"596fb369-0dcb-46d3-91f3-51fe45340a81\" (UID: \"596fb369-0dcb-46d3-91f3-51fe45340a81\") "
Jul 26 18:50:38.275188 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:38.275068    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/596fb369-0dcb-46d3-91f3-51fe45340a81-openshift-service-ca\") pod \"596fb369-0dcb-46d3-91f3-51fe45340a81\" (UID: \"596fb369-0dcb-46d3-91f3-51fe45340a81\") "
Jul 26 18:50:38.275378 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:38.275355    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/596fb369-0dcb-46d3-91f3-51fe45340a81-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "596fb369-0dcb-46d3-91f3-51fe45340a81" (UID: "596fb369-0dcb-46d3-91f3-51fe45340a81"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:50:38.277349 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:38.277315    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/596fb369-0dcb-46d3-91f3-51fe45340a81-kube-api-access-dkdhl" (OuterVolumeSpecName: "kube-api-access-dkdhl") pod "596fb369-0dcb-46d3-91f3-51fe45340a81" (UID: "596fb369-0dcb-46d3-91f3-51fe45340a81"). InnerVolumeSpecName "kube-api-access-dkdhl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:50:38.277349 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:38.277323    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/596fb369-0dcb-46d3-91f3-51fe45340a81-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "596fb369-0dcb-46d3-91f3-51fe45340a81" (UID: "596fb369-0dcb-46d3-91f3-51fe45340a81"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:50:38.376490 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:38.376449    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/596fb369-0dcb-46d3-91f3-51fe45340a81-maas-api-tls\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:50:38.376490 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:38.376480    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dkdhl\" (UniqueName: \"kubernetes.io/projected/596fb369-0dcb-46d3-91f3-51fe45340a81-kube-api-access-dkdhl\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:50:38.376490 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:38.376490    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/596fb369-0dcb-46d3-91f3-51fe45340a81-openshift-service-ca\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:50:38.583175 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:38.583081    2576 generic.go:356] "Generic (PLEG): container finished" podID="596fb369-0dcb-46d3-91f3-51fe45340a81" containerID="a71307db5626d73fac4f9f1e569e18139a7bc6d052c9c21c9d2a7602ad108c21" exitCode=0
Jul 26 18:50:38.583175 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:38.583129    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w"
Jul 26 18:50:38.583175 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:38.583138    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w" event={"ID":"596fb369-0dcb-46d3-91f3-51fe45340a81","Type":"ContainerDied","Data":"a71307db5626d73fac4f9f1e569e18139a7bc6d052c9c21c9d2a7602ad108c21"}
Jul 26 18:50:38.583408 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:38.583179    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w" event={"ID":"596fb369-0dcb-46d3-91f3-51fe45340a81","Type":"ContainerDied","Data":"f5c01597da59bb10fcf7cd9f15bced7f26e922db3530df43130b6ce316128f9e"}
Jul 26 18:50:38.583878 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:38.583200    2576 scope.go:117] "RemoveContainer" containerID="a71307db5626d73fac4f9f1e569e18139a7bc6d052c9c21c9d2a7602ad108c21"
Jul 26 18:50:38.593216 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:38.593198    2576 scope.go:117] "RemoveContainer" containerID="a71307db5626d73fac4f9f1e569e18139a7bc6d052c9c21c9d2a7602ad108c21"
Jul 26 18:50:38.593470 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:50:38.593449    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a71307db5626d73fac4f9f1e569e18139a7bc6d052c9c21c9d2a7602ad108c21\": container with ID starting with a71307db5626d73fac4f9f1e569e18139a7bc6d052c9c21c9d2a7602ad108c21 not found: ID does not exist" containerID="a71307db5626d73fac4f9f1e569e18139a7bc6d052c9c21c9d2a7602ad108c21"
Jul 26 18:50:38.593535 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:38.593478    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a71307db5626d73fac4f9f1e569e18139a7bc6d052c9c21c9d2a7602ad108c21"} err="failed to get container status \"a71307db5626d73fac4f9f1e569e18139a7bc6d052c9c21c9d2a7602ad108c21\": rpc error: code = NotFound desc = could not find container \"a71307db5626d73fac4f9f1e569e18139a7bc6d052c9c21c9d2a7602ad108c21\": container with ID starting with a71307db5626d73fac4f9f1e569e18139a7bc6d052c9c21c9d2a7602ad108c21 not found: ID does not exist"
Jul 26 18:50:38.604735 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:38.604710    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w"]
Jul 26 18:50:38.607940 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:38.607917    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-991a15-6ccdfccf8b-d5x2w"]
Jul 26 18:50:38.849208 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:38.849127    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="596fb369-0dcb-46d3-91f3-51fe45340a81" path="/var/lib/kubelet/pods/596fb369-0dcb-46d3-91f3-51fe45340a81/volumes"
Jul 26 18:50:41.485645 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:41.485595    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:50:42.715463 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:42.715426    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:50:43.075580 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:43.075505    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:50:48.701452 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:48.701379    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:50:48.803034 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:48.802995    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:50:48.939038 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:48.938999    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"]
Jul 26 18:50:48.939414 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:48.939386    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8" podUID="5fe5845c-8eb1-4971-99be-1f66ef1654c0" containerName="istio-proxy" containerID="cri-o://0ef0b58769505f59239458f2c3186932882bdf1b430e2b90eba975a0c10fd3da" gracePeriod=30
Jul 26 18:50:48.966964 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:48.966886    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:50:49.192719 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:49.192682    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:50:54.191413 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.191385    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:50:54.211700 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.211604    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fe5845c-8eb1-4971-99be-1f66ef1654c0-istio-podinfo\") pod \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\" (UID: \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\") "
Jul 26 18:50:54.211700 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.211638    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fe5845c-8eb1-4971-99be-1f66ef1654c0-workload-socket\") pod \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\" (UID: \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\") "
Jul 26 18:50:54.211700 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.211695    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fe5845c-8eb1-4971-99be-1f66ef1654c0-istio-data\") pod \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\" (UID: \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\") "
Jul 26 18:50:54.212586 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.211734    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fe5845c-8eb1-4971-99be-1f66ef1654c0-workload-certs\") pod \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\" (UID: \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\") "
Jul 26 18:50:54.212586 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.211779    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fe5845c-8eb1-4971-99be-1f66ef1654c0-istio-envoy\") pod \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\" (UID: \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\") "
Jul 26 18:50:54.212586 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.211819    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fe5845c-8eb1-4971-99be-1f66ef1654c0-credential-socket\") pod \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\" (UID: \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\") "
Jul 26 18:50:54.212586 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.211844    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j9zsr\" (UniqueName: \"kubernetes.io/projected/5fe5845c-8eb1-4971-99be-1f66ef1654c0-kube-api-access-j9zsr\") pod \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\" (UID: \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\") "
Jul 26 18:50:54.212586 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.211868    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fe5845c-8eb1-4971-99be-1f66ef1654c0-istio-token\") pod \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\" (UID: \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\") "
Jul 26 18:50:54.212586 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.211893    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fe5845c-8eb1-4971-99be-1f66ef1654c0-istiod-ca-cert\") pod \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\" (UID: \"5fe5845c-8eb1-4971-99be-1f66ef1654c0\") "
Jul 26 18:50:54.212586 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.212058    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fe5845c-8eb1-4971-99be-1f66ef1654c0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5fe5845c-8eb1-4971-99be-1f66ef1654c0" (UID: "5fe5845c-8eb1-4971-99be-1f66ef1654c0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:50:54.213050 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.213017    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fe5845c-8eb1-4971-99be-1f66ef1654c0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5fe5845c-8eb1-4971-99be-1f66ef1654c0" (UID: "5fe5845c-8eb1-4971-99be-1f66ef1654c0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:50:54.213480 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.213310    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fe5845c-8eb1-4971-99be-1f66ef1654c0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5fe5845c-8eb1-4971-99be-1f66ef1654c0" (UID: "5fe5845c-8eb1-4971-99be-1f66ef1654c0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:50:54.213480 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.213358    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fe5845c-8eb1-4971-99be-1f66ef1654c0-workload-socket\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:50:54.214471 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.214367    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5fe5845c-8eb1-4971-99be-1f66ef1654c0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5fe5845c-8eb1-4971-99be-1f66ef1654c0" (UID: "5fe5845c-8eb1-4971-99be-1f66ef1654c0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:50:54.214574 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.214500    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fe5845c-8eb1-4971-99be-1f66ef1654c0-istio-data" (OuterVolumeSpecName: "istio-data") pod "5fe5845c-8eb1-4971-99be-1f66ef1654c0" (UID: "5fe5845c-8eb1-4971-99be-1f66ef1654c0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:50:54.215645 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.215614    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fe5845c-8eb1-4971-99be-1f66ef1654c0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5fe5845c-8eb1-4971-99be-1f66ef1654c0" (UID: "5fe5845c-8eb1-4971-99be-1f66ef1654c0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:50:54.224793 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.224720    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5fe5845c-8eb1-4971-99be-1f66ef1654c0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5fe5845c-8eb1-4971-99be-1f66ef1654c0" (UID: "5fe5845c-8eb1-4971-99be-1f66ef1654c0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 26 18:50:54.225015 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.224987    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fe5845c-8eb1-4971-99be-1f66ef1654c0-istio-token" (OuterVolumeSpecName: "istio-token") pod "5fe5845c-8eb1-4971-99be-1f66ef1654c0" (UID: "5fe5845c-8eb1-4971-99be-1f66ef1654c0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:50:54.225015 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.225007    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fe5845c-8eb1-4971-99be-1f66ef1654c0-kube-api-access-j9zsr" (OuterVolumeSpecName: "kube-api-access-j9zsr") pod "5fe5845c-8eb1-4971-99be-1f66ef1654c0" (UID: "5fe5845c-8eb1-4971-99be-1f66ef1654c0"). InnerVolumeSpecName "kube-api-access-j9zsr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:50:54.314234 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.314191    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fe5845c-8eb1-4971-99be-1f66ef1654c0-workload-certs\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:50:54.314234 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.314228    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fe5845c-8eb1-4971-99be-1f66ef1654c0-istio-envoy\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:50:54.314234 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.314238    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fe5845c-8eb1-4971-99be-1f66ef1654c0-credential-socket\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:50:54.314492 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.314270    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9zsr\" (UniqueName: \"kubernetes.io/projected/5fe5845c-8eb1-4971-99be-1f66ef1654c0-kube-api-access-j9zsr\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:50:54.314492 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.314282    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fe5845c-8eb1-4971-99be-1f66ef1654c0-istio-token\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:50:54.314492 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.314290    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fe5845c-8eb1-4971-99be-1f66ef1654c0-istiod-ca-cert\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:50:54.314492 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.314298    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fe5845c-8eb1-4971-99be-1f66ef1654c0-istio-podinfo\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:50:54.314492 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.314313    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fe5845c-8eb1-4971-99be-1f66ef1654c0-istio-data\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:50:54.662863 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.662766    2576 generic.go:356] "Generic (PLEG): container finished" podID="5fe5845c-8eb1-4971-99be-1f66ef1654c0" containerID="0ef0b58769505f59239458f2c3186932882bdf1b430e2b90eba975a0c10fd3da" exitCode=0
Jul 26 18:50:54.662863 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.662811    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8" event={"ID":"5fe5845c-8eb1-4971-99be-1f66ef1654c0","Type":"ContainerDied","Data":"0ef0b58769505f59239458f2c3186932882bdf1b430e2b90eba975a0c10fd3da"}
Jul 26 18:50:54.662863 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.662825    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"
Jul 26 18:50:54.663126 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.662898    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8" event={"ID":"5fe5845c-8eb1-4971-99be-1f66ef1654c0","Type":"ContainerDied","Data":"60b59dcdbc12663e9c26fe636caa8c5bca191f1f6392cc35d12441b2927e4c28"}
Jul 26 18:50:54.663126 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.662929    2576 scope.go:117] "RemoveContainer" containerID="0ef0b58769505f59239458f2c3186932882bdf1b430e2b90eba975a0c10fd3da"
Jul 26 18:50:54.672041 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.672018    2576 scope.go:117] "RemoveContainer" containerID="64bbceaef48ec525e586d52d1c6522da786612eacfa3e6c13468b807ba5e03f6"
Jul 26 18:50:54.679670 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.679652    2576 scope.go:117] "RemoveContainer" containerID="0ef0b58769505f59239458f2c3186932882bdf1b430e2b90eba975a0c10fd3da"
Jul 26 18:50:54.680236 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:50:54.680217    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ef0b58769505f59239458f2c3186932882bdf1b430e2b90eba975a0c10fd3da\": container with ID starting with 0ef0b58769505f59239458f2c3186932882bdf1b430e2b90eba975a0c10fd3da not found: ID does not exist" containerID="0ef0b58769505f59239458f2c3186932882bdf1b430e2b90eba975a0c10fd3da"
Jul 26 18:50:54.680306 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.680246    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ef0b58769505f59239458f2c3186932882bdf1b430e2b90eba975a0c10fd3da"} err="failed to get container status \"0ef0b58769505f59239458f2c3186932882bdf1b430e2b90eba975a0c10fd3da\": rpc error: code = NotFound desc = could not find container \"0ef0b58769505f59239458f2c3186932882bdf1b430e2b90eba975a0c10fd3da\": container with ID starting with 0ef0b58769505f59239458f2c3186932882bdf1b430e2b90eba975a0c10fd3da not found: ID does not exist"
Jul 26 18:50:54.680306 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.680268    2576 scope.go:117] "RemoveContainer" containerID="64bbceaef48ec525e586d52d1c6522da786612eacfa3e6c13468b807ba5e03f6"
Jul 26 18:50:54.680522 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:50:54.680506    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64bbceaef48ec525e586d52d1c6522da786612eacfa3e6c13468b807ba5e03f6\": container with ID starting with 64bbceaef48ec525e586d52d1c6522da786612eacfa3e6c13468b807ba5e03f6 not found: ID does not exist" containerID="64bbceaef48ec525e586d52d1c6522da786612eacfa3e6c13468b807ba5e03f6"
Jul 26 18:50:54.680569 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.680525    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64bbceaef48ec525e586d52d1c6522da786612eacfa3e6c13468b807ba5e03f6"} err="failed to get container status \"64bbceaef48ec525e586d52d1c6522da786612eacfa3e6c13468b807ba5e03f6\": rpc error: code = NotFound desc = could not find container \"64bbceaef48ec525e586d52d1c6522da786612eacfa3e6c13468b807ba5e03f6\": container with ID starting with 64bbceaef48ec525e586d52d1c6522da786612eacfa3e6c13468b807ba5e03f6 not found: ID does not exist"
Jul 26 18:50:54.685055 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.685030    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"]
Jul 26 18:50:54.689282 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.689261    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-991a15-openshift-default-6c56684744-5wpp8"]
Jul 26 18:50:54.849172 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:50:54.849140    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fe5845c-8eb1-4971-99be-1f66ef1654c0" path="/var/lib/kubelet/pods/5fe5845c-8eb1-4971-99be-1f66ef1654c0/volumes"
Jul 26 18:51:04.835504 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:04.835464    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz"]
Jul 26 18:51:04.836012 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:04.835711    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz" podUID="ae9fa5eb-09a9-49b9-b957-e3041741c11c" containerName="maas-api" containerID="cri-o://e35516bd54be96a5b97d5278b52441035f32cc53d88713c954a32f4dc0919de5" gracePeriod=30
Jul 26 18:51:04.873199 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:04.873166    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:51:04.888704 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:04.888663    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b9a04e-5fc8cc88-2zh6g"]
Jul 26 18:51:04.889198 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:04.889167    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-b9a04e-5fc8cc88-2zh6g" podUID="855ecb42-48c1-49e9-9360-cd34d00c05d9" containerName="payload-processing" containerID="cri-o://52e6be9adad57fc8690f9c5419157854f2705eaf0e044e23fa1c5bc0b8b5e922" gracePeriod=30
Jul 26 18:51:04.973429 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:04.973391    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:51:05.073833 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:05.073802    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:51:05.136114 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:05.136090    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz"
Jul 26 18:51:05.167846 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:05.167817    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-b9a04e-5fc8cc88-2zh6g"
Jul 26 18:51:05.208769 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:05.208715    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae9fa5eb-09a9-49b9-b957-e3041741c11c-maas-api-tls\") pod \"ae9fa5eb-09a9-49b9-b957-e3041741c11c\" (UID: \"ae9fa5eb-09a9-49b9-b957-e3041741c11c\") "
Jul 26 18:51:05.208961 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:05.208780    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lz7cn\" (UniqueName: \"kubernetes.io/projected/ae9fa5eb-09a9-49b9-b957-e3041741c11c-kube-api-access-lz7cn\") pod \"ae9fa5eb-09a9-49b9-b957-e3041741c11c\" (UID: \"ae9fa5eb-09a9-49b9-b957-e3041741c11c\") "
Jul 26 18:51:05.208961 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:05.208806    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae9fa5eb-09a9-49b9-b957-e3041741c11c-openshift-service-ca\") pod \"ae9fa5eb-09a9-49b9-b957-e3041741c11c\" (UID: \"ae9fa5eb-09a9-49b9-b957-e3041741c11c\") "
Jul 26 18:51:05.209247 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:05.209225    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae9fa5eb-09a9-49b9-b957-e3041741c11c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ae9fa5eb-09a9-49b9-b957-e3041741c11c" (UID: "ae9fa5eb-09a9-49b9-b957-e3041741c11c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:51:05.210907 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:05.210882    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae9fa5eb-09a9-49b9-b957-e3041741c11c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ae9fa5eb-09a9-49b9-b957-e3041741c11c" (UID: "ae9fa5eb-09a9-49b9-b957-e3041741c11c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:51:05.210907 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:05.210897    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae9fa5eb-09a9-49b9-b957-e3041741c11c-kube-api-access-lz7cn" (OuterVolumeSpecName: "kube-api-access-lz7cn") pod "ae9fa5eb-09a9-49b9-b957-e3041741c11c" (UID: "ae9fa5eb-09a9-49b9-b957-e3041741c11c"). InnerVolumeSpecName "kube-api-access-lz7cn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:51:05.310189 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:05.310149    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tl87m\" (UniqueName: \"kubernetes.io/projected/855ecb42-48c1-49e9-9360-cd34d00c05d9-kube-api-access-tl87m\") pod \"855ecb42-48c1-49e9-9360-cd34d00c05d9\" (UID: \"855ecb42-48c1-49e9-9360-cd34d00c05d9\") "
Jul 26 18:51:05.310370 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:05.310208    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/855ecb42-48c1-49e9-9360-cd34d00c05d9-plugins-config-volume\") pod \"855ecb42-48c1-49e9-9360-cd34d00c05d9\" (UID: \"855ecb42-48c1-49e9-9360-cd34d00c05d9\") "
Jul 26 18:51:05.310370 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:05.310366    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae9fa5eb-09a9-49b9-b957-e3041741c11c-maas-api-tls\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:51:05.310506 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:05.310397    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lz7cn\" (UniqueName: \"kubernetes.io/projected/ae9fa5eb-09a9-49b9-b957-e3041741c11c-kube-api-access-lz7cn\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:51:05.310506 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:05.310407    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae9fa5eb-09a9-49b9-b957-e3041741c11c-openshift-service-ca\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:51:05.310610 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:05.310587    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/855ecb42-48c1-49e9-9360-cd34d00c05d9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "855ecb42-48c1-49e9-9360-cd34d00c05d9" (UID: "855ecb42-48c1-49e9-9360-cd34d00c05d9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:51:05.312323 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:05.312299    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/855ecb42-48c1-49e9-9360-cd34d00c05d9-kube-api-access-tl87m" (OuterVolumeSpecName: "kube-api-access-tl87m") pod "855ecb42-48c1-49e9-9360-cd34d00c05d9" (UID: "855ecb42-48c1-49e9-9360-cd34d00c05d9"). InnerVolumeSpecName "kube-api-access-tl87m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:51:05.411503 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:05.411410    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tl87m\" (UniqueName: \"kubernetes.io/projected/855ecb42-48c1-49e9-9360-cd34d00c05d9-kube-api-access-tl87m\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:51:05.411503 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:05.411444    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/855ecb42-48c1-49e9-9360-cd34d00c05d9-plugins-config-volume\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:51:05.711342 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:05.711306    2576 generic.go:356] "Generic (PLEG): container finished" podID="ae9fa5eb-09a9-49b9-b957-e3041741c11c" containerID="e35516bd54be96a5b97d5278b52441035f32cc53d88713c954a32f4dc0919de5" exitCode=0
Jul 26 18:51:05.711558 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:05.711354    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz"
Jul 26 18:51:05.711558 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:05.711394    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz" event={"ID":"ae9fa5eb-09a9-49b9-b957-e3041741c11c","Type":"ContainerDied","Data":"e35516bd54be96a5b97d5278b52441035f32cc53d88713c954a32f4dc0919de5"}
Jul 26 18:51:05.711558 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:05.711476    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz" event={"ID":"ae9fa5eb-09a9-49b9-b957-e3041741c11c","Type":"ContainerDied","Data":"bf4d80f6dd4eb00952b1d2b5b34df2bc517ccb650af9f95208c9cb8feb68980f"}
Jul 26 18:51:05.711558 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:05.711508    2576 scope.go:117] "RemoveContainer" containerID="e35516bd54be96a5b97d5278b52441035f32cc53d88713c954a32f4dc0919de5"
Jul 26 18:51:05.712672 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:05.712575    2576 generic.go:356] "Generic (PLEG): container finished" podID="855ecb42-48c1-49e9-9360-cd34d00c05d9" containerID="52e6be9adad57fc8690f9c5419157854f2705eaf0e044e23fa1c5bc0b8b5e922" exitCode=0
Jul 26 18:51:05.712672 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:05.712627    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-b9a04e-5fc8cc88-2zh6g"
Jul 26 18:51:05.712840 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:05.712664    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-b9a04e-5fc8cc88-2zh6g" event={"ID":"855ecb42-48c1-49e9-9360-cd34d00c05d9","Type":"ContainerDied","Data":"52e6be9adad57fc8690f9c5419157854f2705eaf0e044e23fa1c5bc0b8b5e922"}
Jul 26 18:51:05.712840 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:05.712708    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-b9a04e-5fc8cc88-2zh6g" event={"ID":"855ecb42-48c1-49e9-9360-cd34d00c05d9","Type":"ContainerDied","Data":"459e7827dbef96ec333d081e0f2e70bc8a88a940b97b20cdda864985eb504fb2"}
Jul 26 18:51:05.720949 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:05.720910    2576 scope.go:117] "RemoveContainer" containerID="e35516bd54be96a5b97d5278b52441035f32cc53d88713c954a32f4dc0919de5"
Jul 26 18:51:05.721290 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:51:05.721271    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e35516bd54be96a5b97d5278b52441035f32cc53d88713c954a32f4dc0919de5\": container with ID starting with e35516bd54be96a5b97d5278b52441035f32cc53d88713c954a32f4dc0919de5 not found: ID does not exist" containerID="e35516bd54be96a5b97d5278b52441035f32cc53d88713c954a32f4dc0919de5"
Jul 26 18:51:05.721354 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:05.721299    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e35516bd54be96a5b97d5278b52441035f32cc53d88713c954a32f4dc0919de5"} err="failed to get container status \"e35516bd54be96a5b97d5278b52441035f32cc53d88713c954a32f4dc0919de5\": rpc error: code = NotFound desc = could not find container \"e35516bd54be96a5b97d5278b52441035f32cc53d88713c954a32f4dc0919de5\": container with ID starting with e35516bd54be96a5b97d5278b52441035f32cc53d88713c954a32f4dc0919de5 not found: ID does not exist"
Jul 26 18:51:05.721354 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:05.721317    2576 scope.go:117] "RemoveContainer" containerID="52e6be9adad57fc8690f9c5419157854f2705eaf0e044e23fa1c5bc0b8b5e922"
Jul 26 18:51:05.729927 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:05.729908    2576 scope.go:117] "RemoveContainer" containerID="52e6be9adad57fc8690f9c5419157854f2705eaf0e044e23fa1c5bc0b8b5e922"
Jul 26 18:51:05.730238 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:51:05.730218    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52e6be9adad57fc8690f9c5419157854f2705eaf0e044e23fa1c5bc0b8b5e922\": container with ID starting with 52e6be9adad57fc8690f9c5419157854f2705eaf0e044e23fa1c5bc0b8b5e922 not found: ID does not exist" containerID="52e6be9adad57fc8690f9c5419157854f2705eaf0e044e23fa1c5bc0b8b5e922"
Jul 26 18:51:05.730300 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:05.730246    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52e6be9adad57fc8690f9c5419157854f2705eaf0e044e23fa1c5bc0b8b5e922"} err="failed to get container status \"52e6be9adad57fc8690f9c5419157854f2705eaf0e044e23fa1c5bc0b8b5e922\": rpc error: code = NotFound desc = could not find container \"52e6be9adad57fc8690f9c5419157854f2705eaf0e044e23fa1c5bc0b8b5e922\": container with ID starting with 52e6be9adad57fc8690f9c5419157854f2705eaf0e044e23fa1c5bc0b8b5e922 not found: ID does not exist"
Jul 26 18:51:05.735955 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:05.735925    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz"]
Jul 26 18:51:05.739389 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:05.739365    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b9a04e-bdd68c57b-l7bdz"]
Jul 26 18:51:05.749290 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:05.749261    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b9a04e-5fc8cc88-2zh6g"]
Jul 26 18:51:05.752515 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:05.752491    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b9a04e-5fc8cc88-2zh6g"]
Jul 26 18:51:06.848827 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:06.848792    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="855ecb42-48c1-49e9-9360-cd34d00c05d9" path="/var/lib/kubelet/pods/855ecb42-48c1-49e9-9360-cd34d00c05d9/volumes"
Jul 26 18:51:06.849213 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:06.849199    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae9fa5eb-09a9-49b9-b957-e3041741c11c" path="/var/lib/kubelet/pods/ae9fa5eb-09a9-49b9-b957-e3041741c11c/volumes"
Jul 26 18:51:09.284094 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:09.284055    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:51:09.517135 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:09.517105    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:51:09.851093 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:09.851058    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:51:15.636214 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:15.636172    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:51:15.737580 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:15.737544    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:51:15.859403 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:15.859368    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"]
Jul 26 18:51:15.859649 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:15.859625    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns" podUID="a6eda582-d5f9-453a-ae0e-77d101298268" containerName="istio-proxy" containerID="cri-o://8d64babb556b6a31b0da938c3f2faa603ab3dafb769303a8ae6f46535e59c822" gracePeriod=30
Jul 26 18:51:15.885332 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:15.885300    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:51:16.094009 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.093969    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:51:16.417481 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.417392    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:51:16.721030 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.720996    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"]
Jul 26 18:51:16.721516 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.721494    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="855ecb42-48c1-49e9-9360-cd34d00c05d9" containerName="payload-processing"
Jul 26 18:51:16.721594 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.721519    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="855ecb42-48c1-49e9-9360-cd34d00c05d9" containerName="payload-processing"
Jul 26 18:51:16.721594 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.721533    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae9fa5eb-09a9-49b9-b957-e3041741c11c" containerName="maas-api"
Jul 26 18:51:16.721594 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.721542    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae9fa5eb-09a9-49b9-b957-e3041741c11c" containerName="maas-api"
Jul 26 18:51:16.721594 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.721560    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fe5845c-8eb1-4971-99be-1f66ef1654c0" containerName="istio-proxy"
Jul 26 18:51:16.721594 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.721569    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fe5845c-8eb1-4971-99be-1f66ef1654c0" containerName="istio-proxy"
Jul 26 18:51:16.721915 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.721596    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="596fb369-0dcb-46d3-91f3-51fe45340a81" containerName="maas-api"
Jul 26 18:51:16.721915 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.721605    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="596fb369-0dcb-46d3-91f3-51fe45340a81" containerName="maas-api"
Jul 26 18:51:16.721915 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.721686    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae9fa5eb-09a9-49b9-b957-e3041741c11c" containerName="maas-api"
Jul 26 18:51:16.721915 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.721700    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="596fb369-0dcb-46d3-91f3-51fe45340a81" containerName="maas-api"
Jul 26 18:51:16.721915 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.721711    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fe5845c-8eb1-4971-99be-1f66ef1654c0" containerName="istio-proxy"
Jul 26 18:51:16.721915 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.721722    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="855ecb42-48c1-49e9-9360-cd34d00c05d9" containerName="payload-processing"
Jul 26 18:51:16.721915 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.721731    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fe5845c-8eb1-4971-99be-1f66ef1654c0" containerName="istio-proxy"
Jul 26 18:51:16.725101 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.725079    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 18:51:16.728168 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.728144    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-2b193e-openshift-default-dockercfg-p4mvl\""
Jul 26 18:51:16.738718 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.738688    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"]
Jul 26 18:51:16.754321 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.754288    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:51:16.778566 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.778517    2576 patch_prober.go:28] interesting pod/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 26 18:51:16.778722 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.778584    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns" podUID="a6eda582-d5f9-453a-ae0e-77d101298268" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 26 18:51:16.812872 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.812828    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7dc0999-26b8-4201-8230-713d58ef3b16-istio-envoy\") pod \"e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm\" (UID: \"c7dc0999-26b8-4201-8230-713d58ef3b16\") " pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 18:51:16.813047 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.812900    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgl9d\" (UniqueName: \"kubernetes.io/projected/c7dc0999-26b8-4201-8230-713d58ef3b16-kube-api-access-tgl9d\") pod \"e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm\" (UID: \"c7dc0999-26b8-4201-8230-713d58ef3b16\") " pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 18:51:16.813047 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.812946    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7dc0999-26b8-4201-8230-713d58ef3b16-istio-data\") pod \"e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm\" (UID: \"c7dc0999-26b8-4201-8230-713d58ef3b16\") " pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 18:51:16.813047 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.813001    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7dc0999-26b8-4201-8230-713d58ef3b16-workload-certs\") pod \"e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm\" (UID: \"c7dc0999-26b8-4201-8230-713d58ef3b16\") " pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 18:51:16.813171 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.813049    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7dc0999-26b8-4201-8230-713d58ef3b16-credential-socket\") pod \"e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm\" (UID: \"c7dc0999-26b8-4201-8230-713d58ef3b16\") " pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 18:51:16.813171 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.813075    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7dc0999-26b8-4201-8230-713d58ef3b16-istiod-ca-cert\") pod \"e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm\" (UID: \"c7dc0999-26b8-4201-8230-713d58ef3b16\") " pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 18:51:16.813250 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.813162    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7dc0999-26b8-4201-8230-713d58ef3b16-istio-podinfo\") pod \"e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm\" (UID: \"c7dc0999-26b8-4201-8230-713d58ef3b16\") " pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 18:51:16.813250 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.813233    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7dc0999-26b8-4201-8230-713d58ef3b16-istio-token\") pod \"e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm\" (UID: \"c7dc0999-26b8-4201-8230-713d58ef3b16\") " pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 18:51:16.813383 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.813271    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7dc0999-26b8-4201-8230-713d58ef3b16-workload-socket\") pod \"e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm\" (UID: \"c7dc0999-26b8-4201-8230-713d58ef3b16\") " pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 18:51:16.855343 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.855276    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:51:16.913925 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.913885    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgl9d\" (UniqueName: \"kubernetes.io/projected/c7dc0999-26b8-4201-8230-713d58ef3b16-kube-api-access-tgl9d\") pod \"e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm\" (UID: \"c7dc0999-26b8-4201-8230-713d58ef3b16\") " pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 18:51:16.914138 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.913945    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7dc0999-26b8-4201-8230-713d58ef3b16-istio-data\") pod \"e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm\" (UID: \"c7dc0999-26b8-4201-8230-713d58ef3b16\") " pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 18:51:16.914138 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.913984    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7dc0999-26b8-4201-8230-713d58ef3b16-workload-certs\") pod \"e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm\" (UID: \"c7dc0999-26b8-4201-8230-713d58ef3b16\") " pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 18:51:16.914138 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.914026    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7dc0999-26b8-4201-8230-713d58ef3b16-credential-socket\") pod \"e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm\" (UID: \"c7dc0999-26b8-4201-8230-713d58ef3b16\") " pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 18:51:16.914328 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.914132    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7dc0999-26b8-4201-8230-713d58ef3b16-istiod-ca-cert\") pod \"e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm\" (UID: \"c7dc0999-26b8-4201-8230-713d58ef3b16\") " pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 18:51:16.914328 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.914182    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7dc0999-26b8-4201-8230-713d58ef3b16-istio-podinfo\") pod \"e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm\" (UID: \"c7dc0999-26b8-4201-8230-713d58ef3b16\") " pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 18:51:16.914328 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.914219    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7dc0999-26b8-4201-8230-713d58ef3b16-istio-token\") pod \"e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm\" (UID: \"c7dc0999-26b8-4201-8230-713d58ef3b16\") " pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 18:51:16.914328 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.914252    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7dc0999-26b8-4201-8230-713d58ef3b16-workload-socket\") pod \"e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm\" (UID: \"c7dc0999-26b8-4201-8230-713d58ef3b16\") " pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 18:51:16.914328 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.914312    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7dc0999-26b8-4201-8230-713d58ef3b16-istio-envoy\") pod \"e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm\" (UID: \"c7dc0999-26b8-4201-8230-713d58ef3b16\") " pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 18:51:16.914581 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.914437    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7dc0999-26b8-4201-8230-713d58ef3b16-credential-socket\") pod \"e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm\" (UID: \"c7dc0999-26b8-4201-8230-713d58ef3b16\") " pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 18:51:16.914581 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.914478    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7dc0999-26b8-4201-8230-713d58ef3b16-workload-certs\") pod \"e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm\" (UID: \"c7dc0999-26b8-4201-8230-713d58ef3b16\") " pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 18:51:16.914581 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.914552    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7dc0999-26b8-4201-8230-713d58ef3b16-istio-data\") pod \"e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm\" (UID: \"c7dc0999-26b8-4201-8230-713d58ef3b16\") " pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 18:51:16.914777 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.914719    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7dc0999-26b8-4201-8230-713d58ef3b16-workload-socket\") pod \"e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm\" (UID: \"c7dc0999-26b8-4201-8230-713d58ef3b16\") " pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 18:51:16.914887 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.914863    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7dc0999-26b8-4201-8230-713d58ef3b16-istiod-ca-cert\") pod \"e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm\" (UID: \"c7dc0999-26b8-4201-8230-713d58ef3b16\") " pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 18:51:16.916929 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.916893    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7dc0999-26b8-4201-8230-713d58ef3b16-istio-envoy\") pod \"e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm\" (UID: \"c7dc0999-26b8-4201-8230-713d58ef3b16\") " pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 18:51:16.924153 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.923492    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgl9d\" (UniqueName: \"kubernetes.io/projected/c7dc0999-26b8-4201-8230-713d58ef3b16-kube-api-access-tgl9d\") pod \"e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm\" (UID: \"c7dc0999-26b8-4201-8230-713d58ef3b16\") " pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 18:51:16.924871 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.924846    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7dc0999-26b8-4201-8230-713d58ef3b16-istio-podinfo\") pod \"e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm\" (UID: \"c7dc0999-26b8-4201-8230-713d58ef3b16\") " pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 18:51:16.925446 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:16.925424    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7dc0999-26b8-4201-8230-713d58ef3b16-istio-token\") pod \"e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm\" (UID: \"c7dc0999-26b8-4201-8230-713d58ef3b16\") " pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 18:51:17.035240 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:17.035144    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 18:51:17.178594 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:17.178568    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"]
Jul 26 18:51:17.181335 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:51:17.181306    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7dc0999_26b8_4201_8230_713d58ef3b16.slice/crio-31a99014f7ccc067d8809094f59ca0703865ea877877347a1a66f704fb3b4b94 WatchSource:0}: Error finding container 31a99014f7ccc067d8809094f59ca0703865ea877877347a1a66f704fb3b4b94: Status 404 returned error can't find the container with id 31a99014f7ccc067d8809094f59ca0703865ea877877347a1a66f704fb3b4b94
Jul 26 18:51:17.183417 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:17.183385    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:51:17.183516 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:17.183467    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:51:17.183578 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:17.183510    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:51:17.769338 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:17.769304    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm" event={"ID":"c7dc0999-26b8-4201-8230-713d58ef3b16","Type":"ContainerStarted","Data":"cf10d86cef8bfaf4511689ba8ec58e3d598ab2650356b4092699f4b1eaeb4e11"}
Jul 26 18:51:17.769776 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:17.769351    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm" event={"ID":"c7dc0999-26b8-4201-8230-713d58ef3b16","Type":"ContainerStarted","Data":"31a99014f7ccc067d8809094f59ca0703865ea877877347a1a66f704fb3b4b94"}
Jul 26 18:51:17.791411 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:17.791348    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm" podStartSLOduration=1.791331156 podStartE2EDuration="1.791331156s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:51:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:51:17.789135213 +0000 UTC m=+2363.496542272" watchObservedRunningTime="2026-07-26 18:51:17.791331156 +0000 UTC m=+2363.498738214"
Jul 26 18:51:18.036385 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:18.036289    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 18:51:18.041232 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:18.041206    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 18:51:18.775024 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:18.774995    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 18:51:18.775966 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:18.775948    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 18:51:19.668332 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:19.668285    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:51:19.748025 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:19.747990    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl"]
Jul 26 18:51:19.748533 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:19.748509    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fe5845c-8eb1-4971-99be-1f66ef1654c0" containerName="istio-proxy"
Jul 26 18:51:19.748701 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:19.748688    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fe5845c-8eb1-4971-99be-1f66ef1654c0" containerName="istio-proxy"
Jul 26 18:51:19.750025 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:19.749999    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:51:19.751261 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:19.751239    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl"
Jul 26 18:51:19.753987 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:19.753961    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-2b193e-serving-cert\""
Jul 26 18:51:19.754134 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:19.754061    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 26 18:51:19.754292 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:19.754272    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 18:51:19.755165 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:19.755142    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9n9b7\""
Jul 26 18:51:19.762814 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:19.762792    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl"]
Jul 26 18:51:19.821128 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:19.821090    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:51:19.839228 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:19.839181    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75fb92d2-3709-4227-a0a4-b2c84ddcea8d-maas-api-tls\") pod \"maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl\" (UID: \"75fb92d2-3709-4227-a0a4-b2c84ddcea8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl"
Jul 26 18:51:19.839408 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:19.839241    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbk5s\" (UniqueName: \"kubernetes.io/projected/75fb92d2-3709-4227-a0a4-b2c84ddcea8d-kube-api-access-cbk5s\") pod \"maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl\" (UID: \"75fb92d2-3709-4227-a0a4-b2c84ddcea8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl"
Jul 26 18:51:19.839408 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:19.839366    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75fb92d2-3709-4227-a0a4-b2c84ddcea8d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl\" (UID: \"75fb92d2-3709-4227-a0a4-b2c84ddcea8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl"
Jul 26 18:51:19.940017 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:19.939972    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75fb92d2-3709-4227-a0a4-b2c84ddcea8d-maas-api-tls\") pod \"maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl\" (UID: \"75fb92d2-3709-4227-a0a4-b2c84ddcea8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl"
Jul 26 18:51:19.940254 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:19.940034    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbk5s\" (UniqueName: \"kubernetes.io/projected/75fb92d2-3709-4227-a0a4-b2c84ddcea8d-kube-api-access-cbk5s\") pod \"maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl\" (UID: \"75fb92d2-3709-4227-a0a4-b2c84ddcea8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl"
Jul 26 18:51:19.940254 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:19.940118    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75fb92d2-3709-4227-a0a4-b2c84ddcea8d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl\" (UID: \"75fb92d2-3709-4227-a0a4-b2c84ddcea8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl"
Jul 26 18:51:19.940254 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:51:19.940158    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-2b193e-serving-cert: secret "maas-api-e2e-shared-a-2b193e-serving-cert" not found
Jul 26 18:51:19.940395 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:51:19.940256    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75fb92d2-3709-4227-a0a4-b2c84ddcea8d-maas-api-tls podName:75fb92d2-3709-4227-a0a4-b2c84ddcea8d nodeName:}" failed. No retries permitted until 2026-07-26 18:51:20.440226271 +0000 UTC m=+2366.147633337 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/75fb92d2-3709-4227-a0a4-b2c84ddcea8d-maas-api-tls") pod "maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl" (UID: "75fb92d2-3709-4227-a0a4-b2c84ddcea8d") : secret "maas-api-e2e-shared-a-2b193e-serving-cert" not found
Jul 26 18:51:19.940892 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:19.940871    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75fb92d2-3709-4227-a0a4-b2c84ddcea8d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl\" (UID: \"75fb92d2-3709-4227-a0a4-b2c84ddcea8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl"
Jul 26 18:51:19.954501 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:19.954467    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbk5s\" (UniqueName: \"kubernetes.io/projected/75fb92d2-3709-4227-a0a4-b2c84ddcea8d-kube-api-access-cbk5s\") pod \"maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl\" (UID: \"75fb92d2-3709-4227-a0a4-b2c84ddcea8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl"
Jul 26 18:51:20.357430 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:20.357326    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:51:20.446188 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:20.446154    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75fb92d2-3709-4227-a0a4-b2c84ddcea8d-maas-api-tls\") pod \"maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl\" (UID: \"75fb92d2-3709-4227-a0a4-b2c84ddcea8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl"
Jul 26 18:51:20.448672 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:20.448649    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75fb92d2-3709-4227-a0a4-b2c84ddcea8d-maas-api-tls\") pod \"maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl\" (UID: \"75fb92d2-3709-4227-a0a4-b2c84ddcea8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl"
Jul 26 18:51:20.456589 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:20.456560    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:51:20.542455 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:20.542423    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:51:20.665148 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:20.665057    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl"
Jul 26 18:51:20.803451 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:20.803423    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl"]
Jul 26 18:51:20.805828 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:51:20.805797    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75fb92d2_3709_4227_a0a4_b2c84ddcea8d.slice/crio-e3bec8a760ac5888e47a828e3042f5a438e89c44d446cfc0de904809b58ac50b WatchSource:0}: Error finding container e3bec8a760ac5888e47a828e3042f5a438e89c44d446cfc0de904809b58ac50b: Status 404 returned error can't find the container with id e3bec8a760ac5888e47a828e3042f5a438e89c44d446cfc0de904809b58ac50b
Jul 26 18:51:21.115464 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.115440    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:51:21.252739 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.252709    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6eda582-d5f9-453a-ae0e-77d101298268-istio-token\") pod \"a6eda582-d5f9-453a-ae0e-77d101298268\" (UID: \"a6eda582-d5f9-453a-ae0e-77d101298268\") "
Jul 26 18:51:21.252877 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.252781    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6eda582-d5f9-453a-ae0e-77d101298268-workload-certs\") pod \"a6eda582-d5f9-453a-ae0e-77d101298268\" (UID: \"a6eda582-d5f9-453a-ae0e-77d101298268\") "
Jul 26 18:51:21.252877 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.252803    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6eda582-d5f9-453a-ae0e-77d101298268-istio-podinfo\") pod \"a6eda582-d5f9-453a-ae0e-77d101298268\" (UID: \"a6eda582-d5f9-453a-ae0e-77d101298268\") "
Jul 26 18:51:21.252877 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.252829    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6eda582-d5f9-453a-ae0e-77d101298268-credential-socket\") pod \"a6eda582-d5f9-453a-ae0e-77d101298268\" (UID: \"a6eda582-d5f9-453a-ae0e-77d101298268\") "
Jul 26 18:51:21.252877 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.252867    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6eda582-d5f9-453a-ae0e-77d101298268-istio-envoy\") pod \"a6eda582-d5f9-453a-ae0e-77d101298268\" (UID: \"a6eda582-d5f9-453a-ae0e-77d101298268\") "
Jul 26 18:51:21.253083 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.252910    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6eda582-d5f9-453a-ae0e-77d101298268-workload-socket\") pod \"a6eda582-d5f9-453a-ae0e-77d101298268\" (UID: \"a6eda582-d5f9-453a-ae0e-77d101298268\") "
Jul 26 18:51:21.253083 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.252950    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6f9k4\" (UniqueName: \"kubernetes.io/projected/a6eda582-d5f9-453a-ae0e-77d101298268-kube-api-access-6f9k4\") pod \"a6eda582-d5f9-453a-ae0e-77d101298268\" (UID: \"a6eda582-d5f9-453a-ae0e-77d101298268\") "
Jul 26 18:51:21.253083 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.252968    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6eda582-d5f9-453a-ae0e-77d101298268-istio-data\") pod \"a6eda582-d5f9-453a-ae0e-77d101298268\" (UID: \"a6eda582-d5f9-453a-ae0e-77d101298268\") "
Jul 26 18:51:21.253083 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.252989    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6eda582-d5f9-453a-ae0e-77d101298268-istiod-ca-cert\") pod \"a6eda582-d5f9-453a-ae0e-77d101298268\" (UID: \"a6eda582-d5f9-453a-ae0e-77d101298268\") "
Jul 26 18:51:21.253302 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.253091    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6eda582-d5f9-453a-ae0e-77d101298268-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a6eda582-d5f9-453a-ae0e-77d101298268" (UID: "a6eda582-d5f9-453a-ae0e-77d101298268"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:51:21.253302 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.253152    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6eda582-d5f9-453a-ae0e-77d101298268-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a6eda582-d5f9-453a-ae0e-77d101298268" (UID: "a6eda582-d5f9-453a-ae0e-77d101298268"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:51:21.253409 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.253304    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6eda582-d5f9-453a-ae0e-77d101298268-workload-certs\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:51:21.253409 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.253325    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6eda582-d5f9-453a-ae0e-77d101298268-credential-socket\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:51:21.253516 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.253406    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6eda582-d5f9-453a-ae0e-77d101298268-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a6eda582-d5f9-453a-ae0e-77d101298268" (UID: "a6eda582-d5f9-453a-ae0e-77d101298268"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:51:21.253571 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.253513    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6eda582-d5f9-453a-ae0e-77d101298268-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a6eda582-d5f9-453a-ae0e-77d101298268" (UID: "a6eda582-d5f9-453a-ae0e-77d101298268"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:51:21.254146 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.254118    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6eda582-d5f9-453a-ae0e-77d101298268-istio-data" (OuterVolumeSpecName: "istio-data") pod "a6eda582-d5f9-453a-ae0e-77d101298268" (UID: "a6eda582-d5f9-453a-ae0e-77d101298268"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:51:21.254999 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.254973    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a6eda582-d5f9-453a-ae0e-77d101298268-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a6eda582-d5f9-453a-ae0e-77d101298268" (UID: "a6eda582-d5f9-453a-ae0e-77d101298268"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 26 18:51:21.255093 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.255018    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6eda582-d5f9-453a-ae0e-77d101298268-istio-token" (OuterVolumeSpecName: "istio-token") pod "a6eda582-d5f9-453a-ae0e-77d101298268" (UID: "a6eda582-d5f9-453a-ae0e-77d101298268"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:51:21.255093 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.255054    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6eda582-d5f9-453a-ae0e-77d101298268-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a6eda582-d5f9-453a-ae0e-77d101298268" (UID: "a6eda582-d5f9-453a-ae0e-77d101298268"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:51:21.255093 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.255056    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6eda582-d5f9-453a-ae0e-77d101298268-kube-api-access-6f9k4" (OuterVolumeSpecName: "kube-api-access-6f9k4") pod "a6eda582-d5f9-453a-ae0e-77d101298268" (UID: "a6eda582-d5f9-453a-ae0e-77d101298268"). InnerVolumeSpecName "kube-api-access-6f9k4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:51:21.354816 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.354772    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6eda582-d5f9-453a-ae0e-77d101298268-istio-token\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:51:21.354816 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.354808    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6eda582-d5f9-453a-ae0e-77d101298268-istio-podinfo\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:51:21.354816 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.354820    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6eda582-d5f9-453a-ae0e-77d101298268-istio-envoy\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:51:21.355048 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.354831    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6eda582-d5f9-453a-ae0e-77d101298268-workload-socket\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:51:21.355048 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.354843    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6f9k4\" (UniqueName: \"kubernetes.io/projected/a6eda582-d5f9-453a-ae0e-77d101298268-kube-api-access-6f9k4\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:51:21.355048 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.354857    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6eda582-d5f9-453a-ae0e-77d101298268-istio-data\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:51:21.355048 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.354868    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6eda582-d5f9-453a-ae0e-77d101298268-istiod-ca-cert\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 18:51:21.791030 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.790985    2576 generic.go:356] "Generic (PLEG): container finished" podID="a6eda582-d5f9-453a-ae0e-77d101298268" containerID="8d64babb556b6a31b0da938c3f2faa603ab3dafb769303a8ae6f46535e59c822" exitCode=0
Jul 26 18:51:21.791242 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.791039    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"
Jul 26 18:51:21.791242 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.791031    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns" event={"ID":"a6eda582-d5f9-453a-ae0e-77d101298268","Type":"ContainerDied","Data":"8d64babb556b6a31b0da938c3f2faa603ab3dafb769303a8ae6f46535e59c822"}
Jul 26 18:51:21.791242 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.791168    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns" event={"ID":"a6eda582-d5f9-453a-ae0e-77d101298268","Type":"ContainerDied","Data":"2a423bd07312a0579478129d50c7446ec275c75e534816bd52d9677d073813a7"}
Jul 26 18:51:21.791242 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.791200    2576 scope.go:117] "RemoveContainer" containerID="8d64babb556b6a31b0da938c3f2faa603ab3dafb769303a8ae6f46535e59c822"
Jul 26 18:51:21.792844 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.792810    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl" event={"ID":"75fb92d2-3709-4227-a0a4-b2c84ddcea8d","Type":"ContainerStarted","Data":"9a0821c2e916758026d6a58ca8dcbabcef3295177cfbfb492ff4a148960e39ce"}
Jul 26 18:51:21.792963 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.792858    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl" event={"ID":"75fb92d2-3709-4227-a0a4-b2c84ddcea8d","Type":"ContainerStarted","Data":"e3bec8a760ac5888e47a828e3042f5a438e89c44d446cfc0de904809b58ac50b"}
Jul 26 18:51:21.792963 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.792952    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl"
Jul 26 18:51:21.799960 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.799941    2576 scope.go:117] "RemoveContainer" containerID="877f35d23c0bc9563bc63b02631fe42bb1f02098e98d48ff79ce67fdd1f7fd59"
Jul 26 18:51:21.807507 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.807489    2576 scope.go:117] "RemoveContainer" containerID="8d64babb556b6a31b0da938c3f2faa603ab3dafb769303a8ae6f46535e59c822"
Jul 26 18:51:21.807768 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:51:21.807735    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d64babb556b6a31b0da938c3f2faa603ab3dafb769303a8ae6f46535e59c822\": container with ID starting with 8d64babb556b6a31b0da938c3f2faa603ab3dafb769303a8ae6f46535e59c822 not found: ID does not exist" containerID="8d64babb556b6a31b0da938c3f2faa603ab3dafb769303a8ae6f46535e59c822"
Jul 26 18:51:21.807834 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.807784    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d64babb556b6a31b0da938c3f2faa603ab3dafb769303a8ae6f46535e59c822"} err="failed to get container status \"8d64babb556b6a31b0da938c3f2faa603ab3dafb769303a8ae6f46535e59c822\": rpc error: code = NotFound desc = could not find container \"8d64babb556b6a31b0da938c3f2faa603ab3dafb769303a8ae6f46535e59c822\": container with ID starting with 8d64babb556b6a31b0da938c3f2faa603ab3dafb769303a8ae6f46535e59c822 not found: ID does not exist"
Jul 26 18:51:21.807834 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.807810    2576 scope.go:117] "RemoveContainer" containerID="877f35d23c0bc9563bc63b02631fe42bb1f02098e98d48ff79ce67fdd1f7fd59"
Jul 26 18:51:21.808070 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:51:21.808053    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"877f35d23c0bc9563bc63b02631fe42bb1f02098e98d48ff79ce67fdd1f7fd59\": container with ID starting with 877f35d23c0bc9563bc63b02631fe42bb1f02098e98d48ff79ce67fdd1f7fd59 not found: ID does not exist" containerID="877f35d23c0bc9563bc63b02631fe42bb1f02098e98d48ff79ce67fdd1f7fd59"
Jul 26 18:51:21.808110 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.808077    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"877f35d23c0bc9563bc63b02631fe42bb1f02098e98d48ff79ce67fdd1f7fd59"} err="failed to get container status \"877f35d23c0bc9563bc63b02631fe42bb1f02098e98d48ff79ce67fdd1f7fd59\": rpc error: code = NotFound desc = could not find container \"877f35d23c0bc9563bc63b02631fe42bb1f02098e98d48ff79ce67fdd1f7fd59\": container with ID starting with 877f35d23c0bc9563bc63b02631fe42bb1f02098e98d48ff79ce67fdd1f7fd59 not found: ID does not exist"
Jul 26 18:51:21.811145 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.811104    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl" podStartSLOduration=2.386573369 podStartE2EDuration="2.811088867s" totalImagesPullingTime="424.515498ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:51:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:51:21.809923508 +0000 UTC m=+2367.517330568" watchObservedRunningTime="2026-07-26 18:51:21.811088867 +0000 UTC m=+2367.518495942"
Jul 26 18:51:21.824567 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.824537    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"]
Jul 26 18:51:21.828991 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:21.828967    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-b9a04e-openshift-default-679d4b54b8-v7jns"]
Jul 26 18:51:22.849789 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:22.849735    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6eda582-d5f9-453a-ae0e-77d101298268" path="/var/lib/kubelet/pods/a6eda582-d5f9-453a-ae0e-77d101298268/volumes"
Jul 26 18:51:24.129966 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.129926    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:51:24.457221 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.457155    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:51:24.707741 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.707652    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"]
Jul 26 18:51:24.708425 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.708400    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6eda582-d5f9-453a-ae0e-77d101298268" containerName="istio-proxy"
Jul 26 18:51:24.708425 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.708424    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6eda582-d5f9-453a-ae0e-77d101298268" containerName="istio-proxy"
Jul 26 18:51:24.708587 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.708450    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6eda582-d5f9-453a-ae0e-77d101298268" containerName="istio-proxy"
Jul 26 18:51:24.708587 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.708459    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6eda582-d5f9-453a-ae0e-77d101298268" containerName="istio-proxy"
Jul 26 18:51:24.708587 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.708536    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6eda582-d5f9-453a-ae0e-77d101298268" containerName="istio-proxy"
Jul 26 18:51:24.708587 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.708549    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6eda582-d5f9-453a-ae0e-77d101298268" containerName="istio-proxy"
Jul 26 18:51:24.711574 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.711551    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 18:51:24.714054 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.714029    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-c75d5a-openshift-default-dockercfg-9p7fz\""
Jul 26 18:51:24.724225 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.724198    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"]
Jul 26 18:51:24.741309 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.741254    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:51:24.843267 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.843232    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:51:24.889021 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.888977    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qn9jz\" (UniqueName: \"kubernetes.io/projected/075468d1-5f16-4760-bc00-999ee7927f2b-kube-api-access-qn9jz\") pod \"e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq\" (UID: \"075468d1-5f16-4760-bc00-999ee7927f2b\") " pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 18:51:24.889021 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.889022    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/075468d1-5f16-4760-bc00-999ee7927f2b-workload-socket\") pod \"e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq\" (UID: \"075468d1-5f16-4760-bc00-999ee7927f2b\") " pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 18:51:24.889238 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.889043    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/075468d1-5f16-4760-bc00-999ee7927f2b-istio-envoy\") pod \"e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq\" (UID: \"075468d1-5f16-4760-bc00-999ee7927f2b\") " pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 18:51:24.889238 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.889150    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/075468d1-5f16-4760-bc00-999ee7927f2b-credential-socket\") pod \"e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq\" (UID: \"075468d1-5f16-4760-bc00-999ee7927f2b\") " pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 18:51:24.889238 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.889200    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/075468d1-5f16-4760-bc00-999ee7927f2b-istio-podinfo\") pod \"e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq\" (UID: \"075468d1-5f16-4760-bc00-999ee7927f2b\") " pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 18:51:24.889238 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.889220    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/075468d1-5f16-4760-bc00-999ee7927f2b-istiod-ca-cert\") pod \"e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq\" (UID: \"075468d1-5f16-4760-bc00-999ee7927f2b\") " pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 18:51:24.889367 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.889251    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/075468d1-5f16-4760-bc00-999ee7927f2b-istio-data\") pod \"e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq\" (UID: \"075468d1-5f16-4760-bc00-999ee7927f2b\") " pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 18:51:24.889367 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.889273    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/075468d1-5f16-4760-bc00-999ee7927f2b-workload-certs\") pod \"e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq\" (UID: \"075468d1-5f16-4760-bc00-999ee7927f2b\") " pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 18:51:24.889367 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.889319    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/075468d1-5f16-4760-bc00-999ee7927f2b-istio-token\") pod \"e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq\" (UID: \"075468d1-5f16-4760-bc00-999ee7927f2b\") " pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 18:51:24.990126 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.990014    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/075468d1-5f16-4760-bc00-999ee7927f2b-istio-envoy\") pod \"e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq\" (UID: \"075468d1-5f16-4760-bc00-999ee7927f2b\") " pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 18:51:24.990126 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.990094    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/075468d1-5f16-4760-bc00-999ee7927f2b-credential-socket\") pod \"e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq\" (UID: \"075468d1-5f16-4760-bc00-999ee7927f2b\") " pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 18:51:24.990387 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.990136    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/075468d1-5f16-4760-bc00-999ee7927f2b-istio-podinfo\") pod \"e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq\" (UID: \"075468d1-5f16-4760-bc00-999ee7927f2b\") " pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 18:51:24.990387 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.990161    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/075468d1-5f16-4760-bc00-999ee7927f2b-istiod-ca-cert\") pod \"e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq\" (UID: \"075468d1-5f16-4760-bc00-999ee7927f2b\") " pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 18:51:24.990387 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.990185    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/075468d1-5f16-4760-bc00-999ee7927f2b-istio-data\") pod \"e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq\" (UID: \"075468d1-5f16-4760-bc00-999ee7927f2b\") " pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 18:51:24.990387 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.990229    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/075468d1-5f16-4760-bc00-999ee7927f2b-workload-certs\") pod \"e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq\" (UID: \"075468d1-5f16-4760-bc00-999ee7927f2b\") " pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 18:51:24.990387 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.990257    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/075468d1-5f16-4760-bc00-999ee7927f2b-istio-token\") pod \"e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq\" (UID: \"075468d1-5f16-4760-bc00-999ee7927f2b\") " pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 18:51:24.990387 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.990316    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qn9jz\" (UniqueName: \"kubernetes.io/projected/075468d1-5f16-4760-bc00-999ee7927f2b-kube-api-access-qn9jz\") pod \"e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq\" (UID: \"075468d1-5f16-4760-bc00-999ee7927f2b\") " pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 18:51:24.990387 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.990352    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/075468d1-5f16-4760-bc00-999ee7927f2b-workload-socket\") pod \"e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq\" (UID: \"075468d1-5f16-4760-bc00-999ee7927f2b\") " pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 18:51:24.990716 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.990548    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/075468d1-5f16-4760-bc00-999ee7927f2b-credential-socket\") pod \"e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq\" (UID: \"075468d1-5f16-4760-bc00-999ee7927f2b\") " pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 18:51:24.990716 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.990662    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/075468d1-5f16-4760-bc00-999ee7927f2b-workload-certs\") pod \"e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq\" (UID: \"075468d1-5f16-4760-bc00-999ee7927f2b\") " pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 18:51:24.990845 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.990739    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/075468d1-5f16-4760-bc00-999ee7927f2b-workload-socket\") pod \"e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq\" (UID: \"075468d1-5f16-4760-bc00-999ee7927f2b\") " pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 18:51:24.990905 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.990862    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/075468d1-5f16-4760-bc00-999ee7927f2b-istio-data\") pod \"e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq\" (UID: \"075468d1-5f16-4760-bc00-999ee7927f2b\") " pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 18:51:24.991079 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.991059    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/075468d1-5f16-4760-bc00-999ee7927f2b-istiod-ca-cert\") pod \"e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq\" (UID: \"075468d1-5f16-4760-bc00-999ee7927f2b\") " pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 18:51:24.992375 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.992349    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/075468d1-5f16-4760-bc00-999ee7927f2b-istio-envoy\") pod \"e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq\" (UID: \"075468d1-5f16-4760-bc00-999ee7927f2b\") " pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 18:51:24.992567 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.992550    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/075468d1-5f16-4760-bc00-999ee7927f2b-istio-podinfo\") pod \"e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq\" (UID: \"075468d1-5f16-4760-bc00-999ee7927f2b\") " pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 18:51:24.998500 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.998478    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/075468d1-5f16-4760-bc00-999ee7927f2b-istio-token\") pod \"e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq\" (UID: \"075468d1-5f16-4760-bc00-999ee7927f2b\") " pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 18:51:24.998644 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:24.998627    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qn9jz\" (UniqueName: \"kubernetes.io/projected/075468d1-5f16-4760-bc00-999ee7927f2b-kube-api-access-qn9jz\") pod \"e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq\" (UID: \"075468d1-5f16-4760-bc00-999ee7927f2b\") " pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 18:51:25.024537 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:25.024497    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 18:51:25.163494 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:25.163461    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"]
Jul 26 18:51:25.166942 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:51:25.166915    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod075468d1_5f16_4760_bc00_999ee7927f2b.slice/crio-3d6e81012b9961bb2065162f3d7e56b2acfff7c49dc963e7b254eb337ff868a6 WatchSource:0}: Error finding container 3d6e81012b9961bb2065162f3d7e56b2acfff7c49dc963e7b254eb337ff868a6: Status 404 returned error can't find the container with id 3d6e81012b9961bb2065162f3d7e56b2acfff7c49dc963e7b254eb337ff868a6
Jul 26 18:51:25.169795 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:25.169729    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:51:25.169870 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:25.169845    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:51:25.169910 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:25.169888    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:51:25.813566 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:25.813529    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq" event={"ID":"075468d1-5f16-4760-bc00-999ee7927f2b","Type":"ContainerStarted","Data":"f262c7a755a7b72229d6aa9b047e054e675f58d416f760fc9f1d81180f139108"}
Jul 26 18:51:25.813770 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:25.813581    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq" event={"ID":"075468d1-5f16-4760-bc00-999ee7927f2b","Type":"ContainerStarted","Data":"3d6e81012b9961bb2065162f3d7e56b2acfff7c49dc963e7b254eb337ff868a6"}
Jul 26 18:51:25.833692 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:25.833635    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq" podStartSLOduration=1.833617947 podStartE2EDuration="1.833617947s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:51:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:51:25.832109029 +0000 UTC m=+2371.539516089" watchObservedRunningTime="2026-07-26 18:51:25.833617947 +0000 UTC m=+2371.541025005"
Jul 26 18:51:26.024970 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:26.024925    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 18:51:26.030223 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:26.030196    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 18:51:26.818615 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:26.818587    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 18:51:26.819737 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:26.819712    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 18:51:27.390119 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:27.390078    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:51:27.684950 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:27.684917    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:51:27.766314 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:27.766282    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q"]
Jul 26 18:51:27.768762 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:27.768723    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:51:27.769229 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:27.769211    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q"
Jul 26 18:51:27.771624 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:27.771566    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-c75d5a-serving-cert\""
Jul 26 18:51:27.781117 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:27.781091    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q"]
Jul 26 18:51:27.803994 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:27.803964    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl"
Jul 26 18:51:27.862002 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:27.861951    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:51:27.918155 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:27.918114    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4eb8c250-24f0-46eb-b606-1f68c4f165ef-openshift-service-ca\") pod \"maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q\" (UID: \"4eb8c250-24f0-46eb-b606-1f68c4f165ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q"
Jul 26 18:51:27.918361 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:27.918209    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4eb8c250-24f0-46eb-b606-1f68c4f165ef-maas-api-tls\") pod \"maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q\" (UID: \"4eb8c250-24f0-46eb-b606-1f68c4f165ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q"
Jul 26 18:51:27.918361 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:27.918234    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcr5v\" (UniqueName: \"kubernetes.io/projected/4eb8c250-24f0-46eb-b606-1f68c4f165ef-kube-api-access-qcr5v\") pod \"maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q\" (UID: \"4eb8c250-24f0-46eb-b606-1f68c4f165ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q"
Jul 26 18:51:28.019400 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:28.019285    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4eb8c250-24f0-46eb-b606-1f68c4f165ef-maas-api-tls\") pod \"maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q\" (UID: \"4eb8c250-24f0-46eb-b606-1f68c4f165ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q"
Jul 26 18:51:28.019400 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:28.019346    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcr5v\" (UniqueName: \"kubernetes.io/projected/4eb8c250-24f0-46eb-b606-1f68c4f165ef-kube-api-access-qcr5v\") pod \"maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q\" (UID: \"4eb8c250-24f0-46eb-b606-1f68c4f165ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q"
Jul 26 18:51:28.019642 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:51:28.019440    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-c75d5a-serving-cert: secret "maas-api-e2e-shared-b-c75d5a-serving-cert" not found
Jul 26 18:51:28.019642 ip-10-0-134-7 kubenswrapper[2576]: E0726 18:51:28.019522    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4eb8c250-24f0-46eb-b606-1f68c4f165ef-maas-api-tls podName:4eb8c250-24f0-46eb-b606-1f68c4f165ef nodeName:}" failed. No retries permitted until 2026-07-26 18:51:28.519500671 +0000 UTC m=+2374.226907712 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4eb8c250-24f0-46eb-b606-1f68c4f165ef-maas-api-tls") pod "maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q" (UID: "4eb8c250-24f0-46eb-b606-1f68c4f165ef") : secret "maas-api-e2e-shared-b-c75d5a-serving-cert" not found
Jul 26 18:51:28.019642 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:28.019444    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4eb8c250-24f0-46eb-b606-1f68c4f165ef-openshift-service-ca\") pod \"maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q\" (UID: \"4eb8c250-24f0-46eb-b606-1f68c4f165ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q"
Jul 26 18:51:28.020119 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:28.020097    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4eb8c250-24f0-46eb-b606-1f68c4f165ef-openshift-service-ca\") pod \"maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q\" (UID: \"4eb8c250-24f0-46eb-b606-1f68c4f165ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q"
Jul 26 18:51:28.036845 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:28.036807    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcr5v\" (UniqueName: \"kubernetes.io/projected/4eb8c250-24f0-46eb-b606-1f68c4f165ef-kube-api-access-qcr5v\") pod \"maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q\" (UID: \"4eb8c250-24f0-46eb-b606-1f68c4f165ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q"
Jul 26 18:51:28.230376 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:28.230336    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:51:28.415529 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:28.415444    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:51:28.511266 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:28.511223    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:51:28.524630 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:28.524586    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4eb8c250-24f0-46eb-b606-1f68c4f165ef-maas-api-tls\") pod \"maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q\" (UID: \"4eb8c250-24f0-46eb-b606-1f68c4f165ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q"
Jul 26 18:51:28.528236 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:28.528202    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4eb8c250-24f0-46eb-b606-1f68c4f165ef-maas-api-tls\") pod \"maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q\" (UID: \"4eb8c250-24f0-46eb-b606-1f68c4f165ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q"
Jul 26 18:51:28.685013 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:28.684973    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q"
Jul 26 18:51:28.901872 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:28.901688    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q"]
Jul 26 18:51:28.903947 ip-10-0-134-7 kubenswrapper[2576]: W0726 18:51:28.903920    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4eb8c250_24f0_46eb_b606_1f68c4f165ef.slice/crio-bb4f2a63fc9d89551791989b9904c724724df0a3678c358fc4dd2f1312c127c9 WatchSource:0}: Error finding container bb4f2a63fc9d89551791989b9904c724724df0a3678c358fc4dd2f1312c127c9: Status 404 returned error can't find the container with id bb4f2a63fc9d89551791989b9904c724724df0a3678c358fc4dd2f1312c127c9
Jul 26 18:51:29.837478 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:29.837438    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q" event={"ID":"4eb8c250-24f0-46eb-b606-1f68c4f165ef","Type":"ContainerStarted","Data":"8dc7ea3d0f032e039d7b85d53ae3529343754482a6c50b8dca01b0cb1f25cda2"}
Jul 26 18:51:29.837696 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:29.837503    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q" event={"ID":"4eb8c250-24f0-46eb-b606-1f68c4f165ef","Type":"ContainerStarted","Data":"bb4f2a63fc9d89551791989b9904c724724df0a3678c358fc4dd2f1312c127c9"}
Jul 26 18:51:29.837696 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:29.837555    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q"
Jul 26 18:51:29.855079 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:29.855006    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q" podStartSLOduration=2.4719921449999998 podStartE2EDuration="2.854984179s" totalImagesPullingTime="382.992034ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:51:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:51:29.853331761 +0000 UTC m=+2375.560738821" watchObservedRunningTime="2026-07-26 18:51:29.854984179 +0000 UTC m=+2375.562391240"
Jul 26 18:51:32.166645 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:32.166610    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:51:34.816959 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:34.816911    2576 patch_prober.go:28] interesting pod/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 26 18:51:34.817355 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:34.816985    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm" podUID="c7dc0999-26b8-4201-8230-713d58ef3b16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 26 18:51:35.857530 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:35.856625    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q"
Jul 26 18:51:35.872713 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:35.872675    2576 generic.go:356] "Generic (PLEG): container finished" podID="c7dc0999-26b8-4201-8230-713d58ef3b16" containerID="cf10d86cef8bfaf4511689ba8ec58e3d598ab2650356b4092699f4b1eaeb4e11" exitCode=0
Jul 26 18:51:35.872917 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:35.872761    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm" event={"ID":"c7dc0999-26b8-4201-8230-713d58ef3b16","Type":"ContainerDied","Data":"cf10d86cef8bfaf4511689ba8ec58e3d598ab2650356b4092699f4b1eaeb4e11"}
Jul 26 18:51:35.873177 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:35.873158    2576 scope.go:117] "RemoveContainer" containerID="cf10d86cef8bfaf4511689ba8ec58e3d598ab2650356b4092699f4b1eaeb4e11"
Jul 26 18:51:35.874494 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:35.874455    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:51:35.874568 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:35.874549    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:51:35.874610 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:35.874587    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:51:36.882660 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:36.882613    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm" event={"ID":"c7dc0999-26b8-4201-8230-713d58ef3b16","Type":"ContainerStarted","Data":"ed47b320c0f059223b8f5b6d8012fcdc0ae33dee4a5fb0eface5674d141d610d"}
Jul 26 18:51:37.036408 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:37.036365    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 18:51:37.036655 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:37.036634    2576 patch_prober.go:28] interesting pod/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 26 18:51:37.036702 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:37.036683    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm" podUID="c7dc0999-26b8-4201-8230-713d58ef3b16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 26 18:51:38.036375 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:38.036335    2576 patch_prober.go:28] interesting pod/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 26 18:51:38.036865 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:38.036408    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm" podUID="c7dc0999-26b8-4201-8230-713d58ef3b16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 26 18:51:39.035920 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:39.035876    2576 patch_prober.go:28] interesting pod/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 26 18:51:39.036103 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:39.035936    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm" podUID="c7dc0999-26b8-4201-8230-713d58ef3b16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 26 18:51:39.175707 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:39.175662    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:51:39.200768 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:39.200711    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:51:40.035992 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:40.035948    2576 patch_prober.go:28] interesting pod/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 26 18:51:40.036169 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:40.036013    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm" podUID="c7dc0999-26b8-4201-8230-713d58ef3b16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 26 18:51:41.039175 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:41.039129    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 18:51:41.039783 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:41.039597    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 18:51:41.040286 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:41.040262    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 18:51:42.330899 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:42.330638    2576 patch_prober.go:28] interesting pod/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": read tcp 10.133.0.2:52932->10.133.0.53:15021: read: connection reset by peer" start-of-body=
Jul 26 18:51:42.330899 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:42.330707    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq" podUID="075468d1-5f16-4760-bc00-999ee7927f2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": read tcp 10.133.0.2:52932->10.133.0.53:15021: read: connection reset by peer"
Jul 26 18:51:42.915896 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:42.915864    2576 generic.go:356] "Generic (PLEG): container finished" podID="075468d1-5f16-4760-bc00-999ee7927f2b" containerID="f262c7a755a7b72229d6aa9b047e054e675f58d416f760fc9f1d81180f139108" exitCode=0
Jul 26 18:51:42.916086 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:42.915944    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq" event={"ID":"075468d1-5f16-4760-bc00-999ee7927f2b","Type":"ContainerDied","Data":"f262c7a755a7b72229d6aa9b047e054e675f58d416f760fc9f1d81180f139108"}
Jul 26 18:51:42.916377 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:42.916351    2576 scope.go:117] "RemoveContainer" containerID="f262c7a755a7b72229d6aa9b047e054e675f58d416f760fc9f1d81180f139108"
Jul 26 18:51:42.917325 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:42.917300    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:51:42.917379 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:42.917368    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:51:42.917414 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:42.917403    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:51:43.923236 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:43.923203    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq" event={"ID":"075468d1-5f16-4760-bc00-999ee7927f2b","Type":"ContainerStarted","Data":"9ee138b6cb9088591eec4ae381cff8111881867f4a229fdd8daf3b72989641d3"}
Jul 26 18:51:44.025582 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:44.025546    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 18:51:44.025840 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:44.025816    2576 patch_prober.go:28] interesting pod/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Jul 26 18:51:44.025903 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:44.025871    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq" podUID="075468d1-5f16-4760-bc00-999ee7927f2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Jul 26 18:51:45.025571 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:45.025528    2576 patch_prober.go:28] interesting pod/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Jul 26 18:51:45.025985 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:45.025591    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq" podUID="075468d1-5f16-4760-bc00-999ee7927f2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Jul 26 18:51:46.025467 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:46.025424    2576 patch_prober.go:28] interesting pod/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Jul 26 18:51:46.025658 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:46.025482    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq" podUID="075468d1-5f16-4760-bc00-999ee7927f2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Jul 26 18:51:47.025560 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:47.025520    2576 patch_prober.go:28] interesting pod/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Jul 26 18:51:47.025786 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:47.025580    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq" podUID="075468d1-5f16-4760-bc00-999ee7927f2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Jul 26 18:51:48.029415 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:48.029377    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 18:51:48.029822 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:48.029605    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 18:51:48.030411 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:48.030393    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 18:51:51.334406 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:51.334372    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:51:54.584147 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:51:54.584109    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:52:14.125108 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:52:14.125076    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:52:17.882626 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:52:17.882529    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:52:43.786552 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:52:43.786513    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:53:04.288089 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:53:04.288051    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:53:21.688054 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:53:21.688014    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:53:49.394847 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:53:49.394759    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:54:07.281635 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:54:07.281595    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:54:30.388377 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:54:30.388343    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:54:40.461279 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:54:40.461241    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:54:49.090580 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:54:49.090543    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:55:10.186591 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:55:10.186553    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:55:25.075700 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:55:25.075624    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:55:27.984815 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:55:27.984773    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:55:55.182297 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:55:55.182260    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:56:08.389183 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:56:08.389149    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:56:19.894173 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:56:19.894139    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:56:38.283638 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:56:38.283600    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:56:41.117801 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:56:41.117764    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:56:42.022181 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:56:42.022141    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:56:46.785255 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:56:46.785218    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:56:52.890482 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:56:52.890448    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:57:00.745652 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:57:00.745617    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:57:05.390057 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:57:05.390019    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:57:14.895856 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:57:14.895818    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:57:38.486493 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:57:38.486453    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:57:57.086644 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:57:57.086603    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:58:11.785535 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:58:11.785494    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:58:18.303736 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:58:18.303649    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:58:19.192696 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:58:19.192665    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:58:29.284022 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:58:29.283989    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:58:41.696502 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:58:41.696466    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:58:45.080148 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:58:45.080112    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:59:00.127819 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:59:00.127782    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 18:59:03.189775 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:59:03.189724    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:59:37.284941 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:59:37.284903    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 18:59:51.882300 ip-10-0-134-7 kubenswrapper[2576]: I0726 18:59:51.882253    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 19:00:00.136183 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:00.136149    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751540-khhq6"]
Jul 26 19:00:00.139308 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:00.139286    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2b193e-29751540-h45vf"]
Jul 26 19:00:00.139446 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:00.139415    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751540-khhq6"
Jul 26 19:00:00.142195 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:00.142171    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxnsx\""
Jul 26 19:00:00.142345 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:00.142274    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c75d5a-29751540-txbrp"]
Jul 26 19:00:00.142409 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:00.142346    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2b193e-29751540-h45vf"
Jul 26 19:00:00.145377 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:00.145358    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c75d5a-29751540-txbrp"
Jul 26 19:00:00.148470 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:00.148451    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751540-khhq6"]
Jul 26 19:00:00.151895 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:00.151704    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751540-khhq6"
Jul 26 19:00:00.152137 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:00.152119    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2b193e-29751540-h45vf"]
Jul 26 19:00:00.154190 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:00.154136    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c75d5a-29751540-txbrp"]
Jul 26 19:00:00.164943 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:00.164917    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2b193e-29751540-h45vf"
Jul 26 19:00:00.197085 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:00.197056    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c75d5a-29751540-txbrp"
Jul 26 19:00:00.322404 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:00.321212    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751540-khhq6"]
Jul 26 19:00:00.324984 ip-10-0-134-7 kubenswrapper[2576]: W0726 19:00:00.324936    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod542390b1_5679_4ae7_9520_cd4e847bb932.slice/crio-6f6f90caa0d24065702758c0aa3a2aad91c24a4b76e02d5677c7ecada71a23b1 WatchSource:0}: Error finding container 6f6f90caa0d24065702758c0aa3a2aad91c24a4b76e02d5677c7ecada71a23b1: Status 404 returned error can't find the container with id 6f6f90caa0d24065702758c0aa3a2aad91c24a4b76e02d5677c7ecada71a23b1
Jul 26 19:00:00.363844 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:00.363820    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2b193e-29751540-h45vf"]
Jul 26 19:00:00.366523 ip-10-0-134-7 kubenswrapper[2576]: W0726 19:00:00.366490    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb6d04c3_21c5_40de_839b_24e220dae207.slice/crio-ed2229bab9bc10e125daf9204ae05a8c06fc1d027db8658ef63b0c609a4d198e WatchSource:0}: Error finding container ed2229bab9bc10e125daf9204ae05a8c06fc1d027db8658ef63b0c609a4d198e: Status 404 returned error can't find the container with id ed2229bab9bc10e125daf9204ae05a8c06fc1d027db8658ef63b0c609a4d198e
Jul 26 19:00:00.388472 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:00.388392    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c75d5a-29751540-txbrp"]
Jul 26 19:00:00.399962 ip-10-0-134-7 kubenswrapper[2576]: W0726 19:00:00.399934    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cf20171_4fd9_4814_85c6_8b9375e0d346.slice/crio-598bfcfd070befe91bf9d19709725d7f921afb5f579e4431e4f10dab11ea46e4 WatchSource:0}: Error finding container 598bfcfd070befe91bf9d19709725d7f921afb5f579e4431e4f10dab11ea46e4: Status 404 returned error can't find the container with id 598bfcfd070befe91bf9d19709725d7f921afb5f579e4431e4f10dab11ea46e4
Jul 26 19:00:01.102994 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:01.102952    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2b193e-29751540-h45vf" event={"ID":"cb6d04c3-21c5-40de-839b-24e220dae207","Type":"ContainerStarted","Data":"17a1e35bb6c99466acd73f1d862569a3a408b516043458cc193c42d34568bc50"}
Jul 26 19:00:01.103221 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:01.103046    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2b193e-29751540-h45vf" event={"ID":"cb6d04c3-21c5-40de-839b-24e220dae207","Type":"ContainerStarted","Data":"ed2229bab9bc10e125daf9204ae05a8c06fc1d027db8658ef63b0c609a4d198e"}
Jul 26 19:00:01.104496 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:01.104469    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c75d5a-29751540-txbrp" event={"ID":"0cf20171-4fd9-4814-85c6-8b9375e0d346","Type":"ContainerStarted","Data":"6c89105ce92bf1c6f849107c5c533bf7ca67dba089d7141912f8053e621e534a"}
Jul 26 19:00:01.104612 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:01.104517    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c75d5a-29751540-txbrp" event={"ID":"0cf20171-4fd9-4814-85c6-8b9375e0d346","Type":"ContainerStarted","Data":"598bfcfd070befe91bf9d19709725d7f921afb5f579e4431e4f10dab11ea46e4"}
Jul 26 19:00:01.105982 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:01.105959    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751540-khhq6" event={"ID":"542390b1-5679-4ae7-9520-cd4e847bb932","Type":"ContainerStarted","Data":"109487f885cd9113c74a677ee8387c39794bf7cae78c8841c469e66c0843a894"}
Jul 26 19:00:01.106056 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:01.106005    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751540-khhq6" event={"ID":"542390b1-5679-4ae7-9520-cd4e847bb932","Type":"ContainerStarted","Data":"6f6f90caa0d24065702758c0aa3a2aad91c24a4b76e02d5677c7ecada71a23b1"}
Jul 26 19:00:01.118533 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:01.118486    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2b193e-29751540-h45vf" podStartSLOduration=1.118471187 podStartE2EDuration="1.118471187s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 19:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 19:00:01.117401741 +0000 UTC m=+2886.824808802" watchObservedRunningTime="2026-07-26 19:00:01.118471187 +0000 UTC m=+2886.825878223"
Jul 26 19:00:01.132277 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:01.132233    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c75d5a-29751540-txbrp" podStartSLOduration=1.132221502 podStartE2EDuration="1.132221502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 19:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 19:00:01.131537103 +0000 UTC m=+2886.838944211" watchObservedRunningTime="2026-07-26 19:00:01.132221502 +0000 UTC m=+2886.839628560"
Jul 26 19:00:01.148906 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:01.148858    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751540-khhq6" podStartSLOduration=1.148844688 podStartE2EDuration="1.148844688s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 19:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 19:00:01.147065784 +0000 UTC m=+2886.854472843" watchObservedRunningTime="2026-07-26 19:00:01.148844688 +0000 UTC m=+2886.856251747"
Jul 26 19:00:02.111383 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:02.111343    2576 generic.go:356] "Generic (PLEG): container finished" podID="cb6d04c3-21c5-40de-839b-24e220dae207" containerID="17a1e35bb6c99466acd73f1d862569a3a408b516043458cc193c42d34568bc50" exitCode=0
Jul 26 19:00:02.111586 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:02.111385    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2b193e-29751540-h45vf" event={"ID":"cb6d04c3-21c5-40de-839b-24e220dae207","Type":"ContainerDied","Data":"17a1e35bb6c99466acd73f1d862569a3a408b516043458cc193c42d34568bc50"}
Jul 26 19:00:02.112960 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:02.112938    2576 generic.go:356] "Generic (PLEG): container finished" podID="0cf20171-4fd9-4814-85c6-8b9375e0d346" containerID="6c89105ce92bf1c6f849107c5c533bf7ca67dba089d7141912f8053e621e534a" exitCode=0
Jul 26 19:00:02.113083 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:02.113006    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c75d5a-29751540-txbrp" event={"ID":"0cf20171-4fd9-4814-85c6-8b9375e0d346","Type":"ContainerDied","Data":"6c89105ce92bf1c6f849107c5c533bf7ca67dba089d7141912f8053e621e534a"}
Jul 26 19:00:02.114391 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:02.114364    2576 generic.go:356] "Generic (PLEG): container finished" podID="542390b1-5679-4ae7-9520-cd4e847bb932" containerID="109487f885cd9113c74a677ee8387c39794bf7cae78c8841c469e66c0843a894" exitCode=0
Jul 26 19:00:02.114479 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:02.114389    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751540-khhq6" event={"ID":"542390b1-5679-4ae7-9520-cd4e847bb932","Type":"ContainerDied","Data":"109487f885cd9113c74a677ee8387c39794bf7cae78c8841c469e66c0843a894"}
Jul 26 19:00:03.262189 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:03.262162    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c75d5a-29751540-txbrp"
Jul 26 19:00:03.318401 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:03.318376    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751540-khhq6"
Jul 26 19:00:03.321066 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:03.321048    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2b193e-29751540-h45vf"
Jul 26 19:00:04.125619 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:04.125590    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2b193e-29751540-h45vf"
Jul 26 19:00:04.125808 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:04.125589    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2b193e-29751540-h45vf" event={"ID":"cb6d04c3-21c5-40de-839b-24e220dae207","Type":"ContainerDied","Data":"ed2229bab9bc10e125daf9204ae05a8c06fc1d027db8658ef63b0c609a4d198e"}
Jul 26 19:00:04.125808 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:04.125720    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ed2229bab9bc10e125daf9204ae05a8c06fc1d027db8658ef63b0c609a4d198e"
Jul 26 19:00:04.126886 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:04.126866    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c75d5a-29751540-txbrp"
Jul 26 19:00:04.127011 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:04.126890    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c75d5a-29751540-txbrp" event={"ID":"0cf20171-4fd9-4814-85c6-8b9375e0d346","Type":"ContainerDied","Data":"598bfcfd070befe91bf9d19709725d7f921afb5f579e4431e4f10dab11ea46e4"}
Jul 26 19:00:04.127011 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:04.126932    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="598bfcfd070befe91bf9d19709725d7f921afb5f579e4431e4f10dab11ea46e4"
Jul 26 19:00:04.128134 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:04.128117    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751540-khhq6"
Jul 26 19:00:04.128222 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:04.128172    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751540-khhq6" event={"ID":"542390b1-5679-4ae7-9520-cd4e847bb932","Type":"ContainerDied","Data":"6f6f90caa0d24065702758c0aa3a2aad91c24a4b76e02d5677c7ecada71a23b1"}
Jul 26 19:00:04.128222 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:04.128207    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6f6f90caa0d24065702758c0aa3a2aad91c24a4b76e02d5677c7ecada71a23b1"
Jul 26 19:00:06.583333 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:06.583292    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 19:00:10.171535 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:10.171498    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:00:11.729387 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:11.729348    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:00:23.183528 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:23.183488    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 19:00:25.193721 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:25.193672    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:00:25.247913 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:25.247876    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:00:30.146377 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:30.146340    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:00:30.454430 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:30.454392    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:00:30.805713 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:30.805634    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:00:33.411988 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:33.411955    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:00:33.510674 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:33.510632    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:00:33.512369 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:33.512345    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh"]
Jul 26 19:00:33.512841 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:33.512826    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb6d04c3-21c5-40de-839b-24e220dae207" containerName="cleanup"
Jul 26 19:00:33.512890 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:33.512844    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb6d04c3-21c5-40de-839b-24e220dae207" containerName="cleanup"
Jul 26 19:00:33.512890 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:33.512873    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="542390b1-5679-4ae7-9520-cd4e847bb932" containerName="cleanup"
Jul 26 19:00:33.512890 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:33.512882    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="542390b1-5679-4ae7-9520-cd4e847bb932" containerName="cleanup"
Jul 26 19:00:33.512982 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:33.512898    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cf20171-4fd9-4814-85c6-8b9375e0d346" containerName="cleanup"
Jul 26 19:00:33.512982 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:33.512908    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cf20171-4fd9-4814-85c6-8b9375e0d346" containerName="cleanup"
Jul 26 19:00:33.513045 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:33.512988    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cf20171-4fd9-4814-85c6-8b9375e0d346" containerName="cleanup"
Jul 26 19:00:33.513045 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:33.513000    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb6d04c3-21c5-40de-839b-24e220dae207" containerName="cleanup"
Jul 26 19:00:33.513045 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:33.513010    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="542390b1-5679-4ae7-9520-cd4e847bb932" containerName="cleanup"
Jul 26 19:00:33.517540 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:33.517520    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh"
Jul 26 19:00:33.520683 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:33.520664    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-4735db-serving-cert\""
Jul 26 19:00:33.528006 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:33.527986    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh"]
Jul 26 19:00:33.592519 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:33.592484    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:00:33.603306 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:33.603269    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08ff5eed-7b25-49c1-b9c2-898d9a3a284a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh\" (UID: \"08ff5eed-7b25-49c1-b9c2-898d9a3a284a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh"
Jul 26 19:00:33.603472 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:33.603326    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08ff5eed-7b25-49c1-b9c2-898d9a3a284a-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh\" (UID: \"08ff5eed-7b25-49c1-b9c2-898d9a3a284a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh"
Jul 26 19:00:33.603472 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:33.603432    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgmbl\" (UniqueName: \"kubernetes.io/projected/08ff5eed-7b25-49c1-b9c2-898d9a3a284a-kube-api-access-bgmbl\") pod \"maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh\" (UID: \"08ff5eed-7b25-49c1-b9c2-898d9a3a284a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh"
Jul 26 19:00:33.709769 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:33.705248    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08ff5eed-7b25-49c1-b9c2-898d9a3a284a-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh\" (UID: \"08ff5eed-7b25-49c1-b9c2-898d9a3a284a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh"
Jul 26 19:00:33.709769 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:33.705384    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgmbl\" (UniqueName: \"kubernetes.io/projected/08ff5eed-7b25-49c1-b9c2-898d9a3a284a-kube-api-access-bgmbl\") pod \"maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh\" (UID: \"08ff5eed-7b25-49c1-b9c2-898d9a3a284a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh"
Jul 26 19:00:33.709769 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:33.705436    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08ff5eed-7b25-49c1-b9c2-898d9a3a284a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh\" (UID: \"08ff5eed-7b25-49c1-b9c2-898d9a3a284a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh"
Jul 26 19:00:33.709769 ip-10-0-134-7 kubenswrapper[2576]: E0726 19:00:33.705618    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-4735db-serving-cert: secret "maas-api-e2e-ipp-a-4735db-serving-cert" not found
Jul 26 19:00:33.709769 ip-10-0-134-7 kubenswrapper[2576]: E0726 19:00:33.705725    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08ff5eed-7b25-49c1-b9c2-898d9a3a284a-maas-api-tls podName:08ff5eed-7b25-49c1-b9c2-898d9a3a284a nodeName:}" failed. No retries permitted until 2026-07-26 19:00:34.205703419 +0000 UTC m=+2919.913110463 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/08ff5eed-7b25-49c1-b9c2-898d9a3a284a-maas-api-tls") pod "maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh" (UID: "08ff5eed-7b25-49c1-b9c2-898d9a3a284a") : secret "maas-api-e2e-ipp-a-4735db-serving-cert" not found
Jul 26 19:00:33.709769 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:33.707593    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08ff5eed-7b25-49c1-b9c2-898d9a3a284a-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh\" (UID: \"08ff5eed-7b25-49c1-b9c2-898d9a3a284a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh"
Jul 26 19:00:33.716263 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:33.716234    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgmbl\" (UniqueName: \"kubernetes.io/projected/08ff5eed-7b25-49c1-b9c2-898d9a3a284a-kube-api-access-bgmbl\") pod \"maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh\" (UID: \"08ff5eed-7b25-49c1-b9c2-898d9a3a284a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh"
Jul 26 19:00:33.900148 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:33.900114    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:00:34.189364 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:34.189325    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:00:34.210563 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:34.210531    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08ff5eed-7b25-49c1-b9c2-898d9a3a284a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh\" (UID: \"08ff5eed-7b25-49c1-b9c2-898d9a3a284a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh"
Jul 26 19:00:34.213159 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:34.213128    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08ff5eed-7b25-49c1-b9c2-898d9a3a284a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh\" (UID: \"08ff5eed-7b25-49c1-b9c2-898d9a3a284a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh"
Jul 26 19:00:34.303367 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:34.303337    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:00:34.429437 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:34.429405    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh"
Jul 26 19:00:34.560685 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:34.560648    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh"]
Jul 26 19:00:34.563457 ip-10-0-134-7 kubenswrapper[2576]: W0726 19:00:34.563427    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08ff5eed_7b25_49c1_b9c2_898d9a3a284a.slice/crio-ee30a737f0b2f29ce92288ff549518ff2508b491fc6546f207aeb399fe9c3c7a WatchSource:0}: Error finding container ee30a737f0b2f29ce92288ff549518ff2508b491fc6546f207aeb399fe9c3c7a: Status 404 returned error can't find the container with id ee30a737f0b2f29ce92288ff549518ff2508b491fc6546f207aeb399fe9c3c7a
Jul 26 19:00:34.564864 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:34.564844    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 19:00:35.268686 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:35.268589    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh" event={"ID":"08ff5eed-7b25-49c1-b9c2-898d9a3a284a","Type":"ContainerStarted","Data":"3255f400c503d34bbf423fb77bc2cf75af6e7a91934be5dfe5aad848a4e6cb22"}
Jul 26 19:00:35.268686 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:35.268648    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh" event={"ID":"08ff5eed-7b25-49c1-b9c2-898d9a3a284a","Type":"ContainerStarted","Data":"ee30a737f0b2f29ce92288ff549518ff2508b491fc6546f207aeb399fe9c3c7a"}
Jul 26 19:00:35.268921 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:35.268729    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh"
Jul 26 19:00:35.284585 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:35.284536    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh" podStartSLOduration=1.8843404750000001 podStartE2EDuration="2.284520861s" totalImagesPullingTime="400.180386ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 19:00:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 19:00:35.28351526 +0000 UTC m=+2920.990922320" watchObservedRunningTime="2026-07-26 19:00:35.284520861 +0000 UTC m=+2920.991927970"
Jul 26 19:00:37.898901 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:37.898867    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:00:41.277957 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:41.277919    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh"
Jul 26 19:00:42.187765 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:42.187722    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 19:00:59.033552 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:59.033520    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:00:59.402802 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:59.402693    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:00:59.532362 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:00:59.532330    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:01:02.614932 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:01:02.614889    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:01:02.711304 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:01:02.711274    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:01:02.791389 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:01:02.791349    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9"]
Jul 26 19:01:02.795663 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:01:02.795642    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9"
Jul 26 19:01:02.798340 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:01:02.798313    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-1318a7-serving-cert\""
Jul 26 19:01:02.805759 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:01:02.805719    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9"]
Jul 26 19:01:02.833565 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:01:02.833532    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:01:02.840664 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:01:02.840639    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db2b53c2-9568-4b09-bc4a-d484802e4901-maas-api-tls\") pod \"maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9\" (UID: \"db2b53c2-9568-4b09-bc4a-d484802e4901\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9"
Jul 26 19:01:02.840800 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:01:02.840711    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db2b53c2-9568-4b09-bc4a-d484802e4901-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9\" (UID: \"db2b53c2-9568-4b09-bc4a-d484802e4901\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9"
Jul 26 19:01:02.840800 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:01:02.840737    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kndv\" (UniqueName: \"kubernetes.io/projected/db2b53c2-9568-4b09-bc4a-d484802e4901-kube-api-access-7kndv\") pod \"maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9\" (UID: \"db2b53c2-9568-4b09-bc4a-d484802e4901\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9"
Jul 26 19:01:02.941705 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:01:02.941671    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db2b53c2-9568-4b09-bc4a-d484802e4901-maas-api-tls\") pod \"maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9\" (UID: \"db2b53c2-9568-4b09-bc4a-d484802e4901\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9"
Jul 26 19:01:02.941955 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:01:02.941737    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db2b53c2-9568-4b09-bc4a-d484802e4901-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9\" (UID: \"db2b53c2-9568-4b09-bc4a-d484802e4901\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9"
Jul 26 19:01:02.941955 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:01:02.941800    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kndv\" (UniqueName: \"kubernetes.io/projected/db2b53c2-9568-4b09-bc4a-d484802e4901-kube-api-access-7kndv\") pod \"maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9\" (UID: \"db2b53c2-9568-4b09-bc4a-d484802e4901\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9"
Jul 26 19:01:02.941955 ip-10-0-134-7 kubenswrapper[2576]: E0726 19:01:02.941862    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-1318a7-serving-cert: secret "maas-api-e2e-ipp-b-1318a7-serving-cert" not found
Jul 26 19:01:02.941955 ip-10-0-134-7 kubenswrapper[2576]: E0726 19:01:02.941954    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db2b53c2-9568-4b09-bc4a-d484802e4901-maas-api-tls podName:db2b53c2-9568-4b09-bc4a-d484802e4901 nodeName:}" failed. No retries permitted until 2026-07-26 19:01:03.441931836 +0000 UTC m=+2949.149338888 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/db2b53c2-9568-4b09-bc4a-d484802e4901-maas-api-tls") pod "maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9" (UID: "db2b53c2-9568-4b09-bc4a-d484802e4901") : secret "maas-api-e2e-ipp-b-1318a7-serving-cert" not found
Jul 26 19:01:02.942600 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:01:02.942577    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db2b53c2-9568-4b09-bc4a-d484802e4901-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9\" (UID: \"db2b53c2-9568-4b09-bc4a-d484802e4901\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9"
Jul 26 19:01:02.950454 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:01:02.950426    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kndv\" (UniqueName: \"kubernetes.io/projected/db2b53c2-9568-4b09-bc4a-d484802e4901-kube-api-access-7kndv\") pod \"maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9\" (UID: \"db2b53c2-9568-4b09-bc4a-d484802e4901\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9"
Jul 26 19:01:03.122150 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:01:03.122107    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:01:03.402934 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:01:03.402849    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:01:03.446316 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:01:03.446263    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db2b53c2-9568-4b09-bc4a-d484802e4901-maas-api-tls\") pod \"maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9\" (UID: \"db2b53c2-9568-4b09-bc4a-d484802e4901\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9"
Jul 26 19:01:03.448979 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:01:03.448957    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db2b53c2-9568-4b09-bc4a-d484802e4901-maas-api-tls\") pod \"maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9\" (UID: \"db2b53c2-9568-4b09-bc4a-d484802e4901\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9"
Jul 26 19:01:03.502708 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:01:03.502677    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:01:03.709525 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:01:03.709494    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9"
Jul 26 19:01:03.846637 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:01:03.846609    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9"]
Jul 26 19:01:03.849284 ip-10-0-134-7 kubenswrapper[2576]: W0726 19:01:03.849256    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb2b53c2_9568_4b09_bc4a_d484802e4901.slice/crio-e1ff12ff6f21bf769475f54d561b44dccf2643b15b7b088be3dae18e8aac29a6 WatchSource:0}: Error finding container e1ff12ff6f21bf769475f54d561b44dccf2643b15b7b088be3dae18e8aac29a6: Status 404 returned error can't find the container with id e1ff12ff6f21bf769475f54d561b44dccf2643b15b7b088be3dae18e8aac29a6
Jul 26 19:01:04.407639 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:01:04.407553    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9" event={"ID":"db2b53c2-9568-4b09-bc4a-d484802e4901","Type":"ContainerStarted","Data":"485032d52e4a09d16c91b5f7451d3337e73b00a9b60e5249caa7735bb7bab7b8"}
Jul 26 19:01:04.407639 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:01:04.407613    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9" event={"ID":"db2b53c2-9568-4b09-bc4a-d484802e4901","Type":"ContainerStarted","Data":"e1ff12ff6f21bf769475f54d561b44dccf2643b15b7b088be3dae18e8aac29a6"}
Jul 26 19:01:04.407850 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:01:04.407674    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9"
Jul 26 19:01:04.423037 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:01:04.422926    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9" podStartSLOduration=2.133172388 podStartE2EDuration="2.422905582s" totalImagesPullingTime="289.733194ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 19:01:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 19:01:04.422186162 +0000 UTC m=+2950.129593224" watchObservedRunningTime="2026-07-26 19:01:04.422905582 +0000 UTC m=+2950.130312642"
Jul 26 19:01:07.418036 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:01:07.417996    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:01:10.417781 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:01:10.417734    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9"
Jul 26 19:01:12.181576 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:01:12.181539    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 19:01:22.616901 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:01:22.616809    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:01:27.727387 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:01:27.727322    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:01:30.994469 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:01:30.994432    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 19:01:39.901205 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:01:39.901168    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:01:39.954854 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:01:39.954823    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:02:01.187899 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:01.187857    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 19:02:05.265355 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:05.265320    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:02:05.368308 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:05.368273    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:02:11.563201 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:11.563161    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:02:11.908312 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:11.908213    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"]
Jul 26 19:02:11.912049 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:11.912027    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:11.916039 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:11.916012    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-9eba87-openshift-default-dockercfg-jv8tc\""
Jul 26 19:02:11.935683 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:11.935654    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"]
Jul 26 19:02:11.963540 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:11.963508    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:02:12.035687 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:12.035646    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b31eb70c-d65a-4166-97ee-c90b3911eb79-istio-podinfo\") pod \"e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x\" (UID: \"b31eb70c-d65a-4166-97ee-c90b3911eb79\") " pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:12.035893 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:12.035698    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gk6wz\" (UniqueName: \"kubernetes.io/projected/b31eb70c-d65a-4166-97ee-c90b3911eb79-kube-api-access-gk6wz\") pod \"e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x\" (UID: \"b31eb70c-d65a-4166-97ee-c90b3911eb79\") " pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:12.035893 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:12.035719    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b31eb70c-d65a-4166-97ee-c90b3911eb79-credential-socket\") pod \"e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x\" (UID: \"b31eb70c-d65a-4166-97ee-c90b3911eb79\") " pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:12.035893 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:12.035799    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b31eb70c-d65a-4166-97ee-c90b3911eb79-istio-envoy\") pod \"e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x\" (UID: \"b31eb70c-d65a-4166-97ee-c90b3911eb79\") " pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:12.035893 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:12.035847    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b31eb70c-d65a-4166-97ee-c90b3911eb79-istio-data\") pod \"e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x\" (UID: \"b31eb70c-d65a-4166-97ee-c90b3911eb79\") " pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:12.036079 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:12.035912    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b31eb70c-d65a-4166-97ee-c90b3911eb79-istio-token\") pod \"e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x\" (UID: \"b31eb70c-d65a-4166-97ee-c90b3911eb79\") " pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:12.036079 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:12.035949    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b31eb70c-d65a-4166-97ee-c90b3911eb79-workload-certs\") pod \"e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x\" (UID: \"b31eb70c-d65a-4166-97ee-c90b3911eb79\") " pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:12.036079 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:12.035979    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b31eb70c-d65a-4166-97ee-c90b3911eb79-workload-socket\") pod \"e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x\" (UID: \"b31eb70c-d65a-4166-97ee-c90b3911eb79\") " pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:12.036079 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:12.036014    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b31eb70c-d65a-4166-97ee-c90b3911eb79-istiod-ca-cert\") pod \"e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x\" (UID: \"b31eb70c-d65a-4166-97ee-c90b3911eb79\") " pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:12.061860 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:12.061824    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:02:12.137099 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:12.137055    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b31eb70c-d65a-4166-97ee-c90b3911eb79-istio-podinfo\") pod \"e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x\" (UID: \"b31eb70c-d65a-4166-97ee-c90b3911eb79\") " pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:12.137306 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:12.137123    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gk6wz\" (UniqueName: \"kubernetes.io/projected/b31eb70c-d65a-4166-97ee-c90b3911eb79-kube-api-access-gk6wz\") pod \"e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x\" (UID: \"b31eb70c-d65a-4166-97ee-c90b3911eb79\") " pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:12.137306 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:12.137160    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b31eb70c-d65a-4166-97ee-c90b3911eb79-credential-socket\") pod \"e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x\" (UID: \"b31eb70c-d65a-4166-97ee-c90b3911eb79\") " pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:12.137306 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:12.137188    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b31eb70c-d65a-4166-97ee-c90b3911eb79-istio-envoy\") pod \"e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x\" (UID: \"b31eb70c-d65a-4166-97ee-c90b3911eb79\") " pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:12.137588 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:12.137551    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b31eb70c-d65a-4166-97ee-c90b3911eb79-istio-data\") pod \"e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x\" (UID: \"b31eb70c-d65a-4166-97ee-c90b3911eb79\") " pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:12.137714 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:12.137611    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b31eb70c-d65a-4166-97ee-c90b3911eb79-istio-token\") pod \"e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x\" (UID: \"b31eb70c-d65a-4166-97ee-c90b3911eb79\") " pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:12.137714 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:12.137652    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b31eb70c-d65a-4166-97ee-c90b3911eb79-workload-certs\") pod \"e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x\" (UID: \"b31eb70c-d65a-4166-97ee-c90b3911eb79\") " pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:12.137714 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:12.137674    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b31eb70c-d65a-4166-97ee-c90b3911eb79-workload-socket\") pod \"e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x\" (UID: \"b31eb70c-d65a-4166-97ee-c90b3911eb79\") " pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:12.137714 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:12.137694    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b31eb70c-d65a-4166-97ee-c90b3911eb79-credential-socket\") pod \"e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x\" (UID: \"b31eb70c-d65a-4166-97ee-c90b3911eb79\") " pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:12.137978 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:12.137881    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b31eb70c-d65a-4166-97ee-c90b3911eb79-istio-data\") pod \"e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x\" (UID: \"b31eb70c-d65a-4166-97ee-c90b3911eb79\") " pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:12.137978 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:12.137930    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b31eb70c-d65a-4166-97ee-c90b3911eb79-istiod-ca-cert\") pod \"e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x\" (UID: \"b31eb70c-d65a-4166-97ee-c90b3911eb79\") " pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:12.138080 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:12.137975    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b31eb70c-d65a-4166-97ee-c90b3911eb79-workload-socket\") pod \"e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x\" (UID: \"b31eb70c-d65a-4166-97ee-c90b3911eb79\") " pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:12.138080 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:12.138053    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b31eb70c-d65a-4166-97ee-c90b3911eb79-workload-certs\") pod \"e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x\" (UID: \"b31eb70c-d65a-4166-97ee-c90b3911eb79\") " pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:12.138507 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:12.138485    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b31eb70c-d65a-4166-97ee-c90b3911eb79-istiod-ca-cert\") pod \"e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x\" (UID: \"b31eb70c-d65a-4166-97ee-c90b3911eb79\") " pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:12.139914 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:12.139887    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b31eb70c-d65a-4166-97ee-c90b3911eb79-istio-podinfo\") pod \"e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x\" (UID: \"b31eb70c-d65a-4166-97ee-c90b3911eb79\") " pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:12.140116 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:12.140098    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b31eb70c-d65a-4166-97ee-c90b3911eb79-istio-envoy\") pod \"e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x\" (UID: \"b31eb70c-d65a-4166-97ee-c90b3911eb79\") " pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:12.145605 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:12.145577    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b31eb70c-d65a-4166-97ee-c90b3911eb79-istio-token\") pod \"e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x\" (UID: \"b31eb70c-d65a-4166-97ee-c90b3911eb79\") " pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:12.145971 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:12.145950    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gk6wz\" (UniqueName: \"kubernetes.io/projected/b31eb70c-d65a-4166-97ee-c90b3911eb79-kube-api-access-gk6wz\") pod \"e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x\" (UID: \"b31eb70c-d65a-4166-97ee-c90b3911eb79\") " pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:12.221871 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:12.221834    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:12.366717 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:12.366691    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"]
Jul 26 19:02:12.368836 ip-10-0-134-7 kubenswrapper[2576]: W0726 19:02:12.368797    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb31eb70c_d65a_4166_97ee_c90b3911eb79.slice/crio-11ceb74b7f757b97dd05c95fa1f54e3c10ef240752ba6acb4fec547bc9787f93 WatchSource:0}: Error finding container 11ceb74b7f757b97dd05c95fa1f54e3c10ef240752ba6acb4fec547bc9787f93: Status 404 returned error can't find the container with id 11ceb74b7f757b97dd05c95fa1f54e3c10ef240752ba6acb4fec547bc9787f93
Jul 26 19:02:12.371153 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:12.371118    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 19:02:12.371250 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:12.371234    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 19:02:12.371294 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:12.371280    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 19:02:12.715031 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:12.714997    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x" event={"ID":"b31eb70c-d65a-4166-97ee-c90b3911eb79","Type":"ContainerStarted","Data":"d2cacc3c3da7f439abf1aea7b6915b2c9fd155c95f15fd3674a93e45b52a9884"}
Jul 26 19:02:12.715520 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:12.715088    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x" event={"ID":"b31eb70c-d65a-4166-97ee-c90b3911eb79","Type":"ContainerStarted","Data":"11ceb74b7f757b97dd05c95fa1f54e3c10ef240752ba6acb4fec547bc9787f93"}
Jul 26 19:02:12.733491 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:12.733445    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x" podStartSLOduration=1.733431633 podStartE2EDuration="1.733431633s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 19:02:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 19:02:12.73308575 +0000 UTC m=+3018.440492834" watchObservedRunningTime="2026-07-26 19:02:12.733431633 +0000 UTC m=+3018.440838693"
Jul 26 19:02:13.222670 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:13.222627    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:13.227853 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:13.227829    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:13.720221 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:13.720192    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:13.721337 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:13.721315    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:14.626215 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:14.626177    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:02:14.708232 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:14.708194    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:02:14.787019 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:14.786988    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:02:15.242309 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:15.242265    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:02:15.456540 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:15.456509    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:02:15.556924 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:15.556840    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:02:15.599721 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:15.599688    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-9eba87-5b6f46794d-rbwhd"]
Jul 26 19:02:15.603085 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:15.603060    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9eba87-5b6f46794d-rbwhd"
Jul 26 19:02:15.606906 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:15.606882    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-9eba87-dockercfg-8brjk\""
Jul 26 19:02:15.607320 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:15.607298    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-9eba87\""
Jul 26 19:02:15.616669 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:15.616641    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-9eba87-5b6f46794d-rbwhd"]
Jul 26 19:02:15.671477 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:15.671345    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bdrg\" (UniqueName: \"kubernetes.io/projected/d4ee08d8-4614-40a5-b3e5-283fb9bb21b1-kube-api-access-4bdrg\") pod \"payload-processing-e2e-ipp-cleanup-9eba87-5b6f46794d-rbwhd\" (UID: \"d4ee08d8-4614-40a5-b3e5-283fb9bb21b1\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9eba87-5b6f46794d-rbwhd"
Jul 26 19:02:15.671477 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:15.671433    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4ee08d8-4614-40a5-b3e5-283fb9bb21b1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-9eba87-5b6f46794d-rbwhd\" (UID: \"d4ee08d8-4614-40a5-b3e5-283fb9bb21b1\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9eba87-5b6f46794d-rbwhd"
Jul 26 19:02:15.773065 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:15.773009    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bdrg\" (UniqueName: \"kubernetes.io/projected/d4ee08d8-4614-40a5-b3e5-283fb9bb21b1-kube-api-access-4bdrg\") pod \"payload-processing-e2e-ipp-cleanup-9eba87-5b6f46794d-rbwhd\" (UID: \"d4ee08d8-4614-40a5-b3e5-283fb9bb21b1\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9eba87-5b6f46794d-rbwhd"
Jul 26 19:02:15.773261 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:15.773093    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4ee08d8-4614-40a5-b3e5-283fb9bb21b1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-9eba87-5b6f46794d-rbwhd\" (UID: \"d4ee08d8-4614-40a5-b3e5-283fb9bb21b1\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9eba87-5b6f46794d-rbwhd"
Jul 26 19:02:15.773853 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:15.773832    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4ee08d8-4614-40a5-b3e5-283fb9bb21b1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-9eba87-5b6f46794d-rbwhd\" (UID: \"d4ee08d8-4614-40a5-b3e5-283fb9bb21b1\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9eba87-5b6f46794d-rbwhd"
Jul 26 19:02:15.783381 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:15.783353    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bdrg\" (UniqueName: \"kubernetes.io/projected/d4ee08d8-4614-40a5-b3e5-283fb9bb21b1-kube-api-access-4bdrg\") pod \"payload-processing-e2e-ipp-cleanup-9eba87-5b6f46794d-rbwhd\" (UID: \"d4ee08d8-4614-40a5-b3e5-283fb9bb21b1\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9eba87-5b6f46794d-rbwhd"
Jul 26 19:02:15.915142 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:15.915047    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9eba87-5b6f46794d-rbwhd"
Jul 26 19:02:16.053726 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:16.053696    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-9eba87-5b6f46794d-rbwhd"]
Jul 26 19:02:16.055654 ip-10-0-134-7 kubenswrapper[2576]: W0726 19:02:16.055620    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4ee08d8_4614_40a5_b3e5_283fb9bb21b1.slice/crio-e9329ec29615151dbc3714d745463499532fb9428bee0cfc35d8a461b288189d WatchSource:0}: Error finding container e9329ec29615151dbc3714d745463499532fb9428bee0cfc35d8a461b288189d: Status 404 returned error can't find the container with id e9329ec29615151dbc3714d745463499532fb9428bee0cfc35d8a461b288189d
Jul 26 19:02:16.737374 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:16.737339    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9eba87-5b6f46794d-rbwhd" event={"ID":"d4ee08d8-4614-40a5-b3e5-283fb9bb21b1","Type":"ContainerStarted","Data":"f601afed40887f5975558a1736b7811890f3f8c2dc089888bdd34d19447fe0f5"}
Jul 26 19:02:16.737562 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:16.737391    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9eba87-5b6f46794d-rbwhd" event={"ID":"d4ee08d8-4614-40a5-b3e5-283fb9bb21b1","Type":"ContainerStarted","Data":"e9329ec29615151dbc3714d745463499532fb9428bee0cfc35d8a461b288189d"}
Jul 26 19:02:16.737562 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:16.737433    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9eba87-5b6f46794d-rbwhd"
Jul 26 19:02:16.756076 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:16.756018    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9eba87-5b6f46794d-rbwhd" podStartSLOduration=1.7560021319999999 podStartE2EDuration="1.756002132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 19:02:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 19:02:16.754187348 +0000 UTC m=+3022.461594407" watchObservedRunningTime="2026-07-26 19:02:16.756002132 +0000 UTC m=+3022.463409191"
Jul 26 19:02:19.112032 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:19.111993    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:02:19.696074 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:19.696039    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 19:02:23.014876 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:23.014836    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:02:27.744355 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:27.744327    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9eba87-5b6f46794d-rbwhd"
Jul 26 19:02:28.721532 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:28.721496    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 26 19:02:28.721739 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:28.721553    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x" podUID="b31eb70c-d65a-4166-97ee-c90b3911eb79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 26 19:02:28.801347 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:28.801313    2576 generic.go:356] "Generic (PLEG): container finished" podID="b31eb70c-d65a-4166-97ee-c90b3911eb79" containerID="d2cacc3c3da7f439abf1aea7b6915b2c9fd155c95f15fd3674a93e45b52a9884" exitCode=0
Jul 26 19:02:28.801770 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:28.801393    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x" event={"ID":"b31eb70c-d65a-4166-97ee-c90b3911eb79","Type":"ContainerDied","Data":"d2cacc3c3da7f439abf1aea7b6915b2c9fd155c95f15fd3674a93e45b52a9884"}
Jul 26 19:02:28.801830 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:28.801812    2576 scope.go:117] "RemoveContainer" containerID="d2cacc3c3da7f439abf1aea7b6915b2c9fd155c95f15fd3674a93e45b52a9884"
Jul 26 19:02:28.802962 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:28.802937    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 19:02:28.803034 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:28.803007    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 19:02:28.803071 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:28.803038    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 19:02:29.807954 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:29.807918    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x" event={"ID":"b31eb70c-d65a-4166-97ee-c90b3911eb79","Type":"ContainerStarted","Data":"be3d61bcc1f07cdc131c2dbb17a8d741c8f1605daf54e0969bcea345053dcd28"}
Jul 26 19:02:30.222329 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:30.222285    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:30.222599 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:30.222572    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 26 19:02:30.222646 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:30.222630    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x" podUID="b31eb70c-d65a-4166-97ee-c90b3911eb79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 26 19:02:31.223130 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:31.223090    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 26 19:02:31.223557 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:31.223162    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x" podUID="b31eb70c-d65a-4166-97ee-c90b3911eb79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 26 19:02:32.222356 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:32.222318    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 26 19:02:32.222530 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:32.222385    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x" podUID="b31eb70c-d65a-4166-97ee-c90b3911eb79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 26 19:02:33.223012 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:33.222965    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 26 19:02:33.223393 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:33.223058    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x" podUID="b31eb70c-d65a-4166-97ee-c90b3911eb79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 26 19:02:34.226393 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:34.226358    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:34.226869 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:34.226739    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:34.227574 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:34.227555    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:40.417471 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:40.417438    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:02:40.435935 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:40.435901    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-9eba87-5b6f46794d-rbwhd"]
Jul 26 19:02:40.436165 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:40.436119    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9eba87-5b6f46794d-rbwhd" podUID="d4ee08d8-4614-40a5-b3e5-283fb9bb21b1" containerName="payload-processing" containerID="cri-o://f601afed40887f5975558a1736b7811890f3f8c2dc089888bdd34d19447fe0f5" gracePeriod=30
Jul 26 19:02:40.518517 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:40.518487    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:02:40.569951 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:40.569916    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:02:40.703451 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:40.703425    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9eba87-5b6f46794d-rbwhd"
Jul 26 19:02:40.792382 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:40.792340    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4ee08d8-4614-40a5-b3e5-283fb9bb21b1-plugins-config-volume\") pod \"d4ee08d8-4614-40a5-b3e5-283fb9bb21b1\" (UID: \"d4ee08d8-4614-40a5-b3e5-283fb9bb21b1\") "
Jul 26 19:02:40.792592 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:40.792426    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4bdrg\" (UniqueName: \"kubernetes.io/projected/d4ee08d8-4614-40a5-b3e5-283fb9bb21b1-kube-api-access-4bdrg\") pod \"d4ee08d8-4614-40a5-b3e5-283fb9bb21b1\" (UID: \"d4ee08d8-4614-40a5-b3e5-283fb9bb21b1\") "
Jul 26 19:02:40.792793 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:40.792736    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4ee08d8-4614-40a5-b3e5-283fb9bb21b1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d4ee08d8-4614-40a5-b3e5-283fb9bb21b1" (UID: "d4ee08d8-4614-40a5-b3e5-283fb9bb21b1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 19:02:40.794571 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:40.794543    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4ee08d8-4614-40a5-b3e5-283fb9bb21b1-kube-api-access-4bdrg" (OuterVolumeSpecName: "kube-api-access-4bdrg") pod "d4ee08d8-4614-40a5-b3e5-283fb9bb21b1" (UID: "d4ee08d8-4614-40a5-b3e5-283fb9bb21b1"). InnerVolumeSpecName "kube-api-access-4bdrg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 19:02:40.861873 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:40.861840    2576 generic.go:356] "Generic (PLEG): container finished" podID="d4ee08d8-4614-40a5-b3e5-283fb9bb21b1" containerID="f601afed40887f5975558a1736b7811890f3f8c2dc089888bdd34d19447fe0f5" exitCode=0
Jul 26 19:02:40.862019 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:40.861882    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9eba87-5b6f46794d-rbwhd"
Jul 26 19:02:40.862019 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:40.861916    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9eba87-5b6f46794d-rbwhd" event={"ID":"d4ee08d8-4614-40a5-b3e5-283fb9bb21b1","Type":"ContainerDied","Data":"f601afed40887f5975558a1736b7811890f3f8c2dc089888bdd34d19447fe0f5"}
Jul 26 19:02:40.862019 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:40.861970    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9eba87-5b6f46794d-rbwhd" event={"ID":"d4ee08d8-4614-40a5-b3e5-283fb9bb21b1","Type":"ContainerDied","Data":"e9329ec29615151dbc3714d745463499532fb9428bee0cfc35d8a461b288189d"}
Jul 26 19:02:40.862019 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:40.861993    2576 scope.go:117] "RemoveContainer" containerID="f601afed40887f5975558a1736b7811890f3f8c2dc089888bdd34d19447fe0f5"
Jul 26 19:02:40.872375 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:40.872356    2576 scope.go:117] "RemoveContainer" containerID="f601afed40887f5975558a1736b7811890f3f8c2dc089888bdd34d19447fe0f5"
Jul 26 19:02:40.872667 ip-10-0-134-7 kubenswrapper[2576]: E0726 19:02:40.872645    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f601afed40887f5975558a1736b7811890f3f8c2dc089888bdd34d19447fe0f5\": container with ID starting with f601afed40887f5975558a1736b7811890f3f8c2dc089888bdd34d19447fe0f5 not found: ID does not exist" containerID="f601afed40887f5975558a1736b7811890f3f8c2dc089888bdd34d19447fe0f5"
Jul 26 19:02:40.872711 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:40.872680    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f601afed40887f5975558a1736b7811890f3f8c2dc089888bdd34d19447fe0f5"} err="failed to get container status \"f601afed40887f5975558a1736b7811890f3f8c2dc089888bdd34d19447fe0f5\": rpc error: code = NotFound desc = could not find container \"f601afed40887f5975558a1736b7811890f3f8c2dc089888bdd34d19447fe0f5\": container with ID starting with f601afed40887f5975558a1736b7811890f3f8c2dc089888bdd34d19447fe0f5 not found: ID does not exist"
Jul 26 19:02:40.879689 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:40.879660    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-9eba87-5b6f46794d-rbwhd"]
Jul 26 19:02:40.882643 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:40.882614    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-9eba87-5b6f46794d-rbwhd"]
Jul 26 19:02:40.895342 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:40.895248    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4ee08d8-4614-40a5-b3e5-283fb9bb21b1-plugins-config-volume\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:02:40.895342 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:40.895281    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bdrg\" (UniqueName: \"kubernetes.io/projected/d4ee08d8-4614-40a5-b3e5-283fb9bb21b1-kube-api-access-4bdrg\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:02:42.849909 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:42.849870    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4ee08d8-4614-40a5-b3e5-283fb9bb21b1" path="/var/lib/kubelet/pods/d4ee08d8-4614-40a5-b3e5-283fb9bb21b1/volumes"
Jul 26 19:02:45.062965 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:45.062935    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:02:45.087431 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:45.087393    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 19:02:45.445188 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:45.445151    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:02:51.009352 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:51.009265    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:02:51.114487 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:51.114455    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:02:51.271470 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:51.271381    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"]
Jul 26 19:02:51.271629 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:51.271591    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x" podUID="b31eb70c-d65a-4166-97ee-c90b3911eb79" containerName="istio-proxy" containerID="cri-o://be3d61bcc1f07cdc131c2dbb17a8d741c8f1605daf54e0969bcea345053dcd28" gracePeriod=30
Jul 26 19:02:51.282502 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:51.282469    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:02:51.516073 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:51.516037    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:02:54.790643 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:54.790601    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4735db-9bjlh"]
Jul 26 19:02:54.791198 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:54.791144    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4ee08d8-4614-40a5-b3e5-283fb9bb21b1" containerName="payload-processing"
Jul 26 19:02:54.791198 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:54.791165    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4ee08d8-4614-40a5-b3e5-283fb9bb21b1" containerName="payload-processing"
Jul 26 19:02:54.791322 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:54.791281    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4ee08d8-4614-40a5-b3e5-283fb9bb21b1" containerName="payload-processing"
Jul 26 19:02:54.795449 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:54.795426    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4735db-9bjlh"
Jul 26 19:02:54.798114 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:54.798094    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxnsx\""
Jul 26 19:02:54.802126 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:54.802099    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4735db-9bjlh"]
Jul 26 19:02:54.924992 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:54.924950    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e536ead-0b97-4f86-9384-6e97a9d14a84-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-4735db-9bjlh\" (UID: \"8e536ead-0b97-4f86-9384-6e97a9d14a84\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4735db-9bjlh"
Jul 26 19:02:55.026579 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:55.026543    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e536ead-0b97-4f86-9384-6e97a9d14a84-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-4735db-9bjlh\" (UID: \"8e536ead-0b97-4f86-9384-6e97a9d14a84\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4735db-9bjlh"
Jul 26 19:02:55.027208 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:55.027188    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e536ead-0b97-4f86-9384-6e97a9d14a84-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-4735db-9bjlh\" (UID: \"8e536ead-0b97-4f86-9384-6e97a9d14a84\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4735db-9bjlh"
Jul 26 19:02:55.107702 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:55.107617    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4735db-9bjlh"
Jul 26 19:02:55.250106 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:55.247027    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4735db-9bjlh"]
Jul 26 19:02:55.253623 ip-10-0-134-7 kubenswrapper[2576]: W0726 19:02:55.253573    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e536ead_0b97_4f86_9384_6e97a9d14a84.slice/crio-fa1f1ce8ef874daccb1afaf3b842eaebf09e91f84adeab3723c668466b7b515b WatchSource:0}: Error finding container fa1f1ce8ef874daccb1afaf3b842eaebf09e91f84adeab3723c668466b7b515b: Status 404 returned error can't find the container with id fa1f1ce8ef874daccb1afaf3b842eaebf09e91f84adeab3723c668466b7b515b
Jul 26 19:02:55.937454 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:55.937410    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4735db-9bjlh" event={"ID":"8e536ead-0b97-4f86-9384-6e97a9d14a84","Type":"ContainerStarted","Data":"ee7edf4937b52e8679a926111f486cbdc0bb4a6f1d3b8c0531219b85ad8bc84b"}
Jul 26 19:02:55.937872 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:55.937480    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4735db-9bjlh" event={"ID":"8e536ead-0b97-4f86-9384-6e97a9d14a84","Type":"ContainerStarted","Data":"fa1f1ce8ef874daccb1afaf3b842eaebf09e91f84adeab3723c668466b7b515b"}
Jul 26 19:02:55.953903 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:55.953853    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4735db-9bjlh" podStartSLOduration=1.953833795 podStartE2EDuration="1.953833795s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 19:02:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 19:02:55.951927556 +0000 UTC m=+3061.659334617" watchObservedRunningTime="2026-07-26 19:02:55.953833795 +0000 UTC m=+3061.661240854"
Jul 26 19:02:56.534321 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.534242    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:56.641900 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.641863    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b31eb70c-d65a-4166-97ee-c90b3911eb79-workload-socket\") pod \"b31eb70c-d65a-4166-97ee-c90b3911eb79\" (UID: \"b31eb70c-d65a-4166-97ee-c90b3911eb79\") "
Jul 26 19:02:56.642115 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.641922    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gk6wz\" (UniqueName: \"kubernetes.io/projected/b31eb70c-d65a-4166-97ee-c90b3911eb79-kube-api-access-gk6wz\") pod \"b31eb70c-d65a-4166-97ee-c90b3911eb79\" (UID: \"b31eb70c-d65a-4166-97ee-c90b3911eb79\") "
Jul 26 19:02:56.642115 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.641948    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b31eb70c-d65a-4166-97ee-c90b3911eb79-istio-envoy\") pod \"b31eb70c-d65a-4166-97ee-c90b3911eb79\" (UID: \"b31eb70c-d65a-4166-97ee-c90b3911eb79\") "
Jul 26 19:02:56.642115 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.641969    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b31eb70c-d65a-4166-97ee-c90b3911eb79-workload-certs\") pod \"b31eb70c-d65a-4166-97ee-c90b3911eb79\" (UID: \"b31eb70c-d65a-4166-97ee-c90b3911eb79\") "
Jul 26 19:02:56.642115 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.642027    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b31eb70c-d65a-4166-97ee-c90b3911eb79-istio-podinfo\") pod \"b31eb70c-d65a-4166-97ee-c90b3911eb79\" (UID: \"b31eb70c-d65a-4166-97ee-c90b3911eb79\") "
Jul 26 19:02:56.642115 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.642053    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b31eb70c-d65a-4166-97ee-c90b3911eb79-istio-data\") pod \"b31eb70c-d65a-4166-97ee-c90b3911eb79\" (UID: \"b31eb70c-d65a-4166-97ee-c90b3911eb79\") "
Jul 26 19:02:56.642115 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.642102    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b31eb70c-d65a-4166-97ee-c90b3911eb79-credential-socket\") pod \"b31eb70c-d65a-4166-97ee-c90b3911eb79\" (UID: \"b31eb70c-d65a-4166-97ee-c90b3911eb79\") "
Jul 26 19:02:56.642417 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.642124    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b31eb70c-d65a-4166-97ee-c90b3911eb79-istio-token\") pod \"b31eb70c-d65a-4166-97ee-c90b3911eb79\" (UID: \"b31eb70c-d65a-4166-97ee-c90b3911eb79\") "
Jul 26 19:02:56.642417 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.642166    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b31eb70c-d65a-4166-97ee-c90b3911eb79-istiod-ca-cert\") pod \"b31eb70c-d65a-4166-97ee-c90b3911eb79\" (UID: \"b31eb70c-d65a-4166-97ee-c90b3911eb79\") "
Jul 26 19:02:56.642417 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.642204    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b31eb70c-d65a-4166-97ee-c90b3911eb79-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b31eb70c-d65a-4166-97ee-c90b3911eb79" (UID: "b31eb70c-d65a-4166-97ee-c90b3911eb79"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 19:02:56.642569 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.642419    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b31eb70c-d65a-4166-97ee-c90b3911eb79-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b31eb70c-d65a-4166-97ee-c90b3911eb79" (UID: "b31eb70c-d65a-4166-97ee-c90b3911eb79"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 19:02:56.642569 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.642424    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b31eb70c-d65a-4166-97ee-c90b3911eb79-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b31eb70c-d65a-4166-97ee-c90b3911eb79" (UID: "b31eb70c-d65a-4166-97ee-c90b3911eb79"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 19:02:56.642569 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.642545    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b31eb70c-d65a-4166-97ee-c90b3911eb79-workload-socket\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:02:56.642569 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.642560    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b31eb70c-d65a-4166-97ee-c90b3911eb79-workload-certs\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:02:56.642805 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.642764    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b31eb70c-d65a-4166-97ee-c90b3911eb79-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b31eb70c-d65a-4166-97ee-c90b3911eb79" (UID: "b31eb70c-d65a-4166-97ee-c90b3911eb79"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 19:02:56.643086 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.643055    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b31eb70c-d65a-4166-97ee-c90b3911eb79-istio-data" (OuterVolumeSpecName: "istio-data") pod "b31eb70c-d65a-4166-97ee-c90b3911eb79" (UID: "b31eb70c-d65a-4166-97ee-c90b3911eb79"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 19:02:56.644329 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.644300    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b31eb70c-d65a-4166-97ee-c90b3911eb79-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b31eb70c-d65a-4166-97ee-c90b3911eb79" (UID: "b31eb70c-d65a-4166-97ee-c90b3911eb79"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 19:02:56.644420 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.644362    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b31eb70c-d65a-4166-97ee-c90b3911eb79-kube-api-access-gk6wz" (OuterVolumeSpecName: "kube-api-access-gk6wz") pod "b31eb70c-d65a-4166-97ee-c90b3911eb79" (UID: "b31eb70c-d65a-4166-97ee-c90b3911eb79"). InnerVolumeSpecName "kube-api-access-gk6wz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 19:02:56.644697 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.644671    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b31eb70c-d65a-4166-97ee-c90b3911eb79-istio-token" (OuterVolumeSpecName: "istio-token") pod "b31eb70c-d65a-4166-97ee-c90b3911eb79" (UID: "b31eb70c-d65a-4166-97ee-c90b3911eb79"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 19:02:56.644821 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.644665    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b31eb70c-d65a-4166-97ee-c90b3911eb79-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b31eb70c-d65a-4166-97ee-c90b3911eb79" (UID: "b31eb70c-d65a-4166-97ee-c90b3911eb79"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 26 19:02:56.743475 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.743437    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gk6wz\" (UniqueName: \"kubernetes.io/projected/b31eb70c-d65a-4166-97ee-c90b3911eb79-kube-api-access-gk6wz\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:02:56.743475 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.743469    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b31eb70c-d65a-4166-97ee-c90b3911eb79-istio-envoy\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:02:56.743475 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.743478    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b31eb70c-d65a-4166-97ee-c90b3911eb79-istio-podinfo\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:02:56.743475 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.743486    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b31eb70c-d65a-4166-97ee-c90b3911eb79-istio-data\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:02:56.743783 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.743495    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b31eb70c-d65a-4166-97ee-c90b3911eb79-credential-socket\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:02:56.743783 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.743504    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b31eb70c-d65a-4166-97ee-c90b3911eb79-istio-token\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:02:56.743783 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.743513    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b31eb70c-d65a-4166-97ee-c90b3911eb79-istiod-ca-cert\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:02:56.944112 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.944072    2576 generic.go:356] "Generic (PLEG): container finished" podID="b31eb70c-d65a-4166-97ee-c90b3911eb79" containerID="be3d61bcc1f07cdc131c2dbb17a8d741c8f1605daf54e0969bcea345053dcd28" exitCode=0
Jul 26 19:02:56.944562 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.944116    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x" event={"ID":"b31eb70c-d65a-4166-97ee-c90b3911eb79","Type":"ContainerDied","Data":"be3d61bcc1f07cdc131c2dbb17a8d741c8f1605daf54e0969bcea345053dcd28"}
Jul 26 19:02:56.944562 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.944178    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x" event={"ID":"b31eb70c-d65a-4166-97ee-c90b3911eb79","Type":"ContainerDied","Data":"11ceb74b7f757b97dd05c95fa1f54e3c10ef240752ba6acb4fec547bc9787f93"}
Jul 26 19:02:56.944562 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.944128    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"
Jul 26 19:02:56.944562 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.944210    2576 scope.go:117] "RemoveContainer" containerID="be3d61bcc1f07cdc131c2dbb17a8d741c8f1605daf54e0969bcea345053dcd28"
Jul 26 19:02:56.945883 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.945852    2576 generic.go:356] "Generic (PLEG): container finished" podID="8e536ead-0b97-4f86-9384-6e97a9d14a84" containerID="ee7edf4937b52e8679a926111f486cbdc0bb4a6f1d3b8c0531219b85ad8bc84b" exitCode=0
Jul 26 19:02:56.945980 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.945925    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4735db-9bjlh" event={"ID":"8e536ead-0b97-4f86-9384-6e97a9d14a84","Type":"ContainerDied","Data":"ee7edf4937b52e8679a926111f486cbdc0bb4a6f1d3b8c0531219b85ad8bc84b"}
Jul 26 19:02:56.953390 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.953372    2576 scope.go:117] "RemoveContainer" containerID="d2cacc3c3da7f439abf1aea7b6915b2c9fd155c95f15fd3674a93e45b52a9884"
Jul 26 19:02:56.961627 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.961612    2576 scope.go:117] "RemoveContainer" containerID="be3d61bcc1f07cdc131c2dbb17a8d741c8f1605daf54e0969bcea345053dcd28"
Jul 26 19:02:56.961973 ip-10-0-134-7 kubenswrapper[2576]: E0726 19:02:56.961941    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be3d61bcc1f07cdc131c2dbb17a8d741c8f1605daf54e0969bcea345053dcd28\": container with ID starting with be3d61bcc1f07cdc131c2dbb17a8d741c8f1605daf54e0969bcea345053dcd28 not found: ID does not exist" containerID="be3d61bcc1f07cdc131c2dbb17a8d741c8f1605daf54e0969bcea345053dcd28"
Jul 26 19:02:56.962045 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.961986    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be3d61bcc1f07cdc131c2dbb17a8d741c8f1605daf54e0969bcea345053dcd28"} err="failed to get container status \"be3d61bcc1f07cdc131c2dbb17a8d741c8f1605daf54e0969bcea345053dcd28\": rpc error: code = NotFound desc = could not find container \"be3d61bcc1f07cdc131c2dbb17a8d741c8f1605daf54e0969bcea345053dcd28\": container with ID starting with be3d61bcc1f07cdc131c2dbb17a8d741c8f1605daf54e0969bcea345053dcd28 not found: ID does not exist"
Jul 26 19:02:56.962045 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.962011    2576 scope.go:117] "RemoveContainer" containerID="d2cacc3c3da7f439abf1aea7b6915b2c9fd155c95f15fd3674a93e45b52a9884"
Jul 26 19:02:56.962282 ip-10-0-134-7 kubenswrapper[2576]: E0726 19:02:56.962245    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2cacc3c3da7f439abf1aea7b6915b2c9fd155c95f15fd3674a93e45b52a9884\": container with ID starting with d2cacc3c3da7f439abf1aea7b6915b2c9fd155c95f15fd3674a93e45b52a9884 not found: ID does not exist" containerID="d2cacc3c3da7f439abf1aea7b6915b2c9fd155c95f15fd3674a93e45b52a9884"
Jul 26 19:02:56.962371 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.962292    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2cacc3c3da7f439abf1aea7b6915b2c9fd155c95f15fd3674a93e45b52a9884"} err="failed to get container status \"d2cacc3c3da7f439abf1aea7b6915b2c9fd155c95f15fd3674a93e45b52a9884\": rpc error: code = NotFound desc = could not find container \"d2cacc3c3da7f439abf1aea7b6915b2c9fd155c95f15fd3674a93e45b52a9884\": container with ID starting with d2cacc3c3da7f439abf1aea7b6915b2c9fd155c95f15fd3674a93e45b52a9884 not found: ID does not exist"
Jul 26 19:02:56.975594 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.975562    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"]
Jul 26 19:02:56.979526 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:56.979494    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-9eba87-openshift-default-558cd4bd88-frc6x"]
Jul 26 19:02:58.095658 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:58.095632    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4735db-9bjlh"
Jul 26 19:02:58.155312 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:58.155274    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e536ead-0b97-4f86-9384-6e97a9d14a84-maas-api-service-ca\") pod \"8e536ead-0b97-4f86-9384-6e97a9d14a84\" (UID: \"8e536ead-0b97-4f86-9384-6e97a9d14a84\") "
Jul 26 19:02:58.155640 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:58.155614    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e536ead-0b97-4f86-9384-6e97a9d14a84-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8e536ead-0b97-4f86-9384-6e97a9d14a84" (UID: "8e536ead-0b97-4f86-9384-6e97a9d14a84"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 19:02:58.256623 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:58.256531    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e536ead-0b97-4f86-9384-6e97a9d14a84-maas-api-service-ca\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:02:58.849240 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:58.849205    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b31eb70c-d65a-4166-97ee-c90b3911eb79" path="/var/lib/kubelet/pods/b31eb70c-d65a-4166-97ee-c90b3911eb79/volumes"
Jul 26 19:02:58.958115 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:58.958090    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4735db-9bjlh"
Jul 26 19:02:58.958283 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:58.958085    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4735db-9bjlh" event={"ID":"8e536ead-0b97-4f86-9384-6e97a9d14a84","Type":"ContainerDied","Data":"fa1f1ce8ef874daccb1afaf3b842eaebf09e91f84adeab3723c668466b7b515b"}
Jul 26 19:02:58.958283 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:02:58.958212    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fa1f1ce8ef874daccb1afaf3b842eaebf09e91f84adeab3723c668466b7b515b"
Jul 26 19:03:09.209091 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:03:09.209052    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 19:03:10.393960 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:03:10.393921    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh"]
Jul 26 19:03:10.394422 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:03:10.394188    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh" podUID="08ff5eed-7b25-49c1-b9c2-898d9a3a284a" containerName="maas-api" containerID="cri-o://3255f400c503d34bbf423fb77bc2cf75af6e7a91934be5dfe5aad848a4e6cb22" gracePeriod=30
Jul 26 19:03:10.428919 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:03:10.428884    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:03:10.524659 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:03:10.524628    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:03:10.622298 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:03:10.622255    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:03:10.660922 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:03:10.660850    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh"
Jul 26 19:03:10.770349 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:03:10.770315    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bgmbl\" (UniqueName: \"kubernetes.io/projected/08ff5eed-7b25-49c1-b9c2-898d9a3a284a-kube-api-access-bgmbl\") pod \"08ff5eed-7b25-49c1-b9c2-898d9a3a284a\" (UID: \"08ff5eed-7b25-49c1-b9c2-898d9a3a284a\") "
Jul 26 19:03:10.770534 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:03:10.770373    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08ff5eed-7b25-49c1-b9c2-898d9a3a284a-openshift-service-ca\") pod \"08ff5eed-7b25-49c1-b9c2-898d9a3a284a\" (UID: \"08ff5eed-7b25-49c1-b9c2-898d9a3a284a\") "
Jul 26 19:03:10.770534 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:03:10.770480    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08ff5eed-7b25-49c1-b9c2-898d9a3a284a-maas-api-tls\") pod \"08ff5eed-7b25-49c1-b9c2-898d9a3a284a\" (UID: \"08ff5eed-7b25-49c1-b9c2-898d9a3a284a\") "
Jul 26 19:03:10.770818 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:03:10.770793    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08ff5eed-7b25-49c1-b9c2-898d9a3a284a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "08ff5eed-7b25-49c1-b9c2-898d9a3a284a" (UID: "08ff5eed-7b25-49c1-b9c2-898d9a3a284a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 19:03:10.772528 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:03:10.772500    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08ff5eed-7b25-49c1-b9c2-898d9a3a284a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "08ff5eed-7b25-49c1-b9c2-898d9a3a284a" (UID: "08ff5eed-7b25-49c1-b9c2-898d9a3a284a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 19:03:10.772623 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:03:10.772553    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08ff5eed-7b25-49c1-b9c2-898d9a3a284a-kube-api-access-bgmbl" (OuterVolumeSpecName: "kube-api-access-bgmbl") pod "08ff5eed-7b25-49c1-b9c2-898d9a3a284a" (UID: "08ff5eed-7b25-49c1-b9c2-898d9a3a284a"). InnerVolumeSpecName "kube-api-access-bgmbl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 19:03:10.871189 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:03:10.871151    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08ff5eed-7b25-49c1-b9c2-898d9a3a284a-maas-api-tls\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:03:10.871189 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:03:10.871184    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgmbl\" (UniqueName: \"kubernetes.io/projected/08ff5eed-7b25-49c1-b9c2-898d9a3a284a-kube-api-access-bgmbl\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:03:10.871189 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:03:10.871198    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08ff5eed-7b25-49c1-b9c2-898d9a3a284a-openshift-service-ca\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:03:11.017179 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:03:11.017143    2576 generic.go:356] "Generic (PLEG): container finished" podID="08ff5eed-7b25-49c1-b9c2-898d9a3a284a" containerID="3255f400c503d34bbf423fb77bc2cf75af6e7a91934be5dfe5aad848a4e6cb22" exitCode=0
Jul 26 19:03:11.017359 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:03:11.017193    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh" event={"ID":"08ff5eed-7b25-49c1-b9c2-898d9a3a284a","Type":"ContainerDied","Data":"3255f400c503d34bbf423fb77bc2cf75af6e7a91934be5dfe5aad848a4e6cb22"}
Jul 26 19:03:11.017359 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:03:11.017204    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh"
Jul 26 19:03:11.017359 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:03:11.017244    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh" event={"ID":"08ff5eed-7b25-49c1-b9c2-898d9a3a284a","Type":"ContainerDied","Data":"ee30a737f0b2f29ce92288ff549518ff2508b491fc6546f207aeb399fe9c3c7a"}
Jul 26 19:03:11.017359 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:03:11.017271    2576 scope.go:117] "RemoveContainer" containerID="3255f400c503d34bbf423fb77bc2cf75af6e7a91934be5dfe5aad848a4e6cb22"
Jul 26 19:03:11.025632 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:03:11.025613    2576 scope.go:117] "RemoveContainer" containerID="3255f400c503d34bbf423fb77bc2cf75af6e7a91934be5dfe5aad848a4e6cb22"
Jul 26 19:03:11.025921 ip-10-0-134-7 kubenswrapper[2576]: E0726 19:03:11.025899    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3255f400c503d34bbf423fb77bc2cf75af6e7a91934be5dfe5aad848a4e6cb22\": container with ID starting with 3255f400c503d34bbf423fb77bc2cf75af6e7a91934be5dfe5aad848a4e6cb22 not found: ID does not exist" containerID="3255f400c503d34bbf423fb77bc2cf75af6e7a91934be5dfe5aad848a4e6cb22"
Jul 26 19:03:11.025969 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:03:11.025930    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3255f400c503d34bbf423fb77bc2cf75af6e7a91934be5dfe5aad848a4e6cb22"} err="failed to get container status \"3255f400c503d34bbf423fb77bc2cf75af6e7a91934be5dfe5aad848a4e6cb22\": rpc error: code = NotFound desc = could not find container \"3255f400c503d34bbf423fb77bc2cf75af6e7a91934be5dfe5aad848a4e6cb22\": container with ID starting with 3255f400c503d34bbf423fb77bc2cf75af6e7a91934be5dfe5aad848a4e6cb22 not found: ID does not exist"
Jul 26 19:03:11.041011 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:03:11.040984    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh"]
Jul 26 19:03:11.045517 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:03:11.045491    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-4735db-76fbd697bb-k9njh"]
Jul 26 19:03:12.849229 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:03:12.849194    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08ff5eed-7b25-49c1-b9c2-898d9a3a284a" path="/var/lib/kubelet/pods/08ff5eed-7b25-49c1-b9c2-898d9a3a284a/volumes"
Jul 26 19:03:14.865191 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:03:14.865149    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:03:14.865723 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:03:14.865681    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4735db-9bjlh"]
Jul 26 19:03:14.867465 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:03:14.867444    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4735db-9bjlh"]
Jul 26 19:03:15.248044 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:03:15.248011    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:03:16.851548 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:03:16.851513    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e536ead-0b97-4f86-9384-6e97a9d14a84" path="/var/lib/kubelet/pods/8e536ead-0b97-4f86-9384-6e97a9d14a84/volumes"
Jul 26 19:03:20.288421 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:03:20.288368    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:03:20.598812 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:03:20.598713    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 19:03:32.525764 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:03:32.525717    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:03:32.586971 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:03:32.586936    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:03:39.945492 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:03:39.945459    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:03:56.571908 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:03:56.571872    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:03:57.295001 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:03:57.294956    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:04:03.016737 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:03.016702    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:04:03.092246 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:03.092207    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:04:03.305578 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:03.305493    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:04:03.535956 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:03.535925    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:04:14.091995 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:14.091584    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9"]
Jul 26 19:04:14.091995 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:14.091855    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9" podUID="db2b53c2-9568-4b09-bc4a-d484802e4901" containerName="maas-api" containerID="cri-o://485032d52e4a09d16c91b5f7451d3337e73b00a9b60e5249caa7735bb7bab7b8" gracePeriod=30
Jul 26 19:04:14.130940 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:14.130896    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:04:14.229846 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:14.229810    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:04:14.319676 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:14.319637    2576 generic.go:356] "Generic (PLEG): container finished" podID="db2b53c2-9568-4b09-bc4a-d484802e4901" containerID="485032d52e4a09d16c91b5f7451d3337e73b00a9b60e5249caa7735bb7bab7b8" exitCode=0
Jul 26 19:04:14.319883 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:14.319727    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9" event={"ID":"db2b53c2-9568-4b09-bc4a-d484802e4901","Type":"ContainerDied","Data":"485032d52e4a09d16c91b5f7451d3337e73b00a9b60e5249caa7735bb7bab7b8"}
Jul 26 19:04:14.330651 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:14.330620    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:04:14.348417 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:14.348352    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9"
Jul 26 19:04:14.430733 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:14.430699    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db2b53c2-9568-4b09-bc4a-d484802e4901-maas-api-tls\") pod \"db2b53c2-9568-4b09-bc4a-d484802e4901\" (UID: \"db2b53c2-9568-4b09-bc4a-d484802e4901\") "
Jul 26 19:04:14.430733 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:14.430733    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db2b53c2-9568-4b09-bc4a-d484802e4901-openshift-service-ca\") pod \"db2b53c2-9568-4b09-bc4a-d484802e4901\" (UID: \"db2b53c2-9568-4b09-bc4a-d484802e4901\") "
Jul 26 19:04:14.430971 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:14.430873    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7kndv\" (UniqueName: \"kubernetes.io/projected/db2b53c2-9568-4b09-bc4a-d484802e4901-kube-api-access-7kndv\") pod \"db2b53c2-9568-4b09-bc4a-d484802e4901\" (UID: \"db2b53c2-9568-4b09-bc4a-d484802e4901\") "
Jul 26 19:04:14.431122 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:14.431099    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db2b53c2-9568-4b09-bc4a-d484802e4901-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "db2b53c2-9568-4b09-bc4a-d484802e4901" (UID: "db2b53c2-9568-4b09-bc4a-d484802e4901"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 19:04:14.432982 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:14.432957    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db2b53c2-9568-4b09-bc4a-d484802e4901-kube-api-access-7kndv" (OuterVolumeSpecName: "kube-api-access-7kndv") pod "db2b53c2-9568-4b09-bc4a-d484802e4901" (UID: "db2b53c2-9568-4b09-bc4a-d484802e4901"). InnerVolumeSpecName "kube-api-access-7kndv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 19:04:14.433083 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:14.432980    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db2b53c2-9568-4b09-bc4a-d484802e4901-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "db2b53c2-9568-4b09-bc4a-d484802e4901" (UID: "db2b53c2-9568-4b09-bc4a-d484802e4901"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 19:04:14.537167 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:14.532174    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7kndv\" (UniqueName: \"kubernetes.io/projected/db2b53c2-9568-4b09-bc4a-d484802e4901-kube-api-access-7kndv\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:04:14.537167 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:14.532230    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db2b53c2-9568-4b09-bc4a-d484802e4901-maas-api-tls\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:04:14.537167 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:14.532245    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db2b53c2-9568-4b09-bc4a-d484802e4901-openshift-service-ca\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:04:15.326059 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:15.326026    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9"
Jul 26 19:04:15.326525 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:15.326054    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9" event={"ID":"db2b53c2-9568-4b09-bc4a-d484802e4901","Type":"ContainerDied","Data":"e1ff12ff6f21bf769475f54d561b44dccf2643b15b7b088be3dae18e8aac29a6"}
Jul 26 19:04:15.326525 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:15.326130    2576 scope.go:117] "RemoveContainer" containerID="485032d52e4a09d16c91b5f7451d3337e73b00a9b60e5249caa7735bb7bab7b8"
Jul 26 19:04:15.345525 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:15.345493    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9"]
Jul 26 19:04:15.349459 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:15.349433    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-1318a7-f49598895-n2vj9"]
Jul 26 19:04:16.848767 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:16.848726    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db2b53c2-9568-4b09-bc4a-d484802e4901" path="/var/lib/kubelet/pods/db2b53c2-9568-4b09-bc4a-d484802e4901/volumes"
Jul 26 19:04:17.093631 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:17.093556    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 19:04:18.741801 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:18.741733    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:04:19.106401 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:19.106322    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:04:25.011733 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:25.011697    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:04:25.115365 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:25.115322    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:04:25.281713 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:25.281622    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:04:25.508687 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:25.508657    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:04:36.088248 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:36.088147    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl"]
Jul 26 19:04:36.114808 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:36.114770    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:04:36.119458 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:36.119432    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2b193e-29751540-h45vf"]
Jul 26 19:04:36.122008 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:36.121984    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2b193e-29751540-h45vf"]
Jul 26 19:04:36.201399 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:36.201367    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:04:36.272178 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:36.272140    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:04:36.423927 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:36.423884    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl" podUID="75fb92d2-3709-4227-a0a4-b2c84ddcea8d" containerName="maas-api" containerID="cri-o://9a0821c2e916758026d6a58ca8dcbabcef3295177cfbfb492ff4a148960e39ce" gracePeriod=30
Jul 26 19:04:36.678189 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:36.678126    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl"
Jul 26 19:04:36.729146 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:36.729112    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75fb92d2-3709-4227-a0a4-b2c84ddcea8d-maas-api-tls\") pod \"75fb92d2-3709-4227-a0a4-b2c84ddcea8d\" (UID: \"75fb92d2-3709-4227-a0a4-b2c84ddcea8d\") "
Jul 26 19:04:36.729313 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:36.729153    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75fb92d2-3709-4227-a0a4-b2c84ddcea8d-openshift-service-ca\") pod \"75fb92d2-3709-4227-a0a4-b2c84ddcea8d\" (UID: \"75fb92d2-3709-4227-a0a4-b2c84ddcea8d\") "
Jul 26 19:04:36.729313 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:36.729250    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbk5s\" (UniqueName: \"kubernetes.io/projected/75fb92d2-3709-4227-a0a4-b2c84ddcea8d-kube-api-access-cbk5s\") pod \"75fb92d2-3709-4227-a0a4-b2c84ddcea8d\" (UID: \"75fb92d2-3709-4227-a0a4-b2c84ddcea8d\") "
Jul 26 19:04:36.729540 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:36.729519    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75fb92d2-3709-4227-a0a4-b2c84ddcea8d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "75fb92d2-3709-4227-a0a4-b2c84ddcea8d" (UID: "75fb92d2-3709-4227-a0a4-b2c84ddcea8d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 19:04:36.731320 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:36.731291    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75fb92d2-3709-4227-a0a4-b2c84ddcea8d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "75fb92d2-3709-4227-a0a4-b2c84ddcea8d" (UID: "75fb92d2-3709-4227-a0a4-b2c84ddcea8d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 19:04:36.731428 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:36.731330    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75fb92d2-3709-4227-a0a4-b2c84ddcea8d-kube-api-access-cbk5s" (OuterVolumeSpecName: "kube-api-access-cbk5s") pod "75fb92d2-3709-4227-a0a4-b2c84ddcea8d" (UID: "75fb92d2-3709-4227-a0a4-b2c84ddcea8d"). InnerVolumeSpecName "kube-api-access-cbk5s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 19:04:36.835917 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:36.831218    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbk5s\" (UniqueName: \"kubernetes.io/projected/75fb92d2-3709-4227-a0a4-b2c84ddcea8d-kube-api-access-cbk5s\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:04:36.835917 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:36.831272    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75fb92d2-3709-4227-a0a4-b2c84ddcea8d-maas-api-tls\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:04:36.835917 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:36.831288    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75fb92d2-3709-4227-a0a4-b2c84ddcea8d-openshift-service-ca\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:04:36.849207 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:36.849162    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb6d04c3-21c5-40de-839b-24e220dae207" path="/var/lib/kubelet/pods/cb6d04c3-21c5-40de-839b-24e220dae207/volumes"
Jul 26 19:04:37.429288 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:37.429252    2576 generic.go:356] "Generic (PLEG): container finished" podID="75fb92d2-3709-4227-a0a4-b2c84ddcea8d" containerID="9a0821c2e916758026d6a58ca8dcbabcef3295177cfbfb492ff4a148960e39ce" exitCode=0
Jul 26 19:04:37.429718 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:37.429306    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl"
Jul 26 19:04:37.429718 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:37.429331    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl" event={"ID":"75fb92d2-3709-4227-a0a4-b2c84ddcea8d","Type":"ContainerDied","Data":"9a0821c2e916758026d6a58ca8dcbabcef3295177cfbfb492ff4a148960e39ce"}
Jul 26 19:04:37.429718 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:37.429384    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl" event={"ID":"75fb92d2-3709-4227-a0a4-b2c84ddcea8d","Type":"ContainerDied","Data":"e3bec8a760ac5888e47a828e3042f5a438e89c44d446cfc0de904809b58ac50b"}
Jul 26 19:04:37.429718 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:37.429405    2576 scope.go:117] "RemoveContainer" containerID="9a0821c2e916758026d6a58ca8dcbabcef3295177cfbfb492ff4a148960e39ce"
Jul 26 19:04:37.438460 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:37.438436    2576 scope.go:117] "RemoveContainer" containerID="9a0821c2e916758026d6a58ca8dcbabcef3295177cfbfb492ff4a148960e39ce"
Jul 26 19:04:37.438731 ip-10-0-134-7 kubenswrapper[2576]: E0726 19:04:37.438711    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a0821c2e916758026d6a58ca8dcbabcef3295177cfbfb492ff4a148960e39ce\": container with ID starting with 9a0821c2e916758026d6a58ca8dcbabcef3295177cfbfb492ff4a148960e39ce not found: ID does not exist" containerID="9a0821c2e916758026d6a58ca8dcbabcef3295177cfbfb492ff4a148960e39ce"
Jul 26 19:04:37.438802 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:37.438762    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a0821c2e916758026d6a58ca8dcbabcef3295177cfbfb492ff4a148960e39ce"} err="failed to get container status \"9a0821c2e916758026d6a58ca8dcbabcef3295177cfbfb492ff4a148960e39ce\": rpc error: code = NotFound desc = could not find container \"9a0821c2e916758026d6a58ca8dcbabcef3295177cfbfb492ff4a148960e39ce\": container with ID starting with 9a0821c2e916758026d6a58ca8dcbabcef3295177cfbfb492ff4a148960e39ce not found: ID does not exist"
Jul 26 19:04:37.448090 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:37.448063    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl"]
Jul 26 19:04:37.452324 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:37.452302    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-2b193e-79f8dff4dc-f4lhl"]
Jul 26 19:04:38.848416 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:38.848380    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75fb92d2-3709-4227-a0a4-b2c84ddcea8d" path="/var/lib/kubelet/pods/75fb92d2-3709-4227-a0a4-b2c84ddcea8d/volumes"
Jul 26 19:04:39.194332 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:39.194293    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 19:04:40.723788 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:40.723722    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:04:41.104651 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:41.104564    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:04:47.449971 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:47.449931    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:04:47.549713 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:47.549681    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:04:47.680841 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:47.680792    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"]
Jul 26 19:04:47.681138 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:47.681105    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm" podUID="c7dc0999-26b8-4201-8230-713d58ef3b16" containerName="istio-proxy" containerID="cri-o://ed47b320c0f059223b8f5b6d8012fcdc0ae33dee4a5fb0eface5674d141d610d" gracePeriod=30
Jul 26 19:04:47.703046 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:47.702950    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:04:47.926626 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:47.926584    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:04:52.940141 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:52.940117    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 19:04:53.079853 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.079769    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7dc0999-26b8-4201-8230-713d58ef3b16-workload-socket\") pod \"c7dc0999-26b8-4201-8230-713d58ef3b16\" (UID: \"c7dc0999-26b8-4201-8230-713d58ef3b16\") "
Jul 26 19:04:53.080000 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.079859    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7dc0999-26b8-4201-8230-713d58ef3b16-istio-data\") pod \"c7dc0999-26b8-4201-8230-713d58ef3b16\" (UID: \"c7dc0999-26b8-4201-8230-713d58ef3b16\") "
Jul 26 19:04:53.080000 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.079909    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7dc0999-26b8-4201-8230-713d58ef3b16-istio-envoy\") pod \"c7dc0999-26b8-4201-8230-713d58ef3b16\" (UID: \"c7dc0999-26b8-4201-8230-713d58ef3b16\") "
Jul 26 19:04:53.080000 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.079932    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7dc0999-26b8-4201-8230-713d58ef3b16-credential-socket\") pod \"c7dc0999-26b8-4201-8230-713d58ef3b16\" (UID: \"c7dc0999-26b8-4201-8230-713d58ef3b16\") "
Jul 26 19:04:53.080000 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.079950    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7dc0999-26b8-4201-8230-713d58ef3b16-istio-podinfo\") pod \"c7dc0999-26b8-4201-8230-713d58ef3b16\" (UID: \"c7dc0999-26b8-4201-8230-713d58ef3b16\") "
Jul 26 19:04:53.080000 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.079966    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7dc0999-26b8-4201-8230-713d58ef3b16-workload-certs\") pod \"c7dc0999-26b8-4201-8230-713d58ef3b16\" (UID: \"c7dc0999-26b8-4201-8230-713d58ef3b16\") "
Jul 26 19:04:53.080000 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.079982    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7dc0999-26b8-4201-8230-713d58ef3b16-istiod-ca-cert\") pod \"c7dc0999-26b8-4201-8230-713d58ef3b16\" (UID: \"c7dc0999-26b8-4201-8230-713d58ef3b16\") "
Jul 26 19:04:53.080293 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.080006    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7dc0999-26b8-4201-8230-713d58ef3b16-istio-token\") pod \"c7dc0999-26b8-4201-8230-713d58ef3b16\" (UID: \"c7dc0999-26b8-4201-8230-713d58ef3b16\") "
Jul 26 19:04:53.080293 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.080032    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tgl9d\" (UniqueName: \"kubernetes.io/projected/c7dc0999-26b8-4201-8230-713d58ef3b16-kube-api-access-tgl9d\") pod \"c7dc0999-26b8-4201-8230-713d58ef3b16\" (UID: \"c7dc0999-26b8-4201-8230-713d58ef3b16\") "
Jul 26 19:04:53.080293 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.080033    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7dc0999-26b8-4201-8230-713d58ef3b16-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c7dc0999-26b8-4201-8230-713d58ef3b16" (UID: "c7dc0999-26b8-4201-8230-713d58ef3b16"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 19:04:53.080293 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.080257    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7dc0999-26b8-4201-8230-713d58ef3b16-workload-socket\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:04:53.080293 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.080265    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7dc0999-26b8-4201-8230-713d58ef3b16-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c7dc0999-26b8-4201-8230-713d58ef3b16" (UID: "c7dc0999-26b8-4201-8230-713d58ef3b16"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 19:04:53.080535 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.080344    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7dc0999-26b8-4201-8230-713d58ef3b16-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c7dc0999-26b8-4201-8230-713d58ef3b16" (UID: "c7dc0999-26b8-4201-8230-713d58ef3b16"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 19:04:53.080589 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.080533    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7dc0999-26b8-4201-8230-713d58ef3b16-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c7dc0999-26b8-4201-8230-713d58ef3b16" (UID: "c7dc0999-26b8-4201-8230-713d58ef3b16"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 19:04:53.080795 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.080759    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7dc0999-26b8-4201-8230-713d58ef3b16-istio-data" (OuterVolumeSpecName: "istio-data") pod "c7dc0999-26b8-4201-8230-713d58ef3b16" (UID: "c7dc0999-26b8-4201-8230-713d58ef3b16"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 19:04:53.082222 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.082193    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7dc0999-26b8-4201-8230-713d58ef3b16-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c7dc0999-26b8-4201-8230-713d58ef3b16" (UID: "c7dc0999-26b8-4201-8230-713d58ef3b16"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 19:04:53.082352 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.082219    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7dc0999-26b8-4201-8230-713d58ef3b16-istio-token" (OuterVolumeSpecName: "istio-token") pod "c7dc0999-26b8-4201-8230-713d58ef3b16" (UID: "c7dc0999-26b8-4201-8230-713d58ef3b16"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 19:04:53.082535 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.082513    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7dc0999-26b8-4201-8230-713d58ef3b16-kube-api-access-tgl9d" (OuterVolumeSpecName: "kube-api-access-tgl9d") pod "c7dc0999-26b8-4201-8230-713d58ef3b16" (UID: "c7dc0999-26b8-4201-8230-713d58ef3b16"). InnerVolumeSpecName "kube-api-access-tgl9d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 19:04:53.082587 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.082572    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c7dc0999-26b8-4201-8230-713d58ef3b16-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c7dc0999-26b8-4201-8230-713d58ef3b16" (UID: "c7dc0999-26b8-4201-8230-713d58ef3b16"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 26 19:04:53.181541 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.181502    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7dc0999-26b8-4201-8230-713d58ef3b16-istio-envoy\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:04:53.181541 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.181534    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7dc0999-26b8-4201-8230-713d58ef3b16-credential-socket\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:04:53.181541 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.181543    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7dc0999-26b8-4201-8230-713d58ef3b16-istio-podinfo\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:04:53.181541 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.181552    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7dc0999-26b8-4201-8230-713d58ef3b16-workload-certs\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:04:53.181843 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.181561    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7dc0999-26b8-4201-8230-713d58ef3b16-istiod-ca-cert\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:04:53.181843 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.181571    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7dc0999-26b8-4201-8230-713d58ef3b16-istio-token\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:04:53.181843 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.181584    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tgl9d\" (UniqueName: \"kubernetes.io/projected/c7dc0999-26b8-4201-8230-713d58ef3b16-kube-api-access-tgl9d\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:04:53.181843 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.181594    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7dc0999-26b8-4201-8230-713d58ef3b16-istio-data\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:04:53.505952 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.505917    2576 generic.go:356] "Generic (PLEG): container finished" podID="c7dc0999-26b8-4201-8230-713d58ef3b16" containerID="ed47b320c0f059223b8f5b6d8012fcdc0ae33dee4a5fb0eface5674d141d610d" exitCode=0
Jul 26 19:04:53.506154 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.505978    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"
Jul 26 19:04:53.506154 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.506005    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm" event={"ID":"c7dc0999-26b8-4201-8230-713d58ef3b16","Type":"ContainerDied","Data":"ed47b320c0f059223b8f5b6d8012fcdc0ae33dee4a5fb0eface5674d141d610d"}
Jul 26 19:04:53.506154 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.506074    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm" event={"ID":"c7dc0999-26b8-4201-8230-713d58ef3b16","Type":"ContainerDied","Data":"31a99014f7ccc067d8809094f59ca0703865ea877877347a1a66f704fb3b4b94"}
Jul 26 19:04:53.506154 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.506105    2576 scope.go:117] "RemoveContainer" containerID="ed47b320c0f059223b8f5b6d8012fcdc0ae33dee4a5fb0eface5674d141d610d"
Jul 26 19:04:53.514989 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.514972    2576 scope.go:117] "RemoveContainer" containerID="cf10d86cef8bfaf4511689ba8ec58e3d598ab2650356b4092699f4b1eaeb4e11"
Jul 26 19:04:53.523068 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.523049    2576 scope.go:117] "RemoveContainer" containerID="ed47b320c0f059223b8f5b6d8012fcdc0ae33dee4a5fb0eface5674d141d610d"
Jul 26 19:04:53.523345 ip-10-0-134-7 kubenswrapper[2576]: E0726 19:04:53.523326    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed47b320c0f059223b8f5b6d8012fcdc0ae33dee4a5fb0eface5674d141d610d\": container with ID starting with ed47b320c0f059223b8f5b6d8012fcdc0ae33dee4a5fb0eface5674d141d610d not found: ID does not exist" containerID="ed47b320c0f059223b8f5b6d8012fcdc0ae33dee4a5fb0eface5674d141d610d"
Jul 26 19:04:53.523396 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.523356    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed47b320c0f059223b8f5b6d8012fcdc0ae33dee4a5fb0eface5674d141d610d"} err="failed to get container status \"ed47b320c0f059223b8f5b6d8012fcdc0ae33dee4a5fb0eface5674d141d610d\": rpc error: code = NotFound desc = could not find container \"ed47b320c0f059223b8f5b6d8012fcdc0ae33dee4a5fb0eface5674d141d610d\": container with ID starting with ed47b320c0f059223b8f5b6d8012fcdc0ae33dee4a5fb0eface5674d141d610d not found: ID does not exist"
Jul 26 19:04:53.523396 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.523376    2576 scope.go:117] "RemoveContainer" containerID="cf10d86cef8bfaf4511689ba8ec58e3d598ab2650356b4092699f4b1eaeb4e11"
Jul 26 19:04:53.523628 ip-10-0-134-7 kubenswrapper[2576]: E0726 19:04:53.523608    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf10d86cef8bfaf4511689ba8ec58e3d598ab2650356b4092699f4b1eaeb4e11\": container with ID starting with cf10d86cef8bfaf4511689ba8ec58e3d598ab2650356b4092699f4b1eaeb4e11 not found: ID does not exist" containerID="cf10d86cef8bfaf4511689ba8ec58e3d598ab2650356b4092699f4b1eaeb4e11"
Jul 26 19:04:53.523670 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.523634    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf10d86cef8bfaf4511689ba8ec58e3d598ab2650356b4092699f4b1eaeb4e11"} err="failed to get container status \"cf10d86cef8bfaf4511689ba8ec58e3d598ab2650356b4092699f4b1eaeb4e11\": rpc error: code = NotFound desc = could not find container \"cf10d86cef8bfaf4511689ba8ec58e3d598ab2650356b4092699f4b1eaeb4e11\": container with ID starting with cf10d86cef8bfaf4511689ba8ec58e3d598ab2650356b4092699f4b1eaeb4e11 not found: ID does not exist"
Jul 26 19:04:53.529160 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.529137    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"]
Jul 26 19:04:53.534957 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:53.534929    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-2b193e-openshift-default-7c59fd96bd-76cdm"]
Jul 26 19:04:54.849563 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:54.849523    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7dc0999-26b8-4201-8230-713d58ef3b16" path="/var/lib/kubelet/pods/c7dc0999-26b8-4201-8230-713d58ef3b16/volumes"
Jul 26 19:04:58.503129 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:58.503084    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q"]
Jul 26 19:04:58.503623 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:58.503320    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q" podUID="4eb8c250-24f0-46eb-b606-1f68c4f165ef" containerName="maas-api" containerID="cri-o://8dc7ea3d0f032e039d7b85d53ae3529343754482a6c50b8dca01b0cb1f25cda2" gracePeriod=30
Jul 26 19:04:58.537241 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:58.537199    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c75d5a-29751540-txbrp"]
Jul 26 19:04:58.539003 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:58.538972    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:04:58.539978 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:58.539957    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c75d5a-29751540-txbrp"]
Jul 26 19:04:58.616367 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:58.616328    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:04:58.682576 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:58.682539    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:04:58.759099 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:58.759027    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q"
Jul 26 19:04:58.850539 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:58.850508    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cf20171-4fd9-4814-85c6-8b9375e0d346" path="/var/lib/kubelet/pods/0cf20171-4fd9-4814-85c6-8b9375e0d346/volumes"
Jul 26 19:04:58.936831 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:58.936790    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4eb8c250-24f0-46eb-b606-1f68c4f165ef-openshift-service-ca\") pod \"4eb8c250-24f0-46eb-b606-1f68c4f165ef\" (UID: \"4eb8c250-24f0-46eb-b606-1f68c4f165ef\") "
Jul 26 19:04:58.937003 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:58.936887    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4eb8c250-24f0-46eb-b606-1f68c4f165ef-maas-api-tls\") pod \"4eb8c250-24f0-46eb-b606-1f68c4f165ef\" (UID: \"4eb8c250-24f0-46eb-b606-1f68c4f165ef\") "
Jul 26 19:04:58.937003 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:58.936920    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qcr5v\" (UniqueName: \"kubernetes.io/projected/4eb8c250-24f0-46eb-b606-1f68c4f165ef-kube-api-access-qcr5v\") pod \"4eb8c250-24f0-46eb-b606-1f68c4f165ef\" (UID: \"4eb8c250-24f0-46eb-b606-1f68c4f165ef\") "
Jul 26 19:04:58.937213 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:58.937182    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4eb8c250-24f0-46eb-b606-1f68c4f165ef-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4eb8c250-24f0-46eb-b606-1f68c4f165ef" (UID: "4eb8c250-24f0-46eb-b606-1f68c4f165ef"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 19:04:58.938941 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:58.938910    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4eb8c250-24f0-46eb-b606-1f68c4f165ef-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4eb8c250-24f0-46eb-b606-1f68c4f165ef" (UID: "4eb8c250-24f0-46eb-b606-1f68c4f165ef"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 19:04:58.939060 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:58.938951    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4eb8c250-24f0-46eb-b606-1f68c4f165ef-kube-api-access-qcr5v" (OuterVolumeSpecName: "kube-api-access-qcr5v") pod "4eb8c250-24f0-46eb-b606-1f68c4f165ef" (UID: "4eb8c250-24f0-46eb-b606-1f68c4f165ef"). InnerVolumeSpecName "kube-api-access-qcr5v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 19:04:59.037958 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:59.037866    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qcr5v\" (UniqueName: \"kubernetes.io/projected/4eb8c250-24f0-46eb-b606-1f68c4f165ef-kube-api-access-qcr5v\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:04:59.037958 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:59.037898    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4eb8c250-24f0-46eb-b606-1f68c4f165ef-openshift-service-ca\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:04:59.037958 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:59.037908    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4eb8c250-24f0-46eb-b606-1f68c4f165ef-maas-api-tls\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:04:59.536524 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:59.536486    2576 generic.go:356] "Generic (PLEG): container finished" podID="4eb8c250-24f0-46eb-b606-1f68c4f165ef" containerID="8dc7ea3d0f032e039d7b85d53ae3529343754482a6c50b8dca01b0cb1f25cda2" exitCode=0
Jul 26 19:04:59.536995 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:59.536532    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q"
Jul 26 19:04:59.536995 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:59.536570    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q" event={"ID":"4eb8c250-24f0-46eb-b606-1f68c4f165ef","Type":"ContainerDied","Data":"8dc7ea3d0f032e039d7b85d53ae3529343754482a6c50b8dca01b0cb1f25cda2"}
Jul 26 19:04:59.536995 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:59.536628    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q" event={"ID":"4eb8c250-24f0-46eb-b606-1f68c4f165ef","Type":"ContainerDied","Data":"bb4f2a63fc9d89551791989b9904c724724df0a3678c358fc4dd2f1312c127c9"}
Jul 26 19:04:59.536995 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:59.536652    2576 scope.go:117] "RemoveContainer" containerID="8dc7ea3d0f032e039d7b85d53ae3529343754482a6c50b8dca01b0cb1f25cda2"
Jul 26 19:04:59.545626 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:59.545608    2576 scope.go:117] "RemoveContainer" containerID="8dc7ea3d0f032e039d7b85d53ae3529343754482a6c50b8dca01b0cb1f25cda2"
Jul 26 19:04:59.545938 ip-10-0-134-7 kubenswrapper[2576]: E0726 19:04:59.545917    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8dc7ea3d0f032e039d7b85d53ae3529343754482a6c50b8dca01b0cb1f25cda2\": container with ID starting with 8dc7ea3d0f032e039d7b85d53ae3529343754482a6c50b8dca01b0cb1f25cda2 not found: ID does not exist" containerID="8dc7ea3d0f032e039d7b85d53ae3529343754482a6c50b8dca01b0cb1f25cda2"
Jul 26 19:04:59.545992 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:59.545947    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dc7ea3d0f032e039d7b85d53ae3529343754482a6c50b8dca01b0cb1f25cda2"} err="failed to get container status \"8dc7ea3d0f032e039d7b85d53ae3529343754482a6c50b8dca01b0cb1f25cda2\": rpc error: code = NotFound desc = could not find container \"8dc7ea3d0f032e039d7b85d53ae3529343754482a6c50b8dca01b0cb1f25cda2\": container with ID starting with 8dc7ea3d0f032e039d7b85d53ae3529343754482a6c50b8dca01b0cb1f25cda2 not found: ID does not exist"
Jul 26 19:04:59.559539 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:59.559502    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q"]
Jul 26 19:04:59.564946 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:04:59.564916    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-c75d5a-7887d9b65f-scp8q"]
Jul 26 19:05:00.848979 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:00.848943    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4eb8c250-24f0-46eb-b606-1f68c4f165ef" path="/var/lib/kubelet/pods/4eb8c250-24f0-46eb-b606-1f68c4f165ef/volumes"
Jul 26 19:05:01.296441 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:01.296399    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pqvlq"]
Jul 26 19:05:03.141295 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:03.141266    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:05:03.514863 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:03.514822    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:05:09.534806 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:09.534768    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:05:09.633583 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:09.633547    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:05:09.785689 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:09.785583    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"]
Jul 26 19:05:09.785878 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:09.785851    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq" podUID="075468d1-5f16-4760-bc00-999ee7927f2b" containerName="istio-proxy" containerID="cri-o://9ee138b6cb9088591eec4ae381cff8111881867f4a229fdd8daf3b72989641d3" gracePeriod=30
Jul 26 19:05:09.800992 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:09.800960    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:05:10.029458 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:10.029425    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:05:15.053422 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.053393    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 19:05:15.057931 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.057908    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/075468d1-5f16-4760-bc00-999ee7927f2b-istio-token\") pod \"075468d1-5f16-4760-bc00-999ee7927f2b\" (UID: \"075468d1-5f16-4760-bc00-999ee7927f2b\") "
Jul 26 19:05:15.058022 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.057957    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/075468d1-5f16-4760-bc00-999ee7927f2b-workload-socket\") pod \"075468d1-5f16-4760-bc00-999ee7927f2b\" (UID: \"075468d1-5f16-4760-bc00-999ee7927f2b\") "
Jul 26 19:05:15.058022 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.057999    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/075468d1-5f16-4760-bc00-999ee7927f2b-workload-certs\") pod \"075468d1-5f16-4760-bc00-999ee7927f2b\" (UID: \"075468d1-5f16-4760-bc00-999ee7927f2b\") "
Jul 26 19:05:15.058115 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.058043    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/075468d1-5f16-4760-bc00-999ee7927f2b-istio-podinfo\") pod \"075468d1-5f16-4760-bc00-999ee7927f2b\" (UID: \"075468d1-5f16-4760-bc00-999ee7927f2b\") "
Jul 26 19:05:15.058115 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.058088    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/075468d1-5f16-4760-bc00-999ee7927f2b-istio-envoy\") pod \"075468d1-5f16-4760-bc00-999ee7927f2b\" (UID: \"075468d1-5f16-4760-bc00-999ee7927f2b\") "
Jul 26 19:05:15.058229 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.058118    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/075468d1-5f16-4760-bc00-999ee7927f2b-credential-socket\") pod \"075468d1-5f16-4760-bc00-999ee7927f2b\" (UID: \"075468d1-5f16-4760-bc00-999ee7927f2b\") "
Jul 26 19:05:15.058229 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.058153    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/075468d1-5f16-4760-bc00-999ee7927f2b-istiod-ca-cert\") pod \"075468d1-5f16-4760-bc00-999ee7927f2b\" (UID: \"075468d1-5f16-4760-bc00-999ee7927f2b\") "
Jul 26 19:05:15.058229 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.058180    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qn9jz\" (UniqueName: \"kubernetes.io/projected/075468d1-5f16-4760-bc00-999ee7927f2b-kube-api-access-qn9jz\") pod \"075468d1-5f16-4760-bc00-999ee7927f2b\" (UID: \"075468d1-5f16-4760-bc00-999ee7927f2b\") "
Jul 26 19:05:15.058229 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.058208    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/075468d1-5f16-4760-bc00-999ee7927f2b-istio-data\") pod \"075468d1-5f16-4760-bc00-999ee7927f2b\" (UID: \"075468d1-5f16-4760-bc00-999ee7927f2b\") "
Jul 26 19:05:15.058436 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.058332    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/075468d1-5f16-4760-bc00-999ee7927f2b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "075468d1-5f16-4760-bc00-999ee7927f2b" (UID: "075468d1-5f16-4760-bc00-999ee7927f2b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 19:05:15.058436 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.058341    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/075468d1-5f16-4760-bc00-999ee7927f2b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "075468d1-5f16-4760-bc00-999ee7927f2b" (UID: "075468d1-5f16-4760-bc00-999ee7927f2b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 19:05:15.058557 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.058532    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/075468d1-5f16-4760-bc00-999ee7927f2b-workload-socket\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:05:15.058617 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.058561    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/075468d1-5f16-4760-bc00-999ee7927f2b-workload-certs\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:05:15.058696 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.058675    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/075468d1-5f16-4760-bc00-999ee7927f2b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "075468d1-5f16-4760-bc00-999ee7927f2b" (UID: "075468d1-5f16-4760-bc00-999ee7927f2b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 19:05:15.058809 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.058783    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/075468d1-5f16-4760-bc00-999ee7927f2b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "075468d1-5f16-4760-bc00-999ee7927f2b" (UID: "075468d1-5f16-4760-bc00-999ee7927f2b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 19:05:15.059252 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.059225    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/075468d1-5f16-4760-bc00-999ee7927f2b-istio-data" (OuterVolumeSpecName: "istio-data") pod "075468d1-5f16-4760-bc00-999ee7927f2b" (UID: "075468d1-5f16-4760-bc00-999ee7927f2b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 19:05:15.060249 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.060222    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/075468d1-5f16-4760-bc00-999ee7927f2b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "075468d1-5f16-4760-bc00-999ee7927f2b" (UID: "075468d1-5f16-4760-bc00-999ee7927f2b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 26 19:05:15.060350 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.060308    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/075468d1-5f16-4760-bc00-999ee7927f2b-istio-token" (OuterVolumeSpecName: "istio-token") pod "075468d1-5f16-4760-bc00-999ee7927f2b" (UID: "075468d1-5f16-4760-bc00-999ee7927f2b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 19:05:15.060350 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.060320    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/075468d1-5f16-4760-bc00-999ee7927f2b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "075468d1-5f16-4760-bc00-999ee7927f2b" (UID: "075468d1-5f16-4760-bc00-999ee7927f2b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 19:05:15.060485 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.060463    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/075468d1-5f16-4760-bc00-999ee7927f2b-kube-api-access-qn9jz" (OuterVolumeSpecName: "kube-api-access-qn9jz") pod "075468d1-5f16-4760-bc00-999ee7927f2b" (UID: "075468d1-5f16-4760-bc00-999ee7927f2b"). InnerVolumeSpecName "kube-api-access-qn9jz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 19:05:15.159074 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.159039    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/075468d1-5f16-4760-bc00-999ee7927f2b-istio-podinfo\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:05:15.159074 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.159071    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/075468d1-5f16-4760-bc00-999ee7927f2b-istio-envoy\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:05:15.159074 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.159080    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/075468d1-5f16-4760-bc00-999ee7927f2b-credential-socket\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:05:15.159074 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.159088    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/075468d1-5f16-4760-bc00-999ee7927f2b-istiod-ca-cert\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:05:15.159354 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.159098    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qn9jz\" (UniqueName: \"kubernetes.io/projected/075468d1-5f16-4760-bc00-999ee7927f2b-kube-api-access-qn9jz\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:05:15.159354 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.159107    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/075468d1-5f16-4760-bc00-999ee7927f2b-istio-data\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:05:15.159354 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.159115    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/075468d1-5f16-4760-bc00-999ee7927f2b-istio-token\") on node \"ip-10-0-134-7.ec2.internal\" DevicePath \"\""
Jul 26 19:05:15.616621 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.616587    2576 generic.go:356] "Generic (PLEG): container finished" podID="075468d1-5f16-4760-bc00-999ee7927f2b" containerID="9ee138b6cb9088591eec4ae381cff8111881867f4a229fdd8daf3b72989641d3" exitCode=0
Jul 26 19:05:15.616819 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.616626    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq" event={"ID":"075468d1-5f16-4760-bc00-999ee7927f2b","Type":"ContainerDied","Data":"9ee138b6cb9088591eec4ae381cff8111881867f4a229fdd8daf3b72989641d3"}
Jul 26 19:05:15.616819 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.616650    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"
Jul 26 19:05:15.616819 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.616670    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq" event={"ID":"075468d1-5f16-4760-bc00-999ee7927f2b","Type":"ContainerDied","Data":"3d6e81012b9961bb2065162f3d7e56b2acfff7c49dc963e7b254eb337ff868a6"}
Jul 26 19:05:15.616819 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.616691    2576 scope.go:117] "RemoveContainer" containerID="9ee138b6cb9088591eec4ae381cff8111881867f4a229fdd8daf3b72989641d3"
Jul 26 19:05:15.625322 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.625291    2576 scope.go:117] "RemoveContainer" containerID="f262c7a755a7b72229d6aa9b047e054e675f58d416f760fc9f1d81180f139108"
Jul 26 19:05:15.632959 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.632941    2576 scope.go:117] "RemoveContainer" containerID="9ee138b6cb9088591eec4ae381cff8111881867f4a229fdd8daf3b72989641d3"
Jul 26 19:05:15.633312 ip-10-0-134-7 kubenswrapper[2576]: E0726 19:05:15.633291    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ee138b6cb9088591eec4ae381cff8111881867f4a229fdd8daf3b72989641d3\": container with ID starting with 9ee138b6cb9088591eec4ae381cff8111881867f4a229fdd8daf3b72989641d3 not found: ID does not exist" containerID="9ee138b6cb9088591eec4ae381cff8111881867f4a229fdd8daf3b72989641d3"
Jul 26 19:05:15.633396 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.633319    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ee138b6cb9088591eec4ae381cff8111881867f4a229fdd8daf3b72989641d3"} err="failed to get container status \"9ee138b6cb9088591eec4ae381cff8111881867f4a229fdd8daf3b72989641d3\": rpc error: code = NotFound desc = could not find container \"9ee138b6cb9088591eec4ae381cff8111881867f4a229fdd8daf3b72989641d3\": container with ID starting with 9ee138b6cb9088591eec4ae381cff8111881867f4a229fdd8daf3b72989641d3 not found: ID does not exist"
Jul 26 19:05:15.633396 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.633336    2576 scope.go:117] "RemoveContainer" containerID="f262c7a755a7b72229d6aa9b047e054e675f58d416f760fc9f1d81180f139108"
Jul 26 19:05:15.633561 ip-10-0-134-7 kubenswrapper[2576]: E0726 19:05:15.633542    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f262c7a755a7b72229d6aa9b047e054e675f58d416f760fc9f1d81180f139108\": container with ID starting with f262c7a755a7b72229d6aa9b047e054e675f58d416f760fc9f1d81180f139108 not found: ID does not exist" containerID="f262c7a755a7b72229d6aa9b047e054e675f58d416f760fc9f1d81180f139108"
Jul 26 19:05:15.633605 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.633567    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f262c7a755a7b72229d6aa9b047e054e675f58d416f760fc9f1d81180f139108"} err="failed to get container status \"f262c7a755a7b72229d6aa9b047e054e675f58d416f760fc9f1d81180f139108\": rpc error: code = NotFound desc = could not find container \"f262c7a755a7b72229d6aa9b047e054e675f58d416f760fc9f1d81180f139108\": container with ID starting with f262c7a755a7b72229d6aa9b047e054e675f58d416f760fc9f1d81180f139108 not found: ID does not exist"
Jul 26 19:05:15.640647 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.640608    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"]
Jul 26 19:05:15.643318 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.643292    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-c75d5a-openshift-default-66774b9f76-v9smq"]
Jul 26 19:05:15.645011 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.644992    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-hwn8m_3128f681-a36c-4e7e-9c9b-a594b32e1567/manager/0.log"
Jul 26 19:05:15.757656 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.757627    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-xrbql_2473a76b-1a39-4732-b37c-f92386120068/manager/0.log"
Jul 26 19:05:15.994335 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:15.994301    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-hcvtf_4e6dd317-bb81-452c-a779-25aec4db179a/server/0.log"
Jul 26 19:05:16.105239 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:16.105197    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-ssbqj_591441e9-ec06-416d-b803-d6688584f504/manager/0.log"
Jul 26 19:05:16.210203 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:16.210173    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-758747dbfc-2l78q_204bb694-4d92-4d85-b6d0-863fe7b98ca0/copy-manifests/0.log"
Jul 26 19:05:16.222138 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:16.222105    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-758747dbfc-2l78q_204bb694-4d92-4d85-b6d0-863fe7b98ca0/manager/0.log"
Jul 26 19:05:16.849023 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:16.848993    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="075468d1-5f16-4760-bc00-999ee7927f2b" path="/var/lib/kubelet/pods/075468d1-5f16-4760-bc00-999ee7927f2b/volumes"
Jul 26 19:05:16.924257 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:16.924223    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29751510-9xzn5_85ebe06d-9b91-4aae-bcc6-5967e656c5f3/cleanup/0.log"
Jul 26 19:05:17.143102 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:17.143020    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29751540-khhq6_542390b1-5679-4ae7-9520-cd4e847bb932/cleanup/0.log"
Jul 26 19:05:19.172239 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:19.172209    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-pqvlq_31bcb2be-8928-438b-b59f-11414eff93ac/limitador/0.log"
Jul 26 19:05:19.285674 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:19.285638    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-867cs_137b7c20-6067-4a91-a05d-50ee376714f5/manager/0.log"
Jul 26 19:05:19.732435 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:19.732403    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-d92ps_a4aab415-9a5c-457c-b5af-18a0ef16ab42/discovery/0.log"
Jul 26 19:05:19.949619 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:19.949588    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c7c67d4f6-wg28w_b2611181-ba1a-488b-be78-c6f4459823e5/kube-auth-proxy/0.log"
Jul 26 19:05:21.061861 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:21.061827    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqd6c2_994d0da6-95aa-48c0-94a3-50e3d0ade5a3/main/0.log"
Jul 26 19:05:26.898741 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:26.898706    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:05:28.001799 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:28.001738    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log" limit=1024
Jul 26 19:05:29.607489 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:29.607455    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-fsm57_54acf4cd-cd75-4990-9ccf-274ef789daa5/global-pull-secret-syncer/0.log"
Jul 26 19:05:29.752163 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:29.752133    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6vnjh_65820b5a-b897-45fc-ae26-e286b5444375/konnectivity-agent/0.log"
Jul 26 19:05:29.862499 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:29.862417    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-7.ec2.internal_4c3d4ae719d8eb92a6e5cea24a51f094/haproxy/0.log"
Jul 26 19:05:35.693587 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:35.693552    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-pqvlq_31bcb2be-8928-438b-b59f-11414eff93ac/limitador/0.log"
Jul 26 19:05:35.799293 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:35.799255    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-867cs_137b7c20-6067-4a91-a05d-50ee376714f5/manager/0.log"
Jul 26 19:05:37.856981 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:37.856952    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cdmwl_cfc21e49-9040-4bbe-8a78-a4717cfe3e92/node-exporter/0.log"
Jul 26 19:05:37.886574 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:37.886542    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cdmwl_cfc21e49-9040-4bbe-8a78-a4717cfe3e92/kube-rbac-proxy/0.log"
Jul 26 19:05:37.910618 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:37.910591    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cdmwl_cfc21e49-9040-4bbe-8a78-a4717cfe3e92/init-textfile/0.log"
Jul 26 19:05:39.485290 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.485241    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fjlfd/perf-node-gather-daemonset-xhqj7"]
Jul 26 19:05:39.485812 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.485724    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08ff5eed-7b25-49c1-b9c2-898d9a3a284a" containerName="maas-api"
Jul 26 19:05:39.485812 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.485762    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="08ff5eed-7b25-49c1-b9c2-898d9a3a284a" containerName="maas-api"
Jul 26 19:05:39.485812 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.485787    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b31eb70c-d65a-4166-97ee-c90b3911eb79" containerName="istio-proxy"
Jul 26 19:05:39.485812 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.485796    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b31eb70c-d65a-4166-97ee-c90b3911eb79" containerName="istio-proxy"
Jul 26 19:05:39.485812 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.485806    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b31eb70c-d65a-4166-97ee-c90b3911eb79" containerName="istio-proxy"
Jul 26 19:05:39.485812 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.485814    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b31eb70c-d65a-4166-97ee-c90b3911eb79" containerName="istio-proxy"
Jul 26 19:05:39.486140 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.485825    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7dc0999-26b8-4201-8230-713d58ef3b16" containerName="istio-proxy"
Jul 26 19:05:39.486140 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.485835    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7dc0999-26b8-4201-8230-713d58ef3b16" containerName="istio-proxy"
Jul 26 19:05:39.486140 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.485844    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7dc0999-26b8-4201-8230-713d58ef3b16" containerName="istio-proxy"
Jul 26 19:05:39.486140 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.485854    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7dc0999-26b8-4201-8230-713d58ef3b16" containerName="istio-proxy"
Jul 26 19:05:39.486140 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.485862    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="075468d1-5f16-4760-bc00-999ee7927f2b" containerName="istio-proxy"
Jul 26 19:05:39.486140 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.485870    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="075468d1-5f16-4760-bc00-999ee7927f2b" containerName="istio-proxy"
Jul 26 19:05:39.486140 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.485882    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="075468d1-5f16-4760-bc00-999ee7927f2b" containerName="istio-proxy"
Jul 26 19:05:39.486140 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.485891    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="075468d1-5f16-4760-bc00-999ee7927f2b" containerName="istio-proxy"
Jul 26 19:05:39.486140 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.485899    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db2b53c2-9568-4b09-bc4a-d484802e4901" containerName="maas-api"
Jul 26 19:05:39.486140 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.485908    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="db2b53c2-9568-4b09-bc4a-d484802e4901" containerName="maas-api"
Jul 26 19:05:39.486140 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.485925    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75fb92d2-3709-4227-a0a4-b2c84ddcea8d" containerName="maas-api"
Jul 26 19:05:39.486140 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.485933    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="75fb92d2-3709-4227-a0a4-b2c84ddcea8d" containerName="maas-api"
Jul 26 19:05:39.486140 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.485943    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e536ead-0b97-4f86-9384-6e97a9d14a84" containerName="revoke-keys"
Jul 26 19:05:39.486140 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.485952    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e536ead-0b97-4f86-9384-6e97a9d14a84" containerName="revoke-keys"
Jul 26 19:05:39.486140 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.485962    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4eb8c250-24f0-46eb-b606-1f68c4f165ef" containerName="maas-api"
Jul 26 19:05:39.486140 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.485971    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4eb8c250-24f0-46eb-b606-1f68c4f165ef" containerName="maas-api"
Jul 26 19:05:39.486140 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.486054    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="075468d1-5f16-4760-bc00-999ee7927f2b" containerName="istio-proxy"
Jul 26 19:05:39.486140 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.486066    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7dc0999-26b8-4201-8230-713d58ef3b16" containerName="istio-proxy"
Jul 26 19:05:39.486140 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.486076    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7dc0999-26b8-4201-8230-713d58ef3b16" containerName="istio-proxy"
Jul 26 19:05:39.486140 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.486086    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="08ff5eed-7b25-49c1-b9c2-898d9a3a284a" containerName="maas-api"
Jul 26 19:05:39.486140 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.486095    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="db2b53c2-9568-4b09-bc4a-d484802e4901" containerName="maas-api"
Jul 26 19:05:39.486140 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.486104    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="75fb92d2-3709-4227-a0a4-b2c84ddcea8d" containerName="maas-api"
Jul 26 19:05:39.486140 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.486112    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4eb8c250-24f0-46eb-b606-1f68c4f165ef" containerName="maas-api"
Jul 26 19:05:39.486140 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.486120    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e536ead-0b97-4f86-9384-6e97a9d14a84" containerName="revoke-keys"
Jul 26 19:05:39.486140 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.486129    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b31eb70c-d65a-4166-97ee-c90b3911eb79" containerName="istio-proxy"
Jul 26 19:05:39.486140 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.486137    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b31eb70c-d65a-4166-97ee-c90b3911eb79" containerName="istio-proxy"
Jul 26 19:05:39.489236 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.489215    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-xhqj7"
Jul 26 19:05:39.492021 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.491988    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fjlfd\"/\"kube-root-ca.crt\""
Jul 26 19:05:39.492153 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.491988    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fjlfd\"/\"openshift-service-ca.crt\""
Jul 26 19:05:39.493174 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.493154    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-fjlfd\"/\"default-dockercfg-shpr9\""
Jul 26 19:05:39.498524 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.498498    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fjlfd/perf-node-gather-daemonset-xhqj7"]
Jul 26 19:05:39.557877 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.557835    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8e37d748-b8c0-48e4-8d76-0be8c66bcd48-lib-modules\") pod \"perf-node-gather-daemonset-xhqj7\" (UID: \"8e37d748-b8c0-48e4-8d76-0be8c66bcd48\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-xhqj7"
Jul 26 19:05:39.558129 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.557897    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e37d748-b8c0-48e4-8d76-0be8c66bcd48-sys\") pod \"perf-node-gather-daemonset-xhqj7\" (UID: \"8e37d748-b8c0-48e4-8d76-0be8c66bcd48\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-xhqj7"
Jul 26 19:05:39.558129 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.557920    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8e37d748-b8c0-48e4-8d76-0be8c66bcd48-podres\") pod \"perf-node-gather-daemonset-xhqj7\" (UID: \"8e37d748-b8c0-48e4-8d76-0be8c66bcd48\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-xhqj7"
Jul 26 19:05:39.558129 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.557998    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6kw7\" (UniqueName: \"kubernetes.io/projected/8e37d748-b8c0-48e4-8d76-0be8c66bcd48-kube-api-access-m6kw7\") pod \"perf-node-gather-daemonset-xhqj7\" (UID: \"8e37d748-b8c0-48e4-8d76-0be8c66bcd48\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-xhqj7"
Jul 26 19:05:39.558129 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.558059    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8e37d748-b8c0-48e4-8d76-0be8c66bcd48-proc\") pod \"perf-node-gather-daemonset-xhqj7\" (UID: \"8e37d748-b8c0-48e4-8d76-0be8c66bcd48\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-xhqj7"
Jul 26 19:05:39.658808 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.658725    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e37d748-b8c0-48e4-8d76-0be8c66bcd48-sys\") pod \"perf-node-gather-daemonset-xhqj7\" (UID: \"8e37d748-b8c0-48e4-8d76-0be8c66bcd48\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-xhqj7"
Jul 26 19:05:39.659020 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.658886    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e37d748-b8c0-48e4-8d76-0be8c66bcd48-sys\") pod \"perf-node-gather-daemonset-xhqj7\" (UID: \"8e37d748-b8c0-48e4-8d76-0be8c66bcd48\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-xhqj7"
Jul 26 19:05:39.659020 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.658894    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8e37d748-b8c0-48e4-8d76-0be8c66bcd48-podres\") pod \"perf-node-gather-daemonset-xhqj7\" (UID: \"8e37d748-b8c0-48e4-8d76-0be8c66bcd48\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-xhqj7"
Jul 26 19:05:39.659020 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.658991    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8e37d748-b8c0-48e4-8d76-0be8c66bcd48-podres\") pod \"perf-node-gather-daemonset-xhqj7\" (UID: \"8e37d748-b8c0-48e4-8d76-0be8c66bcd48\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-xhqj7"
Jul 26 19:05:39.659169 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.658987    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6kw7\" (UniqueName: \"kubernetes.io/projected/8e37d748-b8c0-48e4-8d76-0be8c66bcd48-kube-api-access-m6kw7\") pod \"perf-node-gather-daemonset-xhqj7\" (UID: \"8e37d748-b8c0-48e4-8d76-0be8c66bcd48\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-xhqj7"
Jul 26 19:05:39.659169 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.659102    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8e37d748-b8c0-48e4-8d76-0be8c66bcd48-proc\") pod \"perf-node-gather-daemonset-xhqj7\" (UID: \"8e37d748-b8c0-48e4-8d76-0be8c66bcd48\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-xhqj7"
Jul 26 19:05:39.659169 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.659131    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8e37d748-b8c0-48e4-8d76-0be8c66bcd48-lib-modules\") pod \"perf-node-gather-daemonset-xhqj7\" (UID: \"8e37d748-b8c0-48e4-8d76-0be8c66bcd48\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-xhqj7"
Jul 26 19:05:39.659304 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.659214    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8e37d748-b8c0-48e4-8d76-0be8c66bcd48-proc\") pod \"perf-node-gather-daemonset-xhqj7\" (UID: \"8e37d748-b8c0-48e4-8d76-0be8c66bcd48\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-xhqj7"
Jul 26 19:05:39.659304 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.659247    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8e37d748-b8c0-48e4-8d76-0be8c66bcd48-lib-modules\") pod \"perf-node-gather-daemonset-xhqj7\" (UID: \"8e37d748-b8c0-48e4-8d76-0be8c66bcd48\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-xhqj7"
Jul 26 19:05:39.669558 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.669520    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6kw7\" (UniqueName: \"kubernetes.io/projected/8e37d748-b8c0-48e4-8d76-0be8c66bcd48-kube-api-access-m6kw7\") pod \"perf-node-gather-daemonset-xhqj7\" (UID: \"8e37d748-b8c0-48e4-8d76-0be8c66bcd48\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-xhqj7"
Jul 26 19:05:39.800325 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.800232    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-xhqj7"
Jul 26 19:05:39.936730 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:39.936702    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fjlfd/perf-node-gather-daemonset-xhqj7"]
Jul 26 19:05:39.939424 ip-10-0-134-7 kubenswrapper[2576]: W0726 19:05:39.939396    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8e37d748_b8c0_48e4_8d76_0be8c66bcd48.slice/crio-44b127d5aa6de1654fd4a9638f606d199ceb9c08119bf2e7496af75acbdf5fe8 WatchSource:0}: Error finding container 44b127d5aa6de1654fd4a9638f606d199ceb9c08119bf2e7496af75acbdf5fe8: Status 404 returned error can't find the container with id 44b127d5aa6de1654fd4a9638f606d199ceb9c08119bf2e7496af75acbdf5fe8
Jul 26 19:05:40.727436 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:40.727396    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-xhqj7" event={"ID":"8e37d748-b8c0-48e4-8d76-0be8c66bcd48","Type":"ContainerStarted","Data":"53ff317e3a30f18433f2e6614a40c3d572b13fb81f0a8ca0b8fa3065cfd713c6"}
Jul 26 19:05:40.727981 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:40.727450    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-xhqj7" event={"ID":"8e37d748-b8c0-48e4-8d76-0be8c66bcd48","Type":"ContainerStarted","Data":"44b127d5aa6de1654fd4a9638f606d199ceb9c08119bf2e7496af75acbdf5fe8"}
Jul 26 19:05:40.727981 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:40.727527    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-xhqj7"
Jul 26 19:05:40.744431 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:40.744376    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-xhqj7" podStartSLOduration=1.744360487 podStartE2EDuration="1.744360487s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 19:05:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 19:05:40.743786267 +0000 UTC m=+3226.451193326" watchObservedRunningTime="2026-07-26 19:05:40.744360487 +0000 UTC m=+3226.451767546"
Jul 26 19:05:42.677005 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:42.676972    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jjkbn_265041ab-6c9e-4b4b-834b-812bb4c3040e/dns/0.log"
Jul 26 19:05:42.699188 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:42.699158    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jjkbn_265041ab-6c9e-4b4b-834b-812bb4c3040e/kube-rbac-proxy/0.log"
Jul 26 19:05:42.856762 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:42.856727    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hz9mx_7e895b2b-026e-4228-9fc7-fbe90089f5a9/dns-node-resolver/0.log"
Jul 26 19:05:43.447482 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:43.447448    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6964ccb4d8-t226q_d2839e3a-08fe-4fdd-9e83-792cab573778/registry/0.log"
Jul 26 19:05:43.544069 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:43.544019    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-k9khs_261aba58-ddd5-4f9e-af39-7475dec519f9/node-ca/0.log"
Jul 26 19:05:44.694776 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:44.694725    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-d92ps_a4aab415-9a5c-457c-b5af-18a0ef16ab42/discovery/0.log"
Jul 26 19:05:44.739173 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:44.739146    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c7c67d4f6-wg28w_b2611181-ba1a-488b-be78-c6f4459823e5/kube-auth-proxy/0.log"
Jul 26 19:05:45.837683 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:45.837652    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zwspk_9a3a2c2f-54f6-46c5-8cd3-6f343cba9713/serve-healthcheck-canary/0.log"
Jul 26 19:05:46.402818 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:46.402785    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7glwq_9efa7a18-6a50-4d18-834a-8a50301009c2/kube-rbac-proxy/0.log"
Jul 26 19:05:46.431231 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:46.431204    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7glwq_9efa7a18-6a50-4d18-834a-8a50301009c2/exporter/0.log"
Jul 26 19:05:46.458285 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:46.458251    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7glwq_9efa7a18-6a50-4d18-834a-8a50301009c2/extractor/0.log"
Jul 26 19:05:46.741270 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:46.741240    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-xhqj7"
Jul 26 19:05:48.927789 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:48.927762    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-hwn8m_3128f681-a36c-4e7e-9c9b-a594b32e1567/manager/0.log"
Jul 26 19:05:49.152547 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:49.152509    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-xrbql_2473a76b-1a39-4732-b37c-f92386120068/manager/0.log"
Jul 26 19:05:49.430800 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:49.430774    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-hcvtf_4e6dd317-bb81-452c-a779-25aec4db179a/server/0.log"
Jul 26 19:05:49.508826 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:49.508778    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-ssbqj_591441e9-ec06-416d-b803-d6688584f504/manager/0.log"
Jul 26 19:05:49.594578 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:49.594544    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-758747dbfc-2l78q_204bb694-4d92-4d85-b6d0-863fe7b98ca0/manager/0.log"
Jul 26 19:05:49.614450 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:49.614372    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-758747dbfc-2l78q_204bb694-4d92-4d85-b6d0-863fe7b98ca0/copy-manifests/0.log"
Jul 26 19:05:50.048104 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:50.048071    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-d92ps_a4aab415-9a5c-457c-b5af-18a0ef16ab42/discovery/0.log"
Jul 26 19:05:50.079163 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:50.079133    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c7c67d4f6-wg28w_b2611181-ba1a-488b-be78-c6f4459823e5/kube-auth-proxy/0.log"
Jul 26 19:05:51.771287 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:51.771249    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-d92ps_a4aab415-9a5c-457c-b5af-18a0ef16ab42/discovery/0.log"
Jul 26 19:05:51.799830 ip-10-0-134-7 kubenswrapper[2576]: I0726 19:05:51.799799    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c7c67d4f6-wg28w_b2611181-ba1a-488b-be78-c6f4459823e5/kube-auth-proxy/0.log"